Validity

Validity

Validity means accuracy. Results are valid if they accurately show what they are intended to show (.e.g did you measure what you wanted to? Can you generalise the results?).

You need to know about internal and external validity.

(1) Internal validity: Refers to whether or not the research measured what it intended to measure (e.g. the effects of the IV on the DV) To work out whether a piece of research has high internal validity, ask yourself:

• Were EV’s controlled? (Yes)
• Were there any CV’s? (No)
• Did the research actually measure the effect of the IV on the DV? (Yes)
• If you find the answers above then the research has high internal validity this is good because you can establish cause and effect between the IV and the DV. Highly controlled pieces of research have high internal validity.

Ways of Measuring/Assessing Internal Validity

1. Face validity: Whether a measure appears, at face value, to test what it claims to. For example, does an interview about addiction to alcohol genuinely measure drinking habits or does it simply elicit socially desirable responses? If it includes questions that trigger socially desirable responses, it is likely to have low internal validity.

2. Concurrent validity: Whether a new test produces a similar measure of a variable as existing tests of the same phenomenon. A new questionnaire that identifies risk factors in drug abuse should find many of the same risk factors as an existing, well-known questionnaire if it has high internal validity.

3. Predictive validity: Whether the measure can accurately forecast future consequences. For example, a test designed to identify risk factors for alcoholism could be followed up and if validity is high, those identified as having higher risk factors will be more likely to exhibit signs of alcohol abuse.

It is important when conducting research that internal validity is high and that the researchers can be happy that the IV is the only variable affecting the DV (when this is the case, a cause and effect relationship can be established). In order to improve internal validity, researchers adopt a number of different methods.

Strategies to Improve (Increase) Internal Validity:

(1) Standardised Instructions  a set of instructions/script that is followed by a researcher when carrying out a study. These instructions/script indicate to the experimenter how to welcome the participant, how to introduce the study, how to conduct the study and how to end the study/thank the participants. This script ensures that all trials of the researcher are conducted in exactly the same way for each participant.

What EV does this strategy overcome?

• Experimenter Effects  due to the fact that the experimenter is following a script, they are less likely to lead the participants to behave in a specific way.
• Situational Variable  can also avoid situational variables as usually standardised instructions indicate the experimenter exactly how the research environment should be set up (e.g. temperature of the room, resources etc ) this ensures that the environment is consistent for each participant.

(2) Double-Blind Technique  when the participant is unaware of the true aim of the study that they are taking part in. Further, the key researcher employs a research assistant to carry out the study who is also unaware of the true sims/nature of the experiment that they are conducting.

What EV does this strategy overcome?

• Demand Charcteristics  participants are less likely to change their behaviour if the true aim of the study is not communicated to them until the end of the research. This leads to participants display more accurate behaviour.
• Experimenter Effects  if the research assistant is unaware of the true aims of the research, they will be unable to suggest to the participant how they wish them to behave/less likely to influence participant behaviour.

(3) Single-Blind Technique  when the participant taking part in the study is unaware of the true aims of the research that they are taking part in.

What EV does this strategy overcome?

• Demand Charcteristics  participants are less likely to change their behaviour if the true aim of the study is not communicated to them until the end of the research. This leads to participants display more accurate behaviour.

(4) Automation  when the instructions of an experiment are recorded and are played to the participants (as oppose to receiving instructions directly from the researcher).

What EV does this strategy overcome?

• Experimenter Effects  due to the fact that the researcher doesn’t come into contact with the participant (because instructions are given through a pre-recorded tape), the experimenter will be unable to suggest to the participant how they wish them to behave/less likely to influence participant behaviour.

(5) Experimental Designs can also be used to increase internal-validity.

• Independent Measures Design  when participants take part in just one condition in a piece of research. This helps to overcome demand characteristics as participants taking part in only one condition are unlikely to guess the aim of the research and change their behaviour. Also overcomes order effects, participants only taking part in one condition are less likely to become practiced or bored.

• Repeated Measures Design  when participants take place in every condition in a piece of research. This method would overcome participant variables, because the same participant takes part in each condition there is consistency in relation to participant characteristics (gender, age etc ). This means that (due to the fact participant variables are being controlled) the research is more likely to be measuring just the effects of the IV on the DV.

• Matched Pairs Design  when participants take part in just one condition in a piece of research however, participants in say condition 1, are matched with participants in condition 2 on a certain characteristic (e.g. IQ, age, gender etc ) This method would overcome participant variables, because the key participant characteristics are matched across the conditions  therefore there is consistency in relation to participant characteristics (gender, age etc ). his means that (due to the fact participant variables are being controlled) the research is more likely to be measuring just the effects of the IV on the DV.

(6) Counter-Balancing- Used when a repeated measures design has been used in research in order to avoid order effects. Experimenters fear that when a repeated measures design is used, the result of a piece of research runs the risk of being bias. For example, take an experiment with 2 conditions (condition A and B), if participants complete condition A first followed by condition B last, researchers say that performance in condition A is usually reflective of the participants real-life behaviour. In condition B however, because the participants have already completed part of the experiment, it is possible that their behaviour may change in one of two ways;

• Order Effects  they may become practiced at the task (get better at the research task that they have been asked to do) which could cause participant performance to become unnaturally inflated (therefore not measuring true behaviour).
• Order Effects  the participant may become bored in the second condition and may not try/give the experiment their true attention which again would mean that condition B is measuring unnatural behaviour.

If a researcher asks that participants always complete condition A followed by B, participant performance may always be unnatural in condition B (i.e. in a memory test, participant performance might look better in condition B because they have been practicing remembering things or may become worse because after completing the memory test in condition A, they now can’t be bothered to complete the memory test in condition B). This will lead to bias results and the researcher isn’t accurately measuring what they are intending to measure. Adopting the counter-balancing method involved half of the participants completing condition A first followed by B, and the other half of the participants completing condition B first followed by B. This means that if there are any order effects, this negative effect will be spread across both conditions (rather just one condition usually B) which means that the research will have measured more accurately what it intends to measure. Use the phrase ABBA to help you remember this method (50% of participants complete conditions AB, 50% of participants complete conditions BA).

(7) Random Allocation  when participants are randomly allocated to either condition A or B. This is done to fairly distribute participant variables. This overcomes participant variables, ensuring that their is an even split of participant characteristics balanced across all the conditions in the research.

External Validity

External validity: Refers to whether the research can be generalised outside the research setting to;

* other settings (ecological)

* other people (population)

* other times (temporal) To work out whether a piece of research has high external validity, ask yourself; Has the research been done in a natural setting? (Yes) Is the sample of participants representative of the entire target population? (Yes) Is the time in which the research was conducted reflective of other periods in time (i.e. is there anything socially significant occurring at the time?) (Yes) If you find the answers above then the research has high external validity this is good because you can generalise your findings beyond the research setting, sample and time.

Examples of External Validity:

1. If your experiment uses only men, yet is suppose to represent the whole population (both men and women) then it may be said to have low population validity

2. If you carry out your first experiment in a classroom and find the same results when you repeat it in the canteen your experiment can be said to have high ecological validity

3. If you decide to replicate an experiment that was conducted in 1963 but you find very different results then the original experiment can be said to have low temporal validity.

Improving External Validity

As well as controlling extraneous variables to improve internal validity, it is also possible to improve the external validity of your research.

(1) Improving population validity  researchers need to make sure that they have conducted their research on a wide, representative sample. The sample in the research needs to include all groups in the target population.

(2) Improving ecological validity  researchers need to make sure that they conduct research in a natural, non-artificial environment.

(3) Improving temporal validity  researchers need to make sure that they conducted their studies repeatedly across different times in order to ensure that the results are reflective of the current time period.

Assessing External Validity

External validity can be assessed by:

(1) Replication in real-life settings: To test if the results of a laboratory experiment can be generalised to the real-world, the same methodology can be employed in a real-life setting (e.g. Milgram replicated his obedience study in a run-down office). If similar results are achieved the research can be said to have high ecological validity (a type of external validity).

(2) Replication with different populations: To test if the results of a study on one sample of the population can be generalised to the rest of the population, the same methodology can be employed using a different sample (e.g. by studying a different cultural or subcultural group). If similar results are achieved the research can be said to have high population validity.

(3) Replication in the modern day: To test if results from an old study can be generalised to the modern day, the same methodology can be employed in the modern day (e.g. replicating Milgram’s 1960s study today). If similar results are achieved the research can be said to have high temporal validity.