Internal Validity

• A study that does a good job controlling for the possible effects on the independent variable is said to have strong internal validity.
• Exists if the observed effects of the independent variable on the dependent variable are real and not caused by extraneous factors

• Extraneous factors that allow for alternative explanations as to what caused a given effect on the dependent variable

History Effect

(threat to internal validitiy)

• Threat: An event occuring between pretest and postest other than the independent variable that could affect the dependent variable
• Solution: Use a control group: exactly like the treatment group in every way possible except they recieve no treatment. Then compare the control group to the experimental group (the group receiving treatment).

Maturation

(threat to internal validitiy)

• Threat: changes that are seen in subjects becuase of the time that has elapsed since the study began and that may not be the result of any program effects
• Solution: A control group would eliminate a threat to internal validity. All members of the control group and the experimental group would be aging at the same time. 

Testing

(Threats to internal validity)

• Threat: when taking a pretest somehow affects the taking of the posttest. Pretest is taken to find baseline measurement of a varibale prior to initiatiing a study. 
• Problem: By taking test participants may learn to do better on test next time. Differences on test may be due to the "testing-effect" and not due to the independent variable
• Solution: Use control group that has not received any pretesting to reduce threat to internal validity

Instrumentation

(Threat to Internal Validity)

• Threat: can cause threats to internval validity when measurments are not accurate or procedures are not standardized.
• Solution: Make sure instruments are well designed. Make sure individual(s) are well trained to use equipment and follow consistent procedures. Written instruments must acurately measure what they are supposed to measure (must be carefully designed to ask correct questions)

Selection Bias

(Threats to Internal Validity)

• Threat: Participants selected in a non-random manner differ in some way
• Solution: The use of intact groups and volunteers increases the likelihood that experimental and control groups are not the same. Recruit random volunteers and randomly assign them to groups (do not let volunteers chose their own groups). Pretest groups on measure of the dependent variable to make sure there are no pretreatment differences between groups is another control that can be used.

Selection Maturation Effect

(Threats to Internal Validity)

• Threat: Using intact groups that vary in some element of maturity (example: one group is older or more emotionally mature)
• Solution: Pretesting and/or prescreening groups on maturity levels is a way to avoid this threat to internal validity

Statistical Regression

(Threats to Internal Validity)

• Threat: When extremely high or extremely low scores regress toward mean
• Solution: Use a random sample representing subjects with a wide range of knowledge so subjects are not placed in high scoring or low scoring groups.

Mortality/Attrition

(Threats to Internal Validity)

• Threat: Participants dropout of the study or cannot be located
• Solution: (1) Oversampling and using large group sizes may help. (2) Using incentives to encourage participants to stay. (3) Obtain good demographic information at the beginning of the study so you can determine if group makeup has changed at the end of the study. This can help determine the impact of mortality on the research.

Hawthorne Effect

(Threats to Internal Validity)

Placebo Effect

(Threats to Internal Validity)

• Threat: Altered behavior because of expectations
• Solution: Blind study - measures taken so that participants do not know if they are taking real drug or placbo. Double blind study when the researcher also does not know who is taking real drug and placebo drug. Make sure members of both the control and experimental group are recieving the same information so both groups have similar expectations. You could also give both groups as little infromation as possible so they do not have any expectations. 

Diffusion

(Threats to Internal Validity)

• Threat: treatment of experimental group spills over to comparison or control groups
• Soultion: Use control group in differnet location than experimental group. Experimental group and control grpup will have similar attributes. If subjects are only available in one location than it is important to discuss confidentiality with participants. 

Location

(Threats to Internal Validity)

• Threat: the location of program or data collection affects participant responses.
• Soultion: If it is possible make the location for all participants in study. If this is not possible to use one location make sure that there are minimal location differences (so there is similar comfort levels for all subjects)

Implementation

(Threat to Internal Validity)

• Threat: differences in persons presenting a program affect the program
• Solution: (1) Make sure all persons responsible for implementing the program are equally trained and competent and are following standardized protocol for implementation. (2) whenever possible have someone other than the program developer present the program (make sure presenter is neutral).

• The ability to generalize study results to other groups and  setting beyond those in current experiments

Selection Treatment Interaction

(Threat to External Validity)

• Threat: The ability of the a researcher to generalize the results of a study beyond the groups involved in the study. In general, research sophistication and difficulty increaseas one desires to generalize to larger groups
• Solution: If you want to be able to generalize more you have to use a better sample for example if you want to represent 6th graders in New York instead of sampling 6th graders from one school district you might select 6th graders from school districts across the state. This would take a lot more sophisitication in the study.

Setting Treatment Interaction

(Threats to External Validity)

• Threat: The extent to which the environmental conditions or setting under which an experimental study was conducted can be duplicated in other settings.
• Solution: Make sure subjects come from similar places (whether it be knowledge, family, socioeconomic status, career aspirations).

History Treatment Interaction

(Threat to External Validity)

• Theat: develops when the researcher tries to generalize findings to past, present and future situations.
• Solution: Test subjects at similar time (for example: Test all subjects during lunch breaks, test subjects during summer do not assume test results will be the same in different seasons or at different times of the day)

• External validity cannot exist without internal validity. If the results of an experiment are not inernally valid, there isnothign to generalize. Researchers should alwaysbe concerned about ensuring internal validity before tyring to establish external validity.