Threat to Internal Validity

An event occurring between pretest and posttest that could affect the participants which in turn could impact the dependent variable.

Control:  Designate a control group that receives no treatment

Threat to Internal validity

Changes occurring in subjects due to passage of time and not as a result of treatment.

Control:  Use a control group that receives no treatment.

Threat to internal validity

Subjects taking the test, learn how to do better or improve the next time they take the test.

Control:  Use a control group that does not receive pre-testing.

Threat to internal validity

Instruments used to collect data are not accurate/precise, procedures are not standardized or calibrated.

Control:  Use well-designed instruments.

Threat to internal validity

Subjects are selected in a non-random manner

Control: * recruit volunteers, then randomly assign them to groups

* Match subjects on selected characteristics, then randomly assign them to groups

* Pretest subjects on measures of the dependent variable to make sure there are no pretreatment differences between groups

Threat to internal validity

The use of intact groups that vary in some element of maturity.

Control:  Pretesting subjects/groups on maturity level

Threat to internal validity

Subjects score extremely high or extremely low.  Scores regress to the mean.

Control:  Study design to follow a random sample rather than those with the highest or lowest scores.

Threat to internal validity

Subjects do not complete the study.  They may: drop out, withdraw, die or otherwise can not be located.

Threat can not be completely elimintated.

Control:  *The use of oversampling and using large group sizes

*Using incentives

*Obtaining good demographic information about study subjects

*Maintaining good records 

Threat to internal validity

The behavior/attitude of the subject is alterded due to the effects of the study

Control: *Provide the control group with some type of special treatment that is comparable to the experimental group but would not have a direct impact on the dependent variable

*Prevent subjects from knowing they are in a study.  This may be considered unethical.

Threat to internal validity

Subjects expectations affect the results. 

Control: Both the control group and treatment groups receive the same information or as little information as possible.

Threat to internal validity

Treatment of experimental group spills over to the other group(s)

Control: *Use a control group that does not have contact with the experimental group.

*Request subjects to keep treatment information confidential until the end of the study period.

Threat to internal validity

Differences in the location of treatment/program/data collection affects subjects responses.

Control:  Make locations the same for all subjects or minimize as much as possible differences.

Threat to internal validity

Someone responsible for implementing the treatment/program introduces inequalities or bias into the study.

Control:  Make sure all persons responsible for implementing the program are equally trained and competent.

Threat to external validity

How well the results of the study can be generalized beyond the groups involved in the study.

Control: Using larger group sizes and random sampling of participants allows for more accurate generalization.

Threat to external validity

The extent to which the environmental conditions can be duplicated in other settings.

Control:  To the extent possible, duplicate as closely as possible the original setting.

Threat to external validity

The researcher tries to generalize findings to past and future situations.

Control: *Timing of the experiment should be considered when generalizing findings.

*  Duplicate setting and historical factors as nearly as possible when replicating a program with a different population.

Study results can be generalized to other groups and settings beyond those in the current study.

Results with external validity can be of more use to the profession.