Summary: Events that occur between the pre and posttests, besides the independent variable.

What should the researcher do: Use a control group that would not have access to anything that could misconstrue the results.

Summary: Changes that occur because of the passing of time.

What the researcher can do: Use a control group so that the two groups can be compared.

Summary: When a pretest effects the results of the posttest.

What the researcher can do: Using a control group that does not get any pretesting to compare the noncontrol group to.

Summary: Instruments are not ideal for the study, either they are not up to standards or are not the right tool.

What the researcher should do: Using well designed instruments such as accurately formed questions.

Summary: When participants are not chosen in a random manner.

What the researcher should do: Assigning participants to groups, rather than letting them select one. Or giving a pretest and then form groups based on different results.

Summary: Using preformed groups that vary in maturity.

What the researcher should do: Pretest or prescreen participants on levels of maturity to keep them from combining.

Summary: Extremely high or low scores that regress toward the mean.

What the researcher can do: Do not place students in high or low groups after the pretest. Randomly place them in groups before testing.

Summary: Participants quit the study or cannot be located.

What the researcher can do: Use larger groups. Provide incentives to stay in the study.

Summary: Altered behavior when the participant knows he/she is being observed.

What the researcher can do: Do not let participants know which group is being observed or provide the control group with a special treatment that will not make their results too misconstrued.

Summary: Altered behaviors because of believed changes.

What the researcher can do: Give the same information to both groups and provide them with as little information as possible.

Summary: Experimental group treatment spills over into the control group.

What the researcher should do: Use groups from two different locations and express the concern for not sharing information.

Summary: The location of where the study occurs.

What the researcher should do: Use the same location for all participants.

Summary: The differences that occur when different people present the program.

What the researcher should do: Do not allow the program developer present the program and ensure that the presenter has a neutral view.

Summary: How general are the results from the study? Can they be applied to other groups?

What can the researcher do: Randomly selecting groups from very different backgrounds so the results of the study do not apply only to one population.

Summary: When the environment or setting of the study could cause differences in results.

What the researcher should do: Make sure that numerous settings are involved when forming the survey.

Summary: When the researcher tries to apply study results to the past or future.

What the researcher can do: Use the results only in the situations that they were created for and for the time period that they were created for.