Shared Flashcard Set

Details

External Validity versus Internal Validity
Threats against internal and external validity
20
Other
Graduate
11/10/2014

Additional Other Flashcards

 


 

Cards

Term

Internal Validity

Definition
  • A study that does a good job controlling for the possible effects on the independent variable is said to have strong internal validity.
  • Exists if the observed effects of the independent variable on the dependent variable are real and not caused by extraneous factors
Term
Threats to Inernal Validity
Definition
  • Extraneous factors that allow for alternative explanations as to what caused a given effect on the dependent variable
Term

History Effect

 

 

 

 

 

 

(threat to internal validitiy)

Definition
  • Threat: An event occuring between pretest and postest other than the independent variable that could affect the dependent variable
  • Solution: Use a control group: exactly like the treatment group in every way possible except they recieve no treatment. Then compare the control group to the experimental group (the group receiving treatment).
Term

Maturation







(threat to internal validitiy)

Definition
  • Threat: changes that are seen in subjects becuase of the time that has elapsed since the study began and that may not be the result of any program effects
  • Solution: A control group would eliminate a threat to internal validity. All members of the control group and the experimental group would be aging at the same time. 
Term

Testing




 



(Threats to internal validity)

Definition
  • Threat: when taking a pretest somehow affects the taking of the posttest. Pretest is taken to find baseline measurement of a varibale prior to initiatiing a study. 
  • Problem: By taking test participants may learn to do better on test next time. Differences on test may be due to the "testing-effect" and not due to the independent variable
  • Solution: Use control group that has not received any pretesting to reduce threat to internal validity
Term

Instrumentation






 

(Threat to Internal Validity)

Definition
  • Threat: can cause threats to internval validity when measurments are not accurate or procedures are not standardized.
  • Solution: Make sure instruments are well designed. Make sure individual(s) are well trained to use equipment and follow consistent procedures. Written instruments must acurately measure what they are supposed to measure (must be carefully designed to ask correct questions)
Term

Selection Bias

 

 

 

 

 

(Threats to Internal Validity)

Definition
  • Threat: Participants selected in a non-random manner differ in some way
  • Solution: The use of intact groups and volunteers increases the likelihood that experimental and control groups are not the same. Recruit random volunteers and randomly assign them to groups (do not let volunteers chose their own groups). Pretest groups on measure of the dependent variable to make sure there are no pretreatment differences between groups is another control that can be used.
Term

Selection Maturation Effect

 

 

 

 

 

(Threats to Internal Validity)

Definition
  • Threat: Using intact groups that vary in some element of maturity (example: one group is older or more emotionally mature)
  • Solution: Pretesting and/or prescreening groups on maturity levels is a way to avoid this threat to internal validity
Term

Statistical Regression

 

 

 

 

 

(Threats to Internal Validity)

Definition
  • Threat: When extremely high or extremely low scores regress toward mean
  • Solution: Use a random sample representing subjects with a wide range of knowledge so subjects are not placed in high scoring or low scoring groups.
Term

Mortality/Attrition

 

 

 

 

 

(Threats to Internal Validity)

Definition
  • Threat: Participants dropout of the study or cannot be located
  • Solution: (1) Oversampling and using large group sizes may help. (2) Using incentives to encourage participants to stay. (3) Obtain good demographic information at the beginning of the study so you can determine if group makeup has changed at the end of the study. This can help determine the impact of mortality on the research.
Term

Hawthorne Effect

 

 

 

 

 

(Threats to Internal Validity)

Definition
  • Threat: Altered behavior due to the effects of being studied and observed
  • Solution: (1) Control group should be provided with special treatment that is comparable to that of the experimental group but would not have a direct impact on the dependent variable.  Both groups will be recieving special attention any differences in the dependent variable would likely be from the intervention. (2) Keep participants from knowing they are taking part in a study or being observed (this may not be possible beecause of full disclosure requriements)
Term

Placebo Effect

 

 

 

 

 

(Threats to Internal Validity)

Definition
  • Threat: Altered behavior because of expectations
  • Solution: Blind study - measures taken so that participants do not know if they are taking real drug or placbo. Double blind study when the researcher also does not know who is taking real drug and placebo drug. Make sure members of both the control and experimental group are recieving the same information so both groups have similar expectations. You could also give both groups as little infromation as possible so they do not have any expectations. 
Term

Diffusion

 

 

 

 

 

(Threats to Internal Validity)

Definition
  • Threat: treatment of experimental group spills over to comparison or control groups
  • Soultion: Use control group in differnet location than experimental group. Experimental group and control grpup will have similar attributes. If subjects are only available in one location than it is important to discuss confidentiality with participants. 
Term

Location

 

 

 

 

 

(Threats to Internal Validity)

Definition
  • Threat: the location of program or data collection affects participant responses.
  • Soultion: If it is possible make the location for all participants in study. If this is not possible to use one location make sure that there are minimal location differences (so there is similar comfort levels for all subjects)
Term

Implementation

 

 

 

 

 

(Threat to Internal Validity)

Definition
  • Threat: differences in persons presenting a program affect the program
  • Solution: (1) Make sure all persons responsible for implementing the program are equally trained and competent and are following standardized protocol for implementation. (2) whenever possible have someone other than the program developer present the program (make sure presenter is neutral).
Term
External Validity
Definition
  • The ability to generalize study results to other groups and  setting beyond those in current experiments
Term

Selection Treatment Interaction

 

 

 

 

 

(Threat to External Validity)

Definition
  • Threat: The ability of the a researcher to generalize the results of a study beyond the groups involved in the study. In general, research sophistication and difficulty increaseas one desires to generalize to larger groups
  • Solution: If you want to be able to generalize more you have to use a better sample for example if you want to represent 6th graders in New York instead of sampling 6th graders from one school district you might select 6th graders from school districts across the state. This would take a lot more sophisitication in the study.
Term

Setting Treatment Interaction

 

 

 

 

 

(Threats to External Validity)

Definition
  • Threat: The extent to which the environmental conditions or setting under which an experimental study was conducted can be duplicated in other settings.
  • Solution: Make sure subjects come from similar places (whether it be knowledge, family, socioeconomic status, career aspirations).
Term

History Treatment Interaction

 

 

 

 

 

(Threat to External Validity)

Definition
  • Theat: develops when the researcher tries to generalize findings to past, present and future situations.
  • Solution: Test subjects at similar time (for example: Test all subjects during lunch breaks, test subjects during summer do not assume test results will be the same in different seasons or at different times of the day)
Term
External Validity in relation to Internal Validity
Definition
  • External validity cannot exist without internal validity. If the results of an experiment are not inernally valid, there isnothign to generalize. Researchers should alwaysbe concerned about ensuring internal validity before tyring to establish external validity.
Supporting users have an ad free experience!