Term
What is a time series design? What does it look like? 

Definition
 A series of observations made on the same variable consecutively over time. O O O O O O O O O O X O O O O O O O O O O O 


Term
What are autocorrelations? 

Definition
 A characteristic of timeseries designs, when the value of one observation is related to the value of previous observations that may be one, two, or three lags away. Estimating this ______ usually requires a large number of observations, typically 100, to facilitate correct model identification.  Statistically, one needs enough observations to identify the model, and this identification depends on the amount of error in the data, any periodicity effects, the timing of the intervention within the series, and the number of lags to be modeled. Sometimes fewer observations are used to show how an abbreviated time series can rule out many more threats to validity than is possible in cases in which there are only a few pretest and posttest time points. 


Term
Threats to validity in a time series design 

Definition
Internal: History: Possibility that forces other than the treatment under investigation influenced the dependent variable (e.g., Detroit study of banning alcohol sales to pregnant women). Instrumentation: Change in admin. procedures can lead to a change in how records are kept (person who wants to make perf. look good will change bookkeeping procedures). Selection: If composition of experimental group changes abruptly at the time of the intervention (if treatment causes or even requires attrition from measurement framework; hence, effect might do to difference of P_{s} from the pre and posttreatment series). Statistical Conclusion Validity: Low power, violated test assumptions, and unreliability of measurement; this stuff weakens the causal logic of the design (for Detroit alcohol ban study, problem with lack of specificity at what point the intervention began and its diffusion through the units, pregnant women, exposed to the treatment). Construct Validity: Inadequate explication of constructs or confounding of constructs; when using archival records, consideration of observed results due to evaluation apprehension, demand characteristics, or some similar threat must be considered.
External: Disaggregation (investigating external validity via using available data on the background characteristics of units to stratify them into subgroups, say, males and females, or different age groups) can be problematic because if it is not done correctly it can lower statistical power (e.g., if last age category of archival research is "over 65" and one is interested in studying the so called "old old" (over 75), one cannot do so).



Term
Interpreting the results of a time series design 

Definition
1. A Change in form (level or intercept) 2. A change in slope of the series at the point of interruption (the treatment). 3. Weak or delayed effects (law banning sale of alcoholic bevs to pregnant women took effect 7 months later). 


Term
Common problems with time series designs 

Definition
1. Many treatments are implemented slowly and diffuse through a population, so that the treatment is better modeled as a gradually diffusing process rather than as occurring all at once. 2. Many effects occur with unpredictable time delays that differ among populations and over time. 3. Many data series are much shorter than the 100 observations recommended for statistical analysis. 4. Many archivists are difficult to locate or reluctant to release data. 5. Archived data might involve time intervals between each data point that are longer than one needs; some data may be missing or look suspicious; and there may be undocumented definitional shifts. 


Term
Regression Discontinuity; what is it? 

Definition
 An experiment in which units are assigned to conditions based on exceeding a cutoff on an assignment variable. 


Term

Definition
Any measure taken prior to treatment, in which the units scoring on one side of the cutoff are assigned to one condition and those on the other side to another. 


Term

Definition
 It may be chosen on substantive grounds (professional opinion about who needs a medical treatment).  Statistical power and the estimation of interactions are both facilitated if the cutoff is the mean of the distribution of assignment variable scores (not possible if P_{s }trickle into the study slowly over time). ... 


Term
Function ("functional form") 

Definition
The characteristics of the true relations among variables, represented graphically by the shape of the relations (e.g., is it a curve?) and represented statistically by a model that may include nonlinear terms (e.g., power and interactions) or other transformations. 


Term
Advantages & Disadvantages to Randomized Designs 

Definition
 It ensures that alternative causes are not confounded with a unit's treatment condition.  It reduces the plausibility of threats to validity by distributing them randomly over conditions.  It equates groups on the expected value of all variables at pretest, measured or not.  It allows the researcher to know and model the selection process correctly.  It allows computation of a valid estimate of error variance that is also orthogonal to treatment.                         Randomization does not prevent units from maturing or regressing.  Doesn't prevent events other than treatment from occurring after the study begins.  Pretests can still cause a testing effect.  Changes in instrumentation can still occur. 


Term

Definition
 When you are assigning subjects to conditions, make sure you know what your subject is (a person, team, animal, etc.), know your unit of analysis; do not say you are randomizing participants to assignment when you are randomizing at the wrong units (e.g., when assigning teams, make sure you are assigning participants at the team level not the individual level). 


Term

Definition
These designs use two or more IVs (called factors), each with at least two levels.



Term
Advantages and disadvantages of factorial designs 

Definition
 They often require fewer units  They allow testing combinations of treatments more easily.  They allow testing interactions.            More subjects (you have to have a larger sample)  Rapid statistical and analytical complications as the number of IVs increases. 


Term

Definition
 In a ____ design, each level of each factor is exposed to (crossed with) all levels of all other factors (e.g., educational experiment in which some students in ea. classroom are exposed to treatment and some to control, factor is ____ with classroom).  In a ____ design, some levels of one factor are not exposed to all levels of the other factors (e.g., classrooms receive treatment but no control condition, classrooms are _____ within treatment conditions). 


Term
The 3 principles of quasiexperimental designs 

Definition
1. Identify plausible effects to internal validity. 2. Primacy of control by design 3. Coherent Pattern Matching (aka coherence or pattern matching)



Term
One Shot Case Study Design 

Definition
X O This design obtains one posttest observation on a respondent who experienced a treatment, but there are neither control groups nor pretests. Internal: History, Maturation, Selection, and Mortality. External: Interaction of Selection and X. 


Term
How to improve on the oneshot case study.


Definition
Use multiple substantive posttests.



Term
One Group PretestPosttest Design 

Definition
O_{1} X O_{2} A single pretest observation is taken on a group of respondents, treatment then occurs, and a single posttest observation on the same measure follows.  Internal: History, Maturation, Testing, Instrumentation, possible regression problem, and interaction of selection and maturation.  External: Interaction of testing and x; interaction of selection and x; possible reactive arrangements.



Term
How do you fix problems with onegroup pretestposttest design?


Definition
Use a double pretest or use a nonequivalent dependent variable.



Term
Removed & Repeated Treatment Designs


Definition
O_{1} X O_{2} O_{3} X O_{4}
 This design adds a third group to the onegroup pretestposttest design and then removes the treatment before a final measure is made. Internal: Statistical conclusion validity; design may be unethical or correlated with measures of aggression, satisfaction, or performance; possible resentful demoralization and compensatory rivalry. O_{1} X O_{2} X O_{3 }X O_{4}  The introduction, removal, and reintroduction of a treatment over time to study how treatment and outcome covary over time.  Internal: cyclical maturation, construct validity, history .  External and statistical conclusion validity.



Term
How to improve removed and repeated treatment designs 

Definition
 Using repeated treatment design.  Better with transient effects, unobtrusive treatments, a long delay b/w initial treatment and its reintro, with no confounding temporal cycles of the treatment w/treatment's intro, removal, and reintro; good when reintros of treatment are frequent and randomly distributed across time, creating a randomized experiment to which time blocks are the unit of assignment.



Term
Posttest only design with nonequivalent groups


Definition
 Add a control group to the onegroup posttestonly design. Used if treatment begins before the researcher is consulted so that pretest observation are not available on the scale used at posttest. NR X O_{1}           NR X O_{2} Internal: Not really any unless differential testing effects occur (but it's rare); b/c you are not using pretest, it can entail large costs for detecting selection biases.



Term
How to improve upon static group comparison


Definition
Use an indepenndent pretest sample: NR O_{1} X O_{2}                NR O_{1 }X O_{3}



Term
Matching & Stratifying nonequivalent groups (including using cohorts as controls)


Definition
 Synonymous with blocking, sometimes more specific to imply blocks in which units are exactly equal (rather than just similar) on a matching variable.  Decreases odd of selection biases...  The process of creating homogenous groups of units in which each group has more units than there are experimental conditions. Problems with ______: Undermatching and concern for producing result that is further away from the right answer than if _____ hadn't been used at all.



Term

Definition
 A study that contrasts units with an outcome of interest to those without the outcome to identify retrospectively the predictors or causes of the outcome.  Internal: Difficulty of operationalization and selection of cases; definitions and measures may change over time (instrumentation), attrition, fallible sources such as memory or records (hence, difficulty of classification of whether P_{s }were exposed to the treatment). Improving study relates to fixing problems or preventing these possible problems from occurring in the study...



Term
Nonequivalent control group design (untreated control group design with dependent pretest and posttest samples) 

Definition
 Most common of all quasiexperiments; in the most basic form, uses a treatment group and an untreated comparison group, with both pretest and the joint use of a pretest and a comparison group makes it easier to examine certain threats to validity. Internal: Selection and selection will combine with other threatss additively and interactively: Selectionmaturation Selectioninstrumentation Selectionregression Selectionhistory 


Term
Ways to improve the nonequivalent control group design.


Definition
1. Using a "double pretest": O_{1} O_{2 }X O_{3} _{                } O_{1 }O_{2} O_{3} This allows the researcher to understand possible biases in the main treatment analysis, if "treatment effects" emerge in the analysis of O_{1}O_{2}, similar biases may exist in the analysis of O_{2}O_{3}. 2. Using swtiching replications: O_{1} X O_{2} O_{3} _{                    } O_{1} O_{2} X O_{3} Given the contextual differences between the first and second treatment, the second introduction of the treatment is a modified replication, probing both an internal and external validity issue of whether this new context changes the treatment effect. 3. Using a reveresed treatment control group O_{1} O_{2} X_{+} O_{3} _{                    } O_{1} O_{2} X _{} O_{3} X_{+} represents a treatment expected to produce an effect in one direction and X_{} represents treatement expected to produce an effect in opposite direction. This design can have special construct validity advantages. If treatment X_{+} improves scores and X_{} decreases scores of the comparison group, a statistical interaction should result suggesting a treatment effect. 

