ANOVA Equation

O       Xij = m + (mj-m) + eij = m + tj + eij

o       Xij = the score of Person i in group j.

o       m = population mean

o       mj = group mean

o       Xij = 80 + (75-80) = 80 – 5

o       (mj-m) = tj = the difference between the group mean and the population mean.

o       eij = Xij – uj = the different between person i and the group mean j.

Scope of ANOVA (kinds of effects)

O       compare group mean differences for various categorical IVs as long as sample size allows (cells are filled). main effects, interaction effects and simple effects.

Main effects

O       IVs independent of other IVs

Interaction effects

O       IVs dependent on other IVs

Simple Effects

O       IVs at one level of another IV – you need to compare across simple effects to get the interaction effect. you isolate the professors and compare male and female ignoring grad students. Like dropping a variable without collapsing, this is used to see if interaction is significant and to probe interaction. Have to have one of two sets for a two way.

Techniques Within ANOVA

O       (1) Independent Samples t- test – simplified case for one two levels of one IV. (2) One-way ANOVA – two or more levels of one IV. (3) Factorial ANOVA – two or more levels or two or more IVs (4) ANCOVA – any of the above controlling for one or more IVs (5) Multivariate ANOVA – any of the above except 4 with two or more DVs. (6) Multivariate ANCOVA – the case of 4 with 2 or more DVs.

Logic of ANOVA

O       normality and homogeneity assumptions make sure the distributions are the same in every way except the mean. distributions are creating using only mean (hopefully different) and standard deviation (same). ANOVA essentially tests if groups are drawn from the same population distributions or not.

The F statistic

O       F-ratio or F-test. For between subjects F=MS(between)/MS(within). More general F=MS(effect)/MS(error). Both num and demon provide estimates of population variance. The presence or absence of an effect is determined by comparing the two estimates.

The Error Term (denominator) in ANOVA (between)

O       denominator of F-statistic. Each group’s own variance is an estimate of the population variance, best est. is average of variances. If the sample sizes are not equal, a weighted mean (by df) of the s^2 values is calculated. This represents how different members of the same group are from one another. This is an estimate of the population variance that does not depend on Ho being true or false.

The Group Effect (numerator) in ANOVA (between)

O       numerator of F-statistic. if Ho is true, then the group means are each estimates of the population variance. The variance of the sampling distribution of means: S(M)^2 – variance of the scores/n: sigma^2(e)/n. sigma^2(e) = n*S(M)^2 =MS(treat), MS(group), MS(effect), MS(between). Represents how different members of different groups are from each other.

Two Estimates of Error Variance One Way ANOVA

O       MS(error) does not depend on the truth of Ho

O       MS(effect) depends on Ho being true

O       If the estimates agree, they support the conclusion that Ho is true and vise versa. If they are significantly different we conclude that group differences must have contributed to the differences between the MS(effect) making it larger. The groups are different

Conceptual Basis for One-Way ANOVA

O       All ANOVA analyses have the same underlying conceptual basis – compares two variances, individual differences and group differences. Statistic = difference between means/measure of random variability. Compared to the critical values for the sampling distributions of the statistic to test Ho.

The F statistic

O       A ratio that compares differences between groups to differences within groups. 1=No difference, <1=you screwed up (always retain null hypothesis), 1.5 = differences between groups 1.5 greater that differences within groups, 5.9 = there is a big difference.

Derivation of the equation for variance

O       We start with a score (x). We want to know how far the score is from other scores, so we use the mean as a reference (x-M). We want to know step two for ever score (S(x-M)). This will lead to zero for an answer so (S(x-M)^2). Now we want this score as an average ((S(x-M)^2)/n). Outliers and scores pull in the mean so the deviation is too small. This makes the null hypothesis possible and mathematical simulations show that the sample will be closer to the population if you subtract one from n. ((S(x-M)^2)/(n-1)).

Derivation of the equation for comparing group means

O       Same logic to compare multiple group means. (1) Numerator = MS(b/t) = (S(M-GM)^2/(k-1))*n. n=sample size for each group, GM=grand mean=SM/K, k=number of groups. (2) Denominator = MS(w/i) = s1^2+s2^2+s3^2/k. (3) F=MS(b/t)/MS(w/i).

Hypothesis Testing Examining Effects

O       designs, especially complex designs, allow for many effects/hypothesis. 3-way (7 effects= 3 main + 3 2-way INT + 1 3-way INT. Multiple simple effects to prove any of the four interaction effects. Designs are conducted to test hypotheses.

Planned Analysis vs. Omnibus Analysis

O       planned – easier, faster; omnibus – controls power/Type I error. Traditional approach is to conduct the omnibus analysis, and to follow that analysis with multiple comparison procedures if, and only if the omnibus analysis is significant. (i.e. the default strategy is that only significant interactions get probed).

Simple Effects Naming

O       When you name simple effects use the variable not the constant. (i.e. Gender effect for individualist, gender effect for collectivist). IV 1 Gender (M/F) & IV 2 Societal Structure (individualist, collectivist).

Familywise Error Rates

O       Type I (alpha) error rates over all effects for a set of analyses. (a) error rate per comparison (PC) alpha = alpha’ (b)familywise error rate (FW) alpha=1-(1-alpha)^c [c=# of comparisons]. (c) the limits on FW; PC<=FW<=C. In most reasonable cases, FW is close to alpha*c. But is not really additive because if more than 20 comparisons, alpha would be over 100!

F-test is intrinsically a 1-tailed test

O       the distribution of F is not normal, the statistic measures the magnitude of difference using variances. Therefore, F can only be positive. Hypotheses are not one or two tailed in the sense we have been discussing them. Has a region of reject on one-tail but this is NOT what we consider a one-tailed alternative hypothesis.                                                                                                              

Degrees of Freedom

O       the number of means that are theoretically free to vary before one has to be fixed to achieve the grand mean.

N vs. n

O       N= total number  of participants in study, n = group sample size

Significance Testing ANOVA (between)

O       two types of df are needed. (1) df(b/t) = k -1 (# of groups -1) – all but one group mean can vary (from GM). (2) df(w/i) = N-k (total number of participants in study – number of groups) [aka df(error)] – all but one person in each group can vary. (3) Select alpha and find the critical value for F. If calculated value is less that critical value fail to reject Ho. Retain Ho automatically if F<1.

Effect sizes

O       note: effect size calculations are often saved for primary comparisons of individual means. R^2=SS(b/t)/SS(total). aka eta^2. Indicates the proportion of the variance accounted for by the IVs. Cohen’s conventions for R^2: small (.01), medium (.06), large (.14).

Extension to Unequal Sample Sizes – Examining Sources of Deviations

O       (1) X-GM – yields measure of total variance. (2) Xj-mj – w/I groups variance –unexplained variance. (3) mj-GM – yields b/t-groups variance – explained variance. (4) This approach reveals a general method of calculation that handles unequal group sample sizes more easily.

Calculations (b/t) Groups ANOVA

O       (1) Deviation of scores from the Grand Mean; SS(total) = S(X-GM)^2 = SS(w/i) + SS(b/t). (2) Deviation of group means from grand mean; SS(b/t) =S(M-GM)^2; MS(b/t) = SS(b/t)/df(b/t)[k-1]. (3) Deviation of scores from their group means; SS(w/i) = S(X-M)^2; MS(w/i) = SS(w/i)/df(w/i)[k-1] F=MS(b/t)/MS(w/i). Using unequal sample size results in a modification in which deviations are weighted by individual sample sizes (via df).

Multiple Comparisons

O       comparing the means of the individual groups (a) planned comparisons – conduct focused analysis using the best estimate for population variance (i.e. the omnibus error term). (b) post hoc tests – many options exist depending on the exact situation and extent to which you would like to control the familywise error rate.

Fisher LSD

O       very liberal multiple comparison technique – less preferred because leads to type I error more often. Requires that omnibus F is significant – effective in preventing most Type I errors. Most effective with only a few groups (2 or 3). FW increases unacceptably with more groups/comparisons. Uses omnibus MS(error) in place of pooled variance in the standard t-test calc. Increase you df in error, increase power. more powerful than standard t-tests. MS(error)=weighted averages of all w/I group variance. Used for planned comparisons.

Bonferroni Procedure

O       A simple method derived from Fished LSD which controls the familywise error rate so it is never greater that .05. calculate values of t using same procedure as LSD. Decide on # of comparisons, c; c does not always equal # of groups k. Used for planned comparisons.

(Student) – Newman – Keuls test

O       sort’s means into homogeneous subsets – means w/i ea. subset are not different, but are different from other subsets. Makes comparisons b/t most discrepant means first and more in from there – stops as soon as it finds no different. Uses more stringent df & DV fore means further apart. Last test is most powerful lest extreme C.V. because df go to denominator. Treats it like less & less effects after each comparison. However, familywise error rates climb with great # of groups (like LSD) – anticipates the amount of effect. Used for omnibus comparisons.

Tukey HSD

O       Derived from Newman – Keuls. Also creates homogenous subsets of means. More conservative in that the C.V. for all comparisons is = to the C.V. for the most discrepant comparison. This controls the FW error rate better than Newman-Keuls. Used for omnibus comparisons.

One-Way ANOVA Conclusions

O       There was at least one different in the means. Post hoc comparisons indicated the mean of one group were sig shorter than for another group. No other group differences were found. Implications?

Factorial ANOVA

O       Largest advantage = assessment of interaction effects. Interaction – the effect of the IV on the DV changes depending on the level of the other IV. Interaction effects often give the most information. They allow us to see when, where for whom things happen (moderators)

Factorial ANOVA Equation

O       One way–Xij = m+(mj - m)+eij = m+tj+eij. Two way –Xijk = m+ai+bj+abij+eij (a) Xijk = any observation (person k, level i (A), & level j (B). (b) m=the grand mean (c) ai = the effect for Factor Ai = mAi -m (d) bj = the effect of Factor Bj = mBj-m. (e) abij = interaction effect of Ai and Bj = m-mAi - mBj +mij (f) eijk = the unit of error associate with observation Xijk. The most detailed information we have for prediction is the cell mean = mij.

Interpreting Factorial ANOVA results

O       interaction effects trump main effects. main effects compare marginal means, interactions compare cell means. Main effects = look at the midpoints of the lines or the average of the two endpoints of the lines. Interaction effects = are the lines parallel.

Types of Interactions

O       First we see that the patterns of effects differ – shown in figures by the lines not being parallel (they have different slopes). (1) ordinal interactions – lines are not parallel, but they do not cross. (2) disordinal interactions – lines are not parallel & group differences reverse their sign t some level of the other IV – the lines cross.

Null Hypothesis Factorial ANOVA

O       Ho à Factor 1 – Ho: m1=m2=m3; Factor 2 – Ho:m1=m2=m3; Int – Ho: no interaction between factor 1 & 2.

Assumptions of Factorial ANOVA

O       Same as 1-way ANOVA. (1) Homogeneity of Variance (2) Normality (3) Intendance of Observations

Degrees of Freedom (for a 2-way ANOVA)

O       Factor 1: df=k1-1; Factor 2: df=k2-1; int: df=df(f1)*df(f2); total: df=N-1; error: df=df(total)- (sum of df of effects) = N – (k1*k2).

Conclusions for a 2-way ANOVA

O       interpret the variable in the highest order effect. interaction before main effect.

Simple Effects

O       comparison of all the levels of one IV at only one level of another IV. Four possible simple effects in a 2-way. (1) IV1 L1 over IV2 L1 and L2 (2) IV1 L2 over IV2 L1 and L2 (3) IV2 L1 over IV1 L1 and L2 (4) IV2 L2 over IV1 L1 and L2. Use the two appropriate for your research question. Two make a set. need to calculate a new F-value using the selected MS(group) and the omnibus MS(error). Conclusions IV1 does not affect DV w/ differing levels of IV2 or IV1 does affect DV w/ differing levels of IV2.

Within-groups ANOVA General Research Questions

O       Does knowledge of p’s level of some categorical variable that indicates when a continuous variable was completed provide information about their scores on the continuous variable. Do the same participants differ on the same measures over time

Mixed ANOVA General Research Questions

O       Do the same p’s differ on the same measures over time based on their assignment to same between-groups condition?

Factorial (w/I and mixed) Hypotheses

O       The null and alternative hypotheses remain the same conceptually from the between groups ANOVA case. There are no differences between levels of IV (no main effects). There are no interactions between any of the IVS (w/I x w/i) (b/t x b/t) (w/I x b/t).

Factorial (w/I and mixed) Assumptions

O       (1) Normality – the DV is normally distributed at each time point. (2) Homogeneity of Variance – the variance of the DV are equal at ea time point. (3) individual variances – covariance matricies are the same at all conditions  -- the covariances are equl for each level of the condition. (4) Sphericity – standard errors between pairs of means at different times are constant – people have to be as similar to themselves at other people are. (5) Equal Sample sizes/full factorial assumption – each p’s must complete the depdent measure at each level of each IV – that is, at all time points and for all conditions (except for mixed factorial case). (6) The DV administered at each time point must be equivalent in terms of scaling and construct – or at least comparable. (7) Indepdendence of observations – we violate this assumption.

Why we violate Indepdence of observations for the W/I and Mixed Factorial ANOVAs

O       using the same p’s in more than one condition results in this violation. This would be a problem, but it is reserved by the ability to partiion the depdence. This results in the main advantage of w/I groups designs. The overall variabliility is reduced by using a common participant pool for all conditions (i.e. the same people). This increases power.

Scenarios that use W/I subjects factors

O       there are three scenarios (1) repeated measures – p’s are compared to themselves [most common] (2) related samples – p’s are compared to naturally related others. (3) matched samples – p’s are compared to empirically identified others

Techniques that use w/I subjects factors

O       (1) dependent samples t-test – simplified case for only two levels of one within group variable (2) one-way w/i group ANOVA – 2 or more levels of one IV (w/i) (3) factorial w/I group ANOVA – 2 or more levels of two or more w/I IVs (4) Mixed Factorial ANOVA – 2 or more levels of 2 or more IVs at least 1 of which is w/I and 1 of which is b/t – covariate and multivariate variations are also available.

Scope of the entire ANOVA approach

O       group mean differences may be examined for any number o categorical IVs – may be b/t, w/i or any combination.

Effects Examined by W/I & Mixed Cases

O       main effects, interactions, and simple effects

Underlying Structural Models Compared

O       (a) one-way b/t – the score is equal to the difference between the score and the grand mean and sample mean and error. (b) two-way b/t – looking at main effects, interactions, and error (c) one-way w/I groups à Xij=m + Pi +Ptij + eij; Pi = info about person i (consistency); tj =time taken; Ptj=interaction; eij=error; m=population mean

Underlying Structural Model for Complex Cases

O       the structural model for more complex cases expands in the same way – add more terms to add more info (main effects, interactions, covariates, pearson constants, adjusted and addition error terms). Terms that add no additional info go to zero and essentially drop from the equation for multivariate more than one equation will be created.

Partition the SS w/i groups ANOVA – Start w/ total variation

O       total variation à b/t & w/i; b/tàwe can account w/ person constant; w/i à between time point (dif b/t time 1 and time 2) & error (unexplained). b/t time points – the similarity to each other over time (can be explained)

Partition the df w/i groups ANOVA – start w/ total df

O       total df à kn-1 à SSb/t = n-1 & SSw/i = n(k-1); SSw/iàb/t treatment= k-1 & e = n-1

Calculating df – 1 way repeated measures

O       df(total) = kn-1; df(b/t) = n-1; df(w/i) = n(k-1); b/t treatment df = k-1; df(error) = (n-1)(k-1)

Multiple Comparisons W/i groups Variable

O       to test two different levels of a w/I groups variable. Following a significant main effect (or to test a priori hypothesis). Conduct an ANOVA in which you specify only the two levels to be compared. Use MS(effect) from new ANOVA. Use the MS(error) from the omnibus ANOVA. Use bonferroni correction based on c.

Simple Effects for w/I groups variables

O       interactions among w/I groups factors are probed using the same basic strategies as before. Simple effects are effects across the levels of one IV at only one level of other IV. new ANOVAs are preformed to provide MS values for an IVs effect at each level of the other IV. These MS(effect) values are divided by the error term from the omnibus analysis interaction term.

Mixed Factorial Designs

O       Assesses the effects of the IVs of both b/t groups and w/I groups types. Assesses all possible main effects and interactions. Mixed interactions get partitioned into the w/I groups effects. Multiple error terms are involved in assessing b/t and w/I group effects.

Partition of the Sums of Squares (2-way Mixed Factorial ANOVA)

O       total variation à between participants and within participants; between participants à group variance and error variance (ind diff); within participants à time + time x group + time x Ps w/I groups error (how stable a person is over time)

Partition of the df Mixed Factorial ANOVA

O       main effects df = k-1 for each; interactions df = products of df for the relevant IVs; other df calculated most easily by subtraction à df w/I ps = df(total) – df(b/tps); df ps w/I groups = df(b/tps) – df(groups); df(time x ps w/I groups) = df(w/ips)-df(time)-df(t x g).

Calculating df Mixed Factorial ANOVA

O       df(total)=n data points -1; df(b/tps) = N -1; df(groups) = k-1; df (ps w/I groups) = k(k-1);

Probing Mixed Factorial Interactions

O       the simple effects procedures used previously are just extended. Omnibus error term from original NOVA error for all 3 variables. Begin by identifying one variable to hold constant and then conduct a two way for other two variables. Find interactions between other two variables. If one is significant and the other is not they will have different patterns.

Further Probing of Mixed Factorial Interactions

O       continue to probe in the same way as needed. the most complete probe is conducting all possible comparisons while holding all but one IV constant.

Attrition

O       Cell size decreases over time due to participants dropping out