true score variance

total variance

Test construction

Test administration

Test scoring/interpretation

Other

Good for traits, not for emotional states

Bad--practice effects, memory changes, fatigue, motivation, you know

Coefficient of Equivalents

Used with alternative or parallel forms

An estimate of inter-item consistency

BAD for heterogeneous tests and speed tests

used to adjust split half reliability

good to learn how many items to add to improve reliability or estimate internal consistency

measures different factors, content and topics

MMPI

Kuder-Richardson 20

Stat of choice for determining inter-item consistency

Good for dichotomous items (right or wrong)

Cronbach's

Most widely used for estimating internal consistency

Can be used with nondichotomous or Likerts

.9 or higher = redundancy

All indices of reliability provide and index THAT IS?

Characteristic of a particular group of test scores,

NOT the test

the tests:

Implications

How many tests being used in a situation

The purpose of the reliability

Homo!

Use test retest with hetero

Does test-retest work with dynamic or static characteristics?

11th grade Act effects of 1st year grades in grad school

Only get scores of people who make it to 11th grade and then again to grad school

Compares a testaker to an objective or standard

Traditional reliability is not appropriate for this type of test

Need to take a sample out of the universe of items

Representative

Provide a measure of the precision of an observed test score

HIGHER=lower reliability

Mean plus or minus error

ex. poll is accurate plus or minus 3%

Scores can change from one administration to the other based on stuff other than error

Use this stat to see if it is significantly diff

A process of gathering and evaluating evidence of validity

Da Big Tres

trinitarian view

Feels big 3 is incomplete, he wants to include

societal values

consequences

how the test is being used

Developed a method for giving courts evidence a test is work related

Panel of experts to rate items

essential, useful or not necessary

CONTENT

Content validity ratio

=ne - n/2

N/ 2

Positive is good

How well a test score can be used to infer and individuals probably standing on some measure of interest (criterion)

RELIABILE

RELEVANT

VALID

UNCONTAMINATED

the relationship between test scores and the criterion measured at the same time

good for depression

3 things to look at when doing validity

Appropriate sample

Attrition rates

Appropriate criterion

the likelihood a testaker will score w/in some interval on the criterion

chi squared

Tell the difference in average criterion scores for selected v original

Construct developed to describe or explain behavior

Construct validity--means high and low scores behave as predicted

Factor Analysis

Stat program thing that classes people or things together

Leniency

Severity

Central tendency

Halo effect=me

Test Conceptualization

Test Construction

Test Tryout

Item Analysis

Test Revision

Use known masters and known novices to do preliminary research

Nominal, Ordinal, Interval, Ratio

Age scale

Grade scale

Stanine Scale

Sum up responses

Good for Likert

Horses or Oranges?

MMPI-2

Predetermined categories already named

Weak

Medium

Strong

Very

Short answer, fill in the blank, essay

Credit toward placement in a class or category

DSM checklist for Depression

5-10 people per item

Use target population

P = # of students who got ? right

Total test takers

s= sqrt of (p * (1-p)

IVI= (s*rc)

(s*rt)

rt=correlation btwn item score and total test score

d = [u-L/N]

how well an item separates high from low scores

neg # = bad