Term
| Validity (basic definition) |
|
Definition
| Does the test measure what it's supposed to measure? |
|
|
Term
| Validity (little harder definition but basically the same) |
|
Definition
| degree to which all accumulated evidence supports the intended interpretation of test scores for the intended purpose |
|
|
Term
| What are the three types of validity? |
|
Definition
| content & criterion related validity as well as construct validity |
|
|
Term
| T/F: Reliability does not imply validity but a test must be reliable to be valid. |
|
Definition
|
|
Term
| Be familiar with Griggs vs. Duke Power case |
|
Definition
| Slides 7-8 on 9/22/11 powerpoint |
|
|
Term
| To make a test valid you need what? |
|
Definition
|
|
Term
| In 1985, AERA and NCME came up with another way of understanding validity... what did they introduce? |
|
Definition
| The three types of validity |
|
|
Term
| Content validity is also called...? |
|
Definition
| logical validity because it's based on logic not statistical analysis |
|
|
Term
|
Definition
| The appearance that a measure has validity, not that statistically sound |
|
|
Term
|
Definition
| Is the content of the test valid for the kind of test it is? |
|
|
Term
| Three-step process of content validity |
|
Definition
Step 1: What is the domain?
Step 2: What content matches the domain I want?
Step 3: What specific questions could I use to test this? |
|
|
Term
| Content related validity is influenced by what two things? |
|
Definition
| Construct underrepresentation (you don't cover all the bases, ex. a global math test with no geometry) and construct-irrelevant variance (items are irrelevant to your domain) |
|
|
Term
| Best word to remember with content validity.... |
|
Definition
|
|
Term
| Before things get too confusing, get to know Slide 19 on 9/22/11 slides. |
|
Definition
| It's a diagram that breaks down validity and what types goes with what, etc. |
|
|
Term
| What is criterion related validity? |
|
Definition
| The relationship between the test and a criterion the test should be related to |
|
|
Term
| Criterion related validity is based on a ....? |
|
Definition
| golden standard (the test is compared to a golden standard) |
|
|
Term
| The two babies of criterion related validity... |
|
Definition
| concurrent and predicitve validity |
|
|
Term
| Concurrent validity asks... |
|
Definition
| does the instrument relate to current criterion? |
|
|
Term
| Predictive validity asks... |
|
Definition
| does the instrument relate to future criterion? |
|
|
Term
| Concurrent is to job performance as predictive validity is to an SAT score. |
|
Definition
| Concurrent is used to compare job performance of new applicants to those already skilled in the field. Predictivve validity takes a number and makes a predicition (SAT could/can predict GPA in college) |
|
|
Term
|
Definition
| correlation b/w test and criterion established |
|
|
Term
| Big thing to remember about validity coefficient... |
|
Definition
| NO SET RULES. Aspirin could save one in five people from a heart attack. That's big enough of a reason to recommend aspirin. |
|
|
Term
| Study slides 28-35, 8 questions to ask about validity coefficient |
|
Definition
|
|
Term
|
Definition
| "What is love, curiosity, mental health?" can't observe constructs, not objective |
|
|
Term
|
Definition
| comparing test scores to other well-established tests that measure the same/similar construct |
|
|
Term
|
Definition
| correlate test scores with other, well established tests that assess different constructs |
|
|
Term
| There is a well established, highly developed set of guidelines that have to be met to write good test items, true or false? |
|
Definition
| False, there is no set guidelines for writing test questions. Type of items you can write depend on what you want to test. |
|
|
Term
| I want to see if my students either know the material or not, what type of questions should I use? |
|
Definition
| True/False, Multiple Choice |
|
|
Term
| I simply want to know about people's attitudes, opinions, or get inside their minds about stress, memory, life, etc. What type of test questions should I use? |
|
Definition
|
|
Term
| What are some guidelines for writing test questions? |
|
Definition
| Clearly define what you're measuring, generate an item pool, avoid really long items, appropriate reading level for test taker, avoid double barreled questions (2 ideas/questions in one question), make positvely and negatively worded items |
|
|
Term
| Acquiescnence response set |
|
Definition
| tendency to agree with most items (don't make all test questions positive because people will STOP READING so throw in negative questions to KEEP people PAYING attention) |
|
|
Term
|
Definition
| "yes or no," "true/false," simple, easy to make/administer/score BUT less reliable/precise, have to have a lot of items to be reliable |
|
|
Term
|
Definition
| aka multiple choice format, easy scoring and requires better understanding however, harder to create |
|
|
Term
|
Definition
| Some concepts can't just be "right" or "wrong." Scales from strongly disagree - strongly agree, etc. Really familiar/easy to use but scoring is difficult except when you get to negatively word items (reverse scores) |
|
|
Term
|
Definition
| "On a scale of 1-10,50,100,etc..." creation is easy if you have well defined end points. Scoring is easy but if the reader may not be interested in the questions. |
|
|
Term
|
Definition
| Provides categories of a trait/concept for participant to choose from; comprehensive but requires a yes/no endorsement |
|
|
Term
|
Definition
| Avoids pitfall of checklists; place statements in either "1" pile or "9" pile |
|
|
Term
| Which answer format is a 50/50 chance of getting the answer right or wrong? Also may encourage memorization. |
|
Definition
| Dichotomous format because you have yes or no or true or false, etc. May not need to understand underlying concepts. |
|
|
Term
| Which answer format involves distractors? |
|
Definition
| Polytomous because distractors are answers within the choices that aren't correct - best way to test understanding because if you know the material you should be able to weed out the correct answer. |
|
|
Term
| Which two item formats are used the most in school settings? |
|
Definition
| Dichotomous and polytomous |
|
|
Term
| When should you guess on a multiple choice test? |
|
Definition
| Only if you can eliminate distractors. If you can't, don't guess at all on a test that takes off for wrong answers like the SAT's. |
|
|
Term
| What kind of test format would be good in personality testing? |
|
Definition
| Checklist format because it can supply a list of adjective that a test taker could "check" what applies to them and their personality. |
|
|
Term
| Unconditional love was coined by _____ _____. |
|
Definition
|
|
Term
| Carl Rogers introduced that people who have an idea of who they are but it's not the ideal person you'd like to be then you will experience a lot of anxiety. |
|
Definition
| Actual self vs. ideal self, basis of Q sort test format |
|
|
Term
| Tests should have a mix between both easy level questions and harder level questions. |
|
Definition
| Easy questions boost confidence but harder ones make it easier to weed people out based on scores. Levels of test questions depend on type of test (medical school = harder test, mix of bother = general education) |
|
|
Term
|
Definition
| proportion of tops scorers and low scorers overall |
|
|
Term
|
Definition
| correlation b/w item (correct/incorrect) and total test score - lower the correlation the suckier the test item (should be above .3) |
|
|
Term
| Standardized administration |
|
Definition
| put garbage in and you'll get garbage out; go about a test in a crappy way you'll get crappy results. |
|
|
Term
|
Definition
| familiarity of examiner, rapport, race |
|
|
Term
|
Definition
| translating tests to another language may not produce a reliable/valid test (find their proficient language and test in that) |
|
|
Term
|
Definition
| tendency for test results to conform to examiner expecations and/or motivations |
|
|
Term
|
Definition
| interviewer/interviewee match each others level of activity |
|
|
Term
| A psychologist sets the tone, true or false? |
|
Definition
| Yes, they do. Good cop, bad cop, social facilitation |
|
|
Term
| An interviewer is acting all hyper and energetic. What is likely to happen due to reciprocity? |
|
Definition
| The interviewee will act hyper and energetic. The interviewee will mimick that of that person interviewing them. |
|
|
Term
| What are some good qualities to have as an interviewer? |
|
Definition
| warmth, openness, understanding |
|
|
Term
| Interviewers want to avoid... |
|
Definition
| judgemental and hostile statements |
|
|
Term
|
Definition
a question that makes the interviewee justify a statement or response;
I hate school. "what's wrong with school?" I can't stand my mother. "What happened with your mother?" |
|
|
Term
| True or false, advice from an interviewer is good. |
|
Definition
| FALSE, never give advice. Ask more about why they're making certain statements but DO NOT give advice. It creates a power hierarchy. |
|
|
Term
| A friend telling you that they hoped they passed a hard stat test, you're response of "Don't worry about it. I'm sure you did fine" is an example of what type of response? |
|
Definition
|
|
Term
| The best types of questions to ask in an interview... |
|
Definition
| open ended, make them talk. Close ended questions end with a "yes" or "no." |
|
|
Term
|
Definition
| starts with open ended questions, then response to confrontation |
|
|
Term
| Structured clinical interview |
|
Definition
| when the interviews asks the examinee to respond to pre-established items (helps cover items that may be missed, increases reliability, etc) |
|
|
Term
|
Definition
| extent of appearance impacts judgment of interviewer; better you look the better the interviewer receives you |
|
|
