Term
| Profile Analysis overview |
|
Definition
Looks at differences between index and subtest scores to determine strengths and weaknesses and pattern of them to gather info about an individual's ability.
This will lead to better treatment/intervention decisions. (hypothetically) |
|
|
Term
| Ways to look for differences in profile analysis |
|
Definition
Norm-based - scores compared to means (VCI>115; DS>12)
OR
individual performance (ipsative)
Differences b/t scores
VCI>PRI - strength in verbal |
|
|
Term
| What the test makers say about profile analysis |
|
Definition
An evaluation of performance in terms of patterns of composite and subtest scaled scores
Can be both:
-INTERindividual (normative)- compare child's score patterns to approp. normative reference group
- INTRAindividual (ipsative)- compare child's score patterns across subtests, composites. |
|
|
Term
| Hypotheses generated based on referral question may be either corroborated or refuted by: |
|
Definition
WISC/WAIS profile analysis in conjunction with:
- background info
- direct bx obs
- other evaluation results |
|
|
Term
| Process-Oriented Approach in Profile Analysis |
|
Definition
Integrate WISC/WAIS info with other info about the individual. combines details within a conceptual framework unit to produce a more easily distinguishable pattern
- believes that HOW an individual performs a task is as important (or more) than the score he obtains on subtest
- understanding performance on individual items (including kinds of errors) will give much info if you can establish that what you observed reflects a pattern of bx in multiple contexts |
|
|
Term
| Process Approach requires |
|
Definition
1) that the examiner is a KEEN OBSERVER!
2) that the examiner use a problem solving model that includes forming and testing hypotheses
3) that examiner know what every subtest measures (and how it does it) and what subtests have in common/differences |
|
|
Term
| Suggest Procedures for basic Profile Analysis |
|
Definition
1. Report and describe FSIQ (and/or GAI)
2. Report and describe VCI
3. Report and describe PRI
4. Report and describe WMI
5. Report and describe PSI
6. Evaluate index-level discrepancy comparisons
7. Evaluate strengths and weaknesses
8. Evaluate subtest-level discrepancy comparisons
9. Evaluate patterns of scores within subtests
10. Perform process analysis (Core and Supplemental subtests) |
|
|
Term
| Profile Analysis Process: Begin with FSIQ |
|
Definition
Most reliable score, most representative of general intellectual functioning (g)
Report with corresponding percentile rank and CI
Interp depends on collaborative info and discrepancies among index scores |
|
|
Term
| Profile Analysis Process: Index and Subtest |
|
Definition
Step 2-5: Individual Index Scores
Analysis of four composite score recommended as primary level of clinical interpretation
- Will help id if a subtest/index is pulling FSIQ down overall
- Will help determine when to use GAI (might not for people with LD, TBI)
If subtest scores similar to index scores, can be more confident in interpretation |
|
|
Term
| Step 6: Index Level Discrepancy Comparisons |
|
Definition
| Use the comparisons in the Record Form to evaluate differences b/t pairs of index scores. Evaluate statistical significance and cumulative frequency (base rate) |
|
|
Term
| Statistical vs Clinical Significance (and Frequency) in Profile Analysis |
|
Definition
Stat sig is much less important than how UNUSUAL it is (frequency)
Sig only tells likelihood difference is REAL, frequency tells whether it is unusually large and therefore likely to be important |
|
|
Term
| VCI > PRI interpretation/meaning |
|
Definition
Verbal comprehension skills are better developed than perceptual reasoning skills.
Verbal processing is better developed than visual-spatial processing.
Auditory-vocal processing is better developed than visual discrimination processing.
Knowledge acquired through accumulated experience is better developed than knowledge needed to solve nonverbal problems.
Retrieval of verbal information from long-term memory is better developed than nonverbal problem solving.
Crystallized knowledge is better developed than fluid reasoning. |
|
|
Term
| VCI > PSI Interpretation/Meaning |
|
Definition
Verbal comprehension is better developed than processing speed.
Verbal processing is better developed than speed of mental operation.
Auditory-vocal processing is better developed than visualmotor coordination.
Processing of verbal stimuli is better developed than processing of nonverbal stimuli.
Long-term verbal memory is better developed than short-term visual memory.
Crystallized knowledge is better developed than processing speed. |
|
|
Term
| What does it mean when you find a discrepancy b/t an individual's performance on indexes? |
|
Definition
You have id'd a relative strength or weakness.
- Could be a processing deficit
- Something other than intellectual ability could be affecting a score
- Could indicate specific educational/tx needs |
|
|
Term
| Pattern of Scores within Subtests |
|
Definition
Look for uneven performance within a subtest "intrasubtest scatter"
Could indicate:
- rapport px
- attn px
- language px
- bored (gifted)
- difficulty initiating tasks
Someone who answers 15 questions then hits ceiling is different than someone who never hits ceiling and has a score of 15 |
|
|
Term
| Diagnostic Profiles - SCAD |
|
Definition
Below average scores on Symbol search, Coding, Arithmetic, and Digit span
More common in LD and ADHD subgroups |
|
|
Term
| Diagnostic Profiles - ACID |
|
Definition
Below average scores on Arithmetic, Coding, Information, and Digit span
Common in ADHD samples |
|
|
Term
| Diagnostic Profiles - Learning Disabled Index |
|
Definition
significant difference b/t verbal and performance
VIQ < PIQ
- Left hemispheric dysfunction |
|
|
Term
| MacMann & Barnett (Psychometric Considerations) |
|
Definition
Ran 5000 simulated WISC scores to look at reliability of profile interpretation
Index score comparison unreliable.
- Most did not have same signifiant profile at retest
More problems with ipsative analysis.
- there are 54 Kauffman profiles and many cases had more than one profile significant. Just not helpful, scores less reliable at subtest level.
- This was for Wechsler scale and its the most psychometrically sound, doesn't look good for the other ones either |
|
|
Term
| Problems with Diagnostic Profiles - SCAD, ACID, Learning Disability Index |
|
Definition
SCAD - little diagnostic accuracy. Many false positives
ACID - people with ADHD more likely to have ACID than typical pop BUT most people with ADHD don't have ACID profile
Learning Disability Index - relies on VIQ subtest which is highly correlated with education. Many false positives |
|
|
Term
|
Definition
Individuals have different aptitudes that can lead to more efficient and effective treatments if the two are linked.
Can be DISORDINAL - one for you and one for me, don't work the other way
or ORDINAL - one for me and neither (or both) for us. |
|
|
Term
|
Definition
Modality matching Model - id process strength and match instruction/tx (visual, auditory)
Cognitive/Processing Style - same as modality except terminology
Neurological Model - brain functioning IS the aptitude. Left/Right brain dominate |
|
|
Term
| Do ATI's have empirical support? |
|
Definition
| NO. Might even be backward (Ayers & Cooley, 1986) |
|
|
Term
| What is tx validity? Relation to ATI? |
|
Definition
AKA tx or instructional utility
Degree to which any assmt procedure contributes to beneficial tx outcomes.
Lack of ATI evidence = lack of tx validity of IQ tests. |
|
|
Term
| Gresham and Witt, IQ tests for classification |
|
Definition
IQ results consistently fail to differentiate b/t LD, LA, and mild ID on tests of academic performance
Failure to discriminate among these = lack of tx utility for classifcation
Test scores are frequently ignored anyway when classifying within educational settings |
|
|
Term
| Summary of Gresham & Witt article |
|
Definition
IQ tests have no tx utility
- Cannot monitor/track progress
Classification based on IQ has no utility
- outcomes ignored
- failure to discriminate
- failure to predict achievement at individual level
Waste o money, fool.
Reinforces the idea that the problem lies with the individual |
|
|
Term
|
Definition
Traditional view of validity is incomplete
- content, criterion, and construct alone not enough
- not just a psychometric property but should include social value when decisions are made based on test
- what will scores be used for, are they necessary, implications of a score
Determine adverse impact - poor outcomes for specific groups as a consequence of testing
Turns assmt into a moral social practice! |
|
|
Term
| Messick's 6 points of construct validity (list) |
|
Definition
Content
Substantive
Structural
Generalizability
External
Consequential |
|
|
Term
| Content Validity (Messick) |
|
Definition
| Relevance and representativeness of test content and its quality |
|
|
Term
| Substantive Validity (Messick) |
|
Definition
| Theoretical basis/process for test content as well as evidence that individuals are actually engaged in the process purported to be evaluated |
|
|
Term
| Structural Validity (Messick) |
|
Definition
| Scoring structure of a measure and whether method is appropriate for construct |
|
|
Term
| Generalizability Validity (Messick) |
|
Definition
| degree scores (measure of construct) generalize to other populations, environments, and tasks |
|
|
Term
| External Validity (Messick) |
|
Definition
discriminate and convergent validity
- degree to which measure correlates with measures of same construct (convergent) and degree to which it does not correlate with other measures of different constructs (divergent) |
|
|
Term
| Consequential Validity (Messick) |
|
Definition
| value implication/social consequential of test scores and actual and potential outcomes of test use |
|
|
Term
|
Definition
Construct underrepresentation - test too narrow and fails to full capture construct investigating
- using vocab to determine reading achievement
Construct Irrelevant Variance (Difficult or Easiness)
- Features of test, irrelevant to the construct, that make the task more difficult or allow some groups to answer correctly (but not relevant)
- reading comp test in La about swamp people
- math word px and reading ability |
|
|
Term
| Messick's considerations for test use and interpretations |
|
Definition
Construct Validity (test interp & evidential basis)
Relevance/Utility - benefit test use and interp has for individual (test use & evidential basis + construct validity)
Value implications - changes/impact that will occur as part of testing and interp
(test interp & consequential basis + construct validity)
Social Consequences - Long term implications as a result of the changes to to testing and interp (test use & consequential basis + everything)
TABLE |
|
|
Term
|
Definition
Test sorts ppl into different groups at diff rates than expected given base rates
Conceptualized in terms of predictive validity and bias exists when test performance indicates one group will perform better than another when a true performance difference does not exist.
Detected through regression equations |
|
|
Term
|
Definition
when test items are consistently responded to differently by diff groups and those responses have a negative impact for one of the groups
Detected through item characteristic curve |
|
|
Term
| Frisby - Alternative Tests |
|
Definition
Based on assumption that traditional assmt not appropriate for minority groups
Alt assmt are necessary for non-native english speakers, to reduce performance discrepancy b/t whites and minorities, and to lead to educational reforms |
|
|
Term
| Status of bias and alternative assmts |
|
Definition
Empirical evidence has failed to demonstrate bias in testing against minority groups (except non-native English speakers)
Tests are standardized to be representative of the population
A small examiner effect has been found
IQ performance does predict well for groups but not as well on individual level
There is a tendency to ignore the exception to the rule (some minorities do better)
Alt tests often not real alternatives. Changes in admin or scoring, not content
- have failed to close the gap between whites and minorities
- items judged as unfair haven't been
- more variability within groups than across
- true differences most likely tied to SES and idiosyncratic cultural factors |
|
|
