Shared Flashcard Set

Details

RTI Scientific Evidence Week 2
Evidence Based Practice
45
Psychology
Graduate
10/06/2011

Additional Psychology Flashcards

 


 

Cards

Term
What are evidence-based practices?
Definition
Practices based on controlled scientific research
- Clearly defined independent variables (the intervention + moderators/mediators)
- Clearly defined dependent variables (the outcomes)
- Procedures for monitoring dependent variable and independent variable (tx integrity)
- Design that controls for threats to internal validity – tx caused the change
- Replications to provide evidence for external validity – generalizability of findings
Evidence exists on a continuum
- Strong evidence/support
- Promising evidence/support
- Marginal evidence/support
- No evidence/support
Term
Moderator
Definition
Qualitative (sex, race, SES) or quantitative (achievement scores, depression scores) variable that affects direction or strength of relationships between an IV and DV (3rd variable) – find in meta-analyses often, a correlation
Example: Cognitive behavior therapy reduces depression more in women than in men
Term
Mediator
Definition
Explains or accounts for the relationship between an IV and DV. Specifies how, why, or through what process an IV affects a DV, causal
Example: Coercive parenting behavior leads to the development of antisocial behavior patterns in boys via negative reinforcement  neg reinf is the mediator, explanation of how parenting bx leads to antisocial bx
Term
Moderated and mediated effects are studied via
Definition
Multiple regression

Structural equation modeling
Term
National Autism Center's National Standards Report Requirements for Group Design & Single Case Design
Definition
Group = 2 or more groups, random assignment, n > 10 per group. Procedures for missing data = multiple imputation procedures

Single Case = minimum 3 comparisons (3 pts in time or phase repetitions)
- Min 5 data pts per condition
N = 3 minimum
No missing data possible
Term
Measurement of DV (National Autism Center)
Definition
Test/Scale
- type of measurement (rating/checklist)
- should be standardized
- have psychometric data
- evaluators blind and independent

Systematic Direct Observations
- type of measurement (event/interval/permanent product)
- Dimension of behavior (frequency/intensity/duration/latency)
- IOA > 90%, kappa > .75
% of sessions IOA collected and % session data collected
Term
What does kappa control for that makes it better than IOA?
Definition
chance agreement
Term
Measurement of IV (National Autism Standards)
Definition
> 80% treatment integrity/fidelity, IOA>80%
Term
Generalization Data (National Autism Standards)
Definition
objective data

maintenance data (minimum of 2), across settings, stimuli and persons
Term
Research Approaches & Scientific Evidence
- Experimental Design
- Quasi-experimental design
- Single Case experimental design
- Regressions discontinuity design
Definition
Exp - Randomized clinical trial (RCT - strongest evidence)

Quasi - nonrandomized/intact groups, matching/ANCOVA (ways to control up front or statistically)

Regression - on next card
Term
Regression Discontinuity Design
Definition
Participants assigned to groups based on cut score (above or below)

Assignment variable may be pretest score on outcome variable

Can be used to compare 2 or more tx

Powerful design - good protection against threats to internal validity

Compares slopes of regression lines of exp and control groups
Term
Single Case Experimental Design Facts
Definition
Individual case is unit of intervention & unit of data analysis
Within design, the case provides its own control for comparison purposes
Outcome variable is measured repeatedly within & across conditions, gives more frequent measurement
Experimental control involves replication of intervention effects – allows for you not to need as many participants
Introduction & withdrawal (reversal) of independent variable (ABAB design), demonstrating experimental control
Iterative manipulation of independent variable across different observational phases (alternating treatments or multielement designs)
Staggered introduction of independent variable across different points in time (multiple baseline design)
To meet internal validity standards, study must include at least 3 attempts to demonstrate effect at 3 different points in time with 3 different phase repetitions
Term
Causal Questions that Single Case Designs can answer
Definition
Is there a causal relation b/t the introduction of an IV & change in a DV?
- Does intervention B reduce a px bx for this case or these cases?

What is the effect of altering a component of multi-component IV on a DV?
- Does adding intervention C to intervention B further reduce a px bx? Looking at interaction effects

What are relative effects of 2 or more IVs on a DV?
- Is intervention B or intervention C more effective in reducing px bx? <-- Have issues of sequencing effects here
Term
Quantitative Features of Single Case Designs
- Level
- Trend
- Variability
Definition
Level: Mean score for the data within a phase
Trend: Slope of best-fitting straight line for the data within a phase
Variability: Fluctuation of data around the level within a phase
Term
Quantitative features of Single case designs
Definition
Immediacy of effect
Overlap – PND (percentage of non-overlapping data points), HLM can be a better measure
Consistency of data patterns across similar phases
Intraocular test of significance (Do the eyes have it?)
Term
Criteria for SCDs that meet evidence standards (from What Works Clearinghouse)
Definition
IV (the intervention) must be systematically manipulated with the researcher determining when & how the IV conditions change

Each outcome measure (the DV) must be measured systematically over time by more than one assessor

IOA collected in each phase for minimum of 20% of data points

IOA must be 80-90%; kappa=.60 (minimum)

Study must include 3 attempts to demonstrate intervention effect at 3 different points in time or with 3 different phase repetitions

Each phase must have a minimum of 3 data points
Term
Basic Single Case Experimental Designs
Definition
Reversal/Withdrawl

Multiple Baseline

Multielement (Alternating Treatments)

Changing criterion (accelerating or decelerating)
Term
Types of Reversal Designs
Definition
ABAB (main effects design)

ABAC (main effects)

A-B-B+C-A-C-B+C (interaction effects design)
Term
types of Multiple BL
Definition
Across participants
Across settings
Across behaviors
Term
Threats to internal validity
Definition
Ambiguous temporal precedence (A then B then A then B)

Selection (each participant exposed to all conditions)

History (phase repetition/replication-ABAB)

Maturation (3 replications of effect at 3 different points in time)

Statistical regression (repeated measurement of DV over sessions/days/phases)

Attrition (it happens—people die, move, get in bar fights, get arrested, or otherwise disappear)

Testing (reactivity effects)  this is a bigger threat than the others

Instrumentation (observer drift, bias, complexity of code)  this is a bigger threat than the others
Term
Historical Antecedents of RTI

National Research Council Report (1982) on validity of SPED
Definition
Validity of SPED evaluated on basis of 3 criteria
- Quality of general education program

- Value of special education program in producing important student outcomes

- Accuracy & meaningfulness of assessment process in identification of disability

First 2 criteria emphasized quality of instruction

Third criterion emphasized evidential & consequential bases for test use & interpretation,

The third criterion is basically a validity issue (Messick, 1995)
Term
Historical Context of RTI

PL 94-142 Education of All Handicapped Children Act of 1975
Definition
Free appropriate public education (FAPE)

Least restrictive environment (LRE)

Protection in Evaluation Procedures (PEP) – no single test or measurement should be used to place in SPED, tests should be psychometric, valid, etc

Individualized Education Program (IEP) – appropriate is defined by the IEP
Term
?Historical Context of RTI: Learning Disabilities Initiative (2000, Office of Special Education Programs OSEP)
Definition
Planning meeting for discussing SLD – needed a meeting bc LD is new (1963), over ½ of kids in sped are LD, and no good definition of LD

Synthesize & organize most current & reliable research on key issues in SLD

Brought together 18 diverse group of stakeholders

Selected 9 issues for white papers

Identified potential authors & potential respondents
Term
When did term LD come about?
Definition
1963
Term
Historical Context of RTI

Learning Disabilities Summit (8/27, 8/28, 2001)
Definition
Invitation-only LD Summit in Washington, DC
Each of 9 white papers presented 3 times to different audiences
Historical Perspective (Hallahan & Mercer)
Early Identification of LD (Jenkins & O’Connor)
Classification of LD (Fletcher et al.)
LD As Operationally Defined by Schools (MacMillan & Siperstein)
Discrepancy Models in the Identification of LD (Kavale)
Responsiveness to Intervention in Identification of LD (Gresham)
Direct Assessment of Processing Weaknesses in LD (Torgesen)
Clinical Judgments in identifying & Teaching LD (Wise & Synder)
LD versus Low Achievement Meta-Analysis (Fuchs et al.)
Term
Historical Context of RTI

Stakeholder Roundtables
Definition
First roundtable (October, 2001)
- Select group of researchers including white paper authors
- Shared information & debated issues
- Implications for research, policy, & practice
- Consensus paper (we did have some detractors on certain issues)

Second Roundtable (November, 2001)
- Representatives from all national LD organizations (NJCLD, LDAA, DLD-CEC)
- Responded to white papers consensus chapter
Term
Historical Context of RTI

President's Commission on Excellence in SPED (2002) Findings
Definition
Finding 1: Focus on process rather than results

Finding 2: Rare use of prevention & intervention

Finding 3: Two educational systems: General Education versus Special Education

Finding 4: Current system often fails children and parents

Finding 5: Compliance to law driven by litigation pressures rather than what’s good for kids

Finding 6: Current methods of identification lack validity

Finding 7: Special education requires highly qualified teachers
Term
IDEIA
Definition
Individual With Disabilities Education Improvement Act of 2004

Signed into law by George W. Bush
Term
Why RTI? Advantages?
Definition
Early Identification Academic & Behavioral Difficulties
- Poor readers at end of 1st grade will be poor readers at end of 5th grade
- Conduct/antisocial behavior by end of 3rd grade predicts lifetime antisocial behavior
- Longer behavior persists, the more resistant behavior is to intervention

Risk versus Deficit Approach
- Screen all students for risk (mammogram-PAP-PSA-colonoscopy)
- Refer-Test-Place operates under deficit model—”wait to fail”

Reduction of Identification Biases (gender-SES-minority group membership)

Focus on Student Outcomes
- Positive student outcomes
- Direct measurement of achievement, behavior, & instructional environment
Term
Problem Solving Approach in RTI (Interview schedules)
Definition
Problem Identification: What is the problem?

Problem Analysis: Why is the problem occurring?

Plan Design/Implementation: What should be done about the problem?

Plan Evaluation: Did the plan work in changing behavior?
- Data-based decision making
- Goal attainment
- Social validation
Term
3-Tier Model in RTI
Definition
Universal Interventions (vaccinations)
- All students
- Schoolwide or classwide
- 80-85% respond favorably

Selected Interventions (go to the Dr.)
Some students
- Classroom-based strategies (via consultation/problem solving)
- Small group interventions (protocol-based)
- 10-15% will respond

Targeted/Intensive Interventions (go to a specialist)
- Individualized interventions
- Intense & powerful
- Function-based interventions
- 3-5% will respond
Term
5 Fundamental Principles of RTI
Definition
1) Intensity of intervention matched to degree of unresponsiveness to intervention

2) Movement through levels based on inadequate response

3) Decisions regarding movement through levels based on empirical data collected from variety of sources

4) Increasing body of data collected to inform decision-making

5) SPED & IEP determination considered only after student shows inadequate response to most intensive interventions available
Term
Essential Features of RTI
Definition
Tx Validity (assmt directly linked to intervention)

Progress Monitoring (direct measurment of intervention response)

Tx Integrity (integrity of intervention directly measured)

Durability (maintenance of effects evaluated)

Consumer Satisfaction (social validation of intervention/outcomes)
Term
Types of RTI (Card 2)
Definition
Preventive RTI
- Universal prevention (primary prevention)—To prevent problems
- Universal screening 3 times per year
- Multiple gating procedures – assessment, recommendation by teacher, need to pass multiple

Reactive RTI
- Selected interventions (secondary prevention)—To reverse problems
- Replaces refer-test-place approach
- Moves from assessment-oriented to intervention-oriented practices

Eligibility RTI
- Most intense (tertiary prevention)—To reduce problems
- Used to make eligibility determinations
- Disability versus need decisions
Term
Treatment Integrity
Definition
Degree to which intervention is implemented as intended or planned

Other terms:
- tx fidelity
- procedural reliability
- compliance
- adherence

Logic of Tx Integrity Concept = observed changes in DV MUST be attributed to changes in IV
- Best way to ensure this is to measure extent to which tx is implemented
Term
3 Dismensions of Tx Integrity
Definition
Tx Adherence

Interventionist Competence

Tx differentiation
Term
Tx Adherence
Definition
Degree to which an intervention is implemented as planned or intended
Term
Interventionist Competence
Definition
Interventionist's skill and experience in implementing a particular tx
Term
Tx Differentiation
Definition
Extent to which interventions differ on critical dimensions
Term
Breakdown in any of the 3 dimensions of tx integrity require different courses of action depending on which area is the problem
Definition
Adherence: Performance feedback & systematic monitoring of treatment

Competence: Training & feedback to increase competence

Differentiation: Specification of theoretical differences among treatments
Term
How common is tx integrity assessed in ABA?
Definition
Peterson et al. (1982) reviewed 539 studies in JABA 1968-1980
- Only 20% of studies reported data on treatment integrity
- 16% of these studies did not operationally define the IV (the treatment)

Gresham, Gansle, & Noell (1993) reviewed 158 JABA studies 1980-1990
- Only 16% (25 studies) provided treatment integrity data
- 32% provided operational definition of the IV (the treatment)

Wheeler et al. (2006) reviewed 60 studies on autism 1993-2003
- Only 18% reported data on treatment integrity
- 92% provided operational definition of IV

McIntyre et al. (2007) reviewed 152 JABA studies 1991-2005
- 30% provided data on treatment integrity
- 95% operationally defined the IV
- 39% of studies considered to be at no risk for treatment inaccuracies
- 45% of studies considered to be high risk for treatment inaccuracies
Term
Take homes from tx integrity articles in ABA
Definition
Reporting of integrity data stable & low from 1968-present

Some (many ABA types) argue that behavior change ensures accurate treatment implementation (????)

Other Agencies Now Mandate Collection of Integrity Data
- American Psychological Association (Divisions 12, 16, 53, & 54)
- U.S. Department of Education (Institute of Educational Sciences)
- National Institutes of Health
Term
Ways to measure of Treatment Integrity
Definition
Direct observations of treatment implementation

Checklist assessment based on observation (Likert scale from low to high)

Self report/monitoring of treatment integrity

Permanent product recording of treatment implementation
Term
How high does integrity have to be? Is 100% required in all cases?
Definition
Some problems & participants—70% integrity might be good enough

Other problems & participants might require 90% integrity

Possible Solution: Treatment Integrity Effect Norms
- Treatment effect norm is average outcomes of given intervention (meta-analysis)
- Similar logic could be used to catalog which interventions at what levels of integrity measured by what methods produce given levels of outcomes
- Good Behavior Game: Might find that 80% integrity as measured by direct observations is required to produce socially valid reductions in disruptive behavior (ES=0.85)
Term
Study Questions given by Gresham
Definition
Describe evidence-based practices. How is evidence classified?

Describe and contrast the 4 basic research approaches for establishing scientific evidence.

Describe the logic of single case experimental designs. How do they differ from group experimental designs? Give an example of each of the 4 basic single case experimental designs.

How do single case experimental designs meet internal validity standards?

What are the 7 quantitative features of single case experimental design data?

How do single case experimental designs control for threats to internal validity?

What is RTI? Describe 4 advantages of RTI over the “refer-test-place” approach

What is the 3-tier model? Describe 3 types of RTI.

What is treatment integrity? What are its dimensions? How common is integrity assessment in ABA? How is integrity measured?
Supporting users have an ad free experience!