Term
| 1935 Army Flight Competition |
|
Definition
Boeing Co. won competition
-Test flight in front of Army brass
-Plane crashed & killed 2 of 5 crew members
-Boeing pilots developed checklist
-Flew 1.8 million miles without accident
-Army ordered 13,000 planes (B-17) |
|
|
Term
2001 Johns Hopkins Hospital
ICU Central Line Infections |
|
Definition
SWash hands with soap
SClean skin with antiseptic
SPut sterile drapes over entire patient
SWear a mask, hat, sterile gown, & gloves
SPut sterile dressing over insertion site once line is in
SLine infection went from 11% to 0%
SMortality rate also decreased substantially |
|
|
Term
| Treatment Integrity/Treatment Fidelity |
|
Definition
Integrity: Rigid adherence to a code or standard of values.
The state of being unimpaired: soundness
Fidelity: Faithfulness to obligations, duties, or observances.
Exact correspondence with fact or with a given quality, condition, or event: accuracy
|
|
|
Term
Treatment Integrity & RTI
Why is it important? |
|
Definition
SIn RTI, one must demonstrate that measurable changes in behavior (academic or social) are related to systematic & controlled changes in the environment (intervention)
SObserved changes in behavior must be attributed to observed implementation of the intervention
SWithout objective & documented specification that the intervention was implemented as planned or intended, we cannot conclude that inadequate response to intervention was due to a poor intervention
SSimilarly, intervention success cannot be attributed to the intervention if we do not know how or if the intervention was implemented
SIn RTI, the systematic and frequent measurement of treatment integrity is essential |
|
|
Term
| Overview of Treatment Integrity Concept |
|
Definition
SChange in behavior attributable to systematic changes in environment
SConsistent application of intervention necessary, but insufficient for change
STreatment integrity has been referred to as:
STreatment fidelity (psychotherapy)
SReliability of treatment implementation (internal validity)
SProcedural reliability (applied behavior analysis)
SCompliance to regimens (medicine)
S“Curious double standard” |
|
|
Term
| Treatment Integrity Defined |
|
Definition
SDegree to which intervention is implemented as planned or intended
SChange in behavior attributable to systematic changes in environment
S3 Dimensions of Treatment Integrity
STreatment Adherence
SInterventionist Competence
STreatment Differentiation
SBreakdown in any of above requires different remedies |
|
|
Term
| Treatment Integrity & Treatment Outcomes |
|
Definition
|
|
Term
| Treatment Integrity & Treatment Outcomes |
|
Definition
1.Adequate integrity- good outcome --> treatment implemented as planned & response adequate
2.Adequate integrity- no outcome--> treatment implemented as planned but inadequate response
3.Inadequate integrity- good outcome --> little or no integrity but adequate response
4.Inadequate integrity- no response --> little or no integrity and inadequate response |
|
|
Term
Treatment Integrity
Some Important Concepts |
|
Definition
STreatment adherence
SDegree to which treatment implemented as planned or intended
SBased on % of steps correctly implemented
SImplementer competence
SSkill of implementer delivering treatment
STraining & experience
STreatment differentiation
SDegree to which different treatments differ in intended manner
STreatment receipt
SDegree to which implementer understands treatment procedures
STreatment enactment
SDegree to which implementer applies the treatment |
|
|
Term
| Variables Influencing Treatment Integrity |
|
Definition
SExternal Environment
SLevel of support from stakeholders (teachers, administrators, parents)
SLevel of opposition from stakeholders
SConsistency of intervention with policies/mandates
SCoordination with other agencies (mental health, social services, etc.)
SOrganization
SAccess to materials/supplies
SAdequate staffing
SAdequate funding
SOpportunities for communication among consultants/consultees
School-based leadership/support (principal, team members, etc.)
|
|
|
Term
| variables influencing treatment integrity- Intervention |
|
Definition
SIntervention
SIntervention complexity
STime/duration required
SMaterials/resources required
SNumber of interventionists required
SActual effectiveness
SCompatibility with classroom routines/school culture
SEase of implementation
SDegree of improvement over previous practices
SRate of behavior change |
|
|
Term
| Variables Influencing Treatment Integrity- Interventionist |
|
Definition
SInterventionist
SPerceived effectiveness
SPerceived need
SMotivation to implement
SWillingness to try intervention
SPerceptions of role compatibility
SSkill proficiency
SSelf-efficacy
Shared decision-making/buy-in |
|
|
Term
|
Definition
SReviewed 539 studies between 1968-1980
SOnly 20% of studies reported data on treatment integrity
S16% of studies did not report an operational definition of the IV
SNo trends over time in reporting integrity
S“Curious double standard” |
|
|
Term
SGresham, Gansle, & Noell (1993) |
|
Definition
SReviewed 158 studies between 1980-1990
SOnly 16% reported data on treatment integrity
SOnly 32% of studies provided operational definition of the IV |
|
|
Term
|
Definition
SFocused on intervention studies in autism published 1993-2003
S60 studies (more than half) published in JABA
SOnly 18% reported data on treatment integrity
S92% of studies operationally defined the IV |
|
|
Term
SMcIntyre, Gresham, DiGennaro, & Reed (2007) |
|
Definition
SReviewed 152 studies 1991-2005
S30% of studies reported data on treatment integrity
SMajority of studies reported integrity data had 90% or more integrity
SOver 60% of studies did not report integrity data or monitor integrity
SJust under half (45%) of studies considered “high risk” for treatment inaccuracies
S95% of studies operationally defined the IV |
|
|
Term
|
Definition
SAverage rate of reported integrity 1968-2005 is 21% (834 studies)
SReported integrity has not appreciably increased over 37 years
SOperational definition of IV increased substantially
SBehavioral interventions in schools at high risk for treatment inaccuracies
SGranting agencies now require integrity data be collected
SInstitute of Educational Sciences
SNational Institutes of Health |
|
|
Term
| Technical Issues in Treatment Integrity |
|
Definition
SSpecification of Treatment Components
SClear, concise description of intervention components
SLess than 30% of interventions provide operational definitions
SLevel of specificity important (global-intermediate-molecular)
SWeighting of each component important (some more important than others)
SDeviations from Integrity
SHow far can we drift away from integrity & still be effective?
SSubstantial deviations from integrity may produce positive results
SPossible to have perfect integrity & no positive outcomes
S
Psychometric/Measurement Concerns
|
|
|
Term
Treatment Integrity Assessment
Practical Considerations |
|
Definition
SDirect Assessment of Treatment Integrity
SBased on systematic observation of treatment implementation
SBased on task analysis of major treatment components
SOccurrence & nonoccurrence of each component recorded
SLevel of integrity calculated as % of components implemented
SMajor drawbacks: Reactivity & time sampling
SIndirect Assessment of Treatment Integrity
SSelf-reports/self-monitoring (demand characteristics/social desirability)
SInterviews
SPermanent products (Daily Behavior Ratings/worksheets)
SManualized Treatments |
|
|
Term
| Measurement Threats to Treatment Integrity |
|
Definition
SReactivity
SDrift
SComplexity- more complex code/treatment à lower integrity
SExpectancies- bias, conform to expectancy bias (observers)
SCentral Threat: ACCURACY |
|
|
Term
|
Definition
SAccuracy often inferred from interobserver agreement (IOA)
SAccuracy & agreement not the same
SAccuracy is extent to which observations match predetermined standard
SIntervention components known a priori (i.e., known standard)
SManualized treatments can be used as known standard
SDependent variable only known a posteriori
|
|
|
Term
|
Definition
STeachers aware of integrity assessments in classrooms
SConsultants often serve as SD for treatment implementation
SAbsence of consultants serve as SΔ for poor or no treatment implementation
SSuggestions:
SSpot check integrity assessments
SManualized treatments (Intervention “Tip Sheet”)
SSubterfuge |
|
|
Term
|
Definition
STeachers often drift away from the original intervention
SInterventions often “modified” by teachers
SDrift may occur for number of reasons
SSuggestions:
SPerformance feedback on intervention implementation
SWritten intervention “Tip Sheet”
SPlan Implementation meetings |
|
|
Term
|
Definition
SNumber of treatment components in intervention
SResponse effort on part of teachers
STechnical difficulty of intervention
SSuggestions:
SKeep number of component to minimum
SImplementation training sessions
SModeling implementation of intervention |
|
|
Term
|
Definition
STeachers often hold expectancies regarding behavior change
SSome teachers may expect the plan to fail
SSome lapses in integrity result from negative expectancies
SSuggestions:
SShare past experiences you’ve had with the intervention
SShare past findings using the intervention
SUse referent power (“I saw this work with a kid just like your kid.”) |
|
|
Term
| Standards for Treatment Integrity |
|
Definition
SNo database for optimal levels for different treatments
SSome treatments for some students may require 70% integrity
SOther treatments for other students may require 90% integrity
STreatment Effect Norms (Yeaton, 1988)
SAverage outcome for given treatment for given problem
SMeta-analyses provides such estimates
SExample: Good Behavior Game produces consistent reductions in disruptive behavior when implemented with 80% integrity
SOne could establish treatment effect norms across multiple treatments, multiple target behaviors, measured by multiple methods
|
|
|
Term
Treatment Effect Norms
Some Evidence-Based Treatments |
|
Definition
SCheck In/Check Out (Tier 2)
SGood Behavior Game (Tier 1)
SRepeated Reading (Tier 2)
SBehavioral Consultation (Tier 2)
SBehavioral Parent Training (Tier 3)
SCoping Cats (Tier 3)
SCoping Power (Tier 3)
SClasswide Peer Tutoring (Tier 1)
SPull Out Reading Groups (Tier 2) |
|
|
Term
|
Definition
SSpecific components of treatment should be operationally defined & measured (like a target behavior)
STreatment integrity expressed as % of components implemented
SComponent integrity (consistency of implementation over time)
SSession integrity (integrity within a session or day)
STotal integrity (component + session/day)
SSpecific performance feedback greatly improves integrity |
|
|
Term
Future Directions for Treatment Integrity
Practice |
|
Definition
STreatment integrity should be made a consistent practice in schools
SAs RTI practices increase, treatment integrity becomes an essential practice
STreatment integrity training should be an essential part of all preservice training programs for school professionals
SProfessional conferences should include presentations regarding conceptual issues & research findings related to treatment integrity |
|
|
