Outcome
Type |
Measure |
Description |
Time frame |
Safety issue |
Primary |
Frequency and Duration of Use of Office Discipline Referrals, In-School Suspension, and Out-of-School Suspension |
We will collect data on the use of office discipline referrals (ODRs), in-school (ISS), and out-of-school suspensions (OSS). As part of the National TA Center on PBIS, all schools that will be recruited for this project use the School-Wide Information Systems (SWIS; May et al., 2013), a web-based behavior tracking platform, to enter and review behavior incidents and actions taken by school personnel including office discipline referrals (ODRs), in-school (ISS), and out-of-school suspensions (OSS). |
baseline, 1 year, 2 years |
|
Primary |
Engagement vs. Disaffection with Learning Student-report |
The Engagement vs. Disaffection with Learning Student-report assesses engagement as a key component of motivated action leading to student learning and achievement. The assessment measures engagement versus disaffection in the classroom ranging from enthusiastic, effortful, emotionally positive interactions with learning activities to apathetic withdrawal and frustrated alienation, with both positive manifestations of behavioral and emotional participation in classroom and withdrawal of behavioral and emotional participation and alienation from learning. All items have five response options: (1) Not at all True; (2) A little True; (3) Mostly True; (4) Very True; (5) Prefer not to answer. |
baseline, 1 year, 2 years |
|
Primary |
Inclusive Teaching Practices |
The WOW+ (Welcome at the door, Own your environment, Wrap up with intention +) Self-Report measures inclusive teaching practices, both general and specific teaching strategies that are part of the ISLA intervention. All school personnel in all treatment schools will receive training on the ISLA strategy for universal relationship building, known as the WOW strategy, which consists of three concrete skills rooted in behavioral science that educators can apply every day to create positive classroom environments: welcome students, own your classroom environment, and wrap up class with the intention. The WOW+ Self-Report is an adaption of the WOW Classroom Observation Tool which is a five-item measure with individual item scores ranging from 0 to 2, where each item is an operationally defined behavior based on the elements of the WOW strategy training. |
baseline, 1 year, 2 years |
|
Primary |
School Climate Survey- Student |
The School Climate Survey: Secondary (SCS-S) is a 36-item, computerized school climate rating scale (Center on PBIS, 2022). The purpose of the SCS-S is to obtain middle and high school student perception ratings of school climate. Respondents use a 4-point Likert scale: 1=Strongly Disagree, 2 = Disagree, 3 = Agree, and 4 = Strongly Agree. To compute the overall school climate score, item responses are summed and then divided by the total number of items. Higher overall scores reflect a positive sense of the school environment with respect to (a) interpersonal relations among and between peers and adults, (b) learning environment, and (c) feelings of belonging and acceptance. |
baseline, 1 year, 2 years |
|
Primary |
School Climate- Teacher |
The School Climate Survey: School Personnel (Center on PBIS, 2022) includes 29 items and 5 subscales, including staff connectedness, structure for learning, school safety, physical environment, peer and adult relations, and parent involvement. School personnel rate items as Strongly Agree, Somewhat Disagree, Somewhat Agree or Strongly Agree. The personnel survey also demonstrates adequate psychometric properties with an internal consistency of .94 for the overall school climate scale. |
baseline, 1 year, 2 years |
|
Primary |
Student-Teacher Relationships |
The Adult Social Support subscale from the School Climate Survey: Student (SCS-S; Center on PBIS, 2022) will be used to measure student-teacher relationships in schools. The Adult Social Support subscale contains four items and has strong internal consistency (a = .91). A high mean score on the Adult Social Support subscale represents a positive feeling of adult support and encouragement, whereas a low score suggests poor interpersonal relationships among students and adults within the school. From this subscale, we expect to understand data about student engagement, school climate, opioid and other substance misuse, internalizing and externalizing behaviors. |
baseline, 1 year, 2 years |
|
Primary |
Opioid and Other Substance Misuse |
Thirteen opioid and other substance misuse questions and response criteria are derived from the Monitoring the Future survey conducted annually with adolescent youth throughout the United States and the National Institute on Drug Abuse-modified Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) measure. The survey is funded by NIDA as an on-going surveillance effort to understand and report substance use trends among youth. Respondents use an ordinal response scale with seven options (0 Occasions; 1-2; 3-5; 6-9; 10-19; 20-39; 40 or more) to describe substance use during the last 12 months. |
baseline, 1 year, 2 years |
|
Primary |
Student Internalizing Behaviors |
The Strengths and Difficulties Questionnaire (SDQ; Goodman 1997) is a brief (25 items) behavioral screening questionnaire about psychosocial problems for youth, and includes competencies or strengths in addition to assessing problems. The SDQ is equally divided across five scales measuring emotional symptoms, conduct problems, hyperactivity-inattention, peer problems, and prosocial behavior. Participants rate items as Not True, Somewhat True, or Certainly True. The SDQ has adequate psychometric properties with internal consistency of .68 and test-retest reliability of .62. |
baseline, 1 year, 2 years |
|
Primary |
Student Externalizing Behaviors |
Nine items from the Youth Self-Report (YSR; Achenbach, 1991) will be used to measure student externalizing behaviors. The YSR asks students now or within the past 6 months, how often they had the following problems: Hangs around bad kids; Swears; Uses alcohol or drugs; Truant; Runs away; Lies or cheats; Prefers older kids; Doesn't feel guilty. Each item has three categories: 0 = not true or not at all, 1 = sometimes or somewhat true, or 2 = very true or often. The internal consistency of these items was a = .68 (Song et al., 1994). |
baseline, 1 year, 2 years |
|
Primary |
ISLA Acceptability |
The Primary Intervention Rating Scale (PIRS) will be used as a measure of the feasibility, usability, and acceptability of the ISLA intervention by gathering educator perceptions of the social validity of intervention goals and procedures (Lane et al., 2002). The PIRS is a 17-item survey completed on a Likert scale with allowable responses of Strongly Disagree, Disagree, Slightly Disagree, Slightly Agree, Agree, and Strongly Agree. The PIRS has adequate psychometric properties, with internal consistency of .97, and the correlation between schools' PIRS scores and treatment integrity scores was r = .71 (Lane et al., 2009). To the PIRS we have added two items relating to sustainment, and four open-ended items that have been applied in previous research (Furjanic et al., 2021). |
1 year and 2 years for intervention condition schools |
|
Primary |
ISLA Implementation Fidelity |
The fidelity if ISLA implementation will be measured using student-level ISLA fidelity tracking sheets (maintained by the ISLA support staff member in each school), and the school-wide ISLA Implementation Checklist (completed by the school team). The ISLA fidelity tracking sheet is used to record information about each student that accesses the ISLA support room, and the ISLA support staff completes Yes/No questions about ISLA delivery for each student. |
baseline, 1 year, 2 years |
|