DOI: 10.3886/icpsr37908.v1
title: Interconnecting Positive Behavioral Interventions and Supports (PBIS) and School Mental Health to Improve School Safety, South Carolina and Florida, 2013-2020
title: Version 1

description: Bullying, fighting, and other forms of interpersonal violence occur frequently in elementary schools, and are associated with student distress, poor school functioning, and increases in aggression, delinquency, and other behavior problems. Positive Behavioral Intervention and Supports (PBIS) is a holistic, multi-tiered, evidence-based approach for preventing and reducing aggression and other problem behavior in school. However, the majority of PBIS schools struggle with more intensive interventions, which many students who present aggressive and disruptive behaviors need. School mental health (SMH) offers promise for addressing these limitations in PBIS. However, SMH lacks an implementation structure and as a result a student must effectively be at a crisis level to be referred for services. Because PBIS and SMH have operated separately, the impacts of both initiatives have been limited. To address these limitations, the Interconnected Systems Framework (ISF) has been developed by leaders from national centers for both initiatives, providing specific guidance on PBIS-SMH interconnection through effective teams, data-based decision making, implementation support for evidence-based practices, and ongoing quality improvement to assure responsiveness to school and student needs. Involving partnerships with school districts and community mental health agencies in two school districts located in South Carolina and Florida, 24 schools implementing PBIS with fidelity were randomly assigned to the three conditions: the ISF, PBIS and SMH, or PBIS alone (8 schools per condition). Data were collected from school records, teacher and student reports, and school implementation teams. The impacts of ISF were compared to the other two conditions on school climate and safety, student exposure to violence, problem behavior and discipline problems, and access to and quality of services.
description: There is growing momentum to improve school climate and safety by implementing comprehensive strategies that address the underlying causes of misbehavior, including exposure to violence and other mental health issues, and to replace punitive disciplinary practices with efforts to help students develop prosocial skills and behaviors. Two primary approaches include Positive Behavioral Interventions and Supports (PBIS) and School Mental Health (SMH). PBIS is a holistic, multi-tiered, evidence-based approach for preventing and reducing aggression and other problem behavior in school through the implementation of universal prevention (Tier 1) for all children, targeted intervention (Tier 2) for children at risk or showing early signs of problems, and intensive interventions (Tier 3) for children and youth with more significant problems. Despite widespread adoption, in general, intervention effects of PBIS have been modest. Two key limitations are that the majority of PBIS schools struggle with intensive interventions at Tier 2 and Tier 3 and the emphasis is on behavior with limited attention on issues that can exacerbate problem behavior such as depression, anxiety, emotional dysregulation, and trauma. SMH services have been shown to significantly improve access to care and early identification and intervention, as well as improvements to student outcomes and positively influence school safety. However, SMH lacks an implementation structure resulting in students effectively needing to be at crisis level to be referred for services. The Interconnected Systems Framework (ISF) addresses limitations of PBIS and SMH in addressing school safety by providing specific guidance on their systematic interconnection. Key components of ISF emphasize interdisciplinary collaboration, the functioning of teams, improving data-based decision making, and improving the selection and implementation of evidence-based practices. The purpose of this longitudinal study was to evaluate the contribution of the ISF in improving school safety, school climate, behavioral and discipline problems, and school outcomes in students. This randomized controlled study was the first experimental evaluation of the ISF's contributions to school and student safety and functioning above the effects of PBIS alone or PBIS and SMH clinicians operating in normative co-located fashion.
description: School-wide assessment of teacher/staff reports of school climate and safety and student records were collected for all students in grades K-5. Data were entered, scored, and maintained by the school year (June 15th to June 15th annually). Student-level analyses focused on students who entered the study at the end of 4th grade, and had post-assessment at the end of 5th grade, and follow-up assessments at the end of 6th grade. For the 2016-2017 school year, there were 15,649 students (46.5 percent White) in schools across all three conditions and one combined condition. There were 3,773 students in Positive Behavioral Interventions and Supports (PBIS) only schools (49.3 percent White); 5,802 students in PBIS and School Mental Health (SMH) schools (36.2 percent White); 5,627 students in Interconnected Systems Framework (ISF) schools (57.6 percent White); and one school that changed from PBIS only in year 1 to PBIS+SMH in year 2 with 447 students (15.4 percent White). There were 4,789 4th and 5th grade students (49.34 percent White, 49.84 percent male). Of teachers/school staff who completed the universal teacher survey for the 2016-2017 school year, 91.32 percent were female (N=682), and of the 678 who indicated race, 79.02 percent were White. For the 2017-2018 school year, there were 14,978 students (48.1 percent White) across all three conditions and one combined condition. There were 3,613 students in PBIS only schools (49.3 percent White); 5,842 students in PBIS+SMH schools (40.7 percent White); 5,072 students in ISF schools (58.6 percent White); and the one school that added SMH to PBIS in Year 2, with 451 students (17.8 percent White). There were 4,572 4th and 5th grade students (48.18 percent White, 50.26 percent male). Of teachers/school staff who completed the universal teacher survey for the 2017-2018 school year, 88.45 percent were female (N=708), and of the 698 who indicated race, 78.26 percent were White. <strong>Faculty/staff measures:</strong> School staff surveys were collected during the 2015-2016 (pre-intervention), 2016-2017 (post-intervention 1), and 2017-2018 (post-intervention 2) school years. All school staff were invited to participate in the schoolwide surveys. Teacher perceptions of school climate were assessed using scales from the Authoritative School Climate Survey (ASCS), including teacher perceptions of fairness, student willingness to seek help, teacher respect for students, student affective engagement, student cognitive engagement, and extent of teasing and bullying. Teacher perceptions of school safety were measured using the Safety scale from the teacher version of the Effective School Battery (ESB). <strong>Teacher ratings of selected students:</strong> Teachers completed the Strengths and Difficulties Questionnaire (SDQ) for youth aged 3-17 years, which assessed a range of emotional and behavioral problems (EB) and prosocial behavior. Teachers also completed the Physical Aggression subscale of the New York Teacher Rating Scale (NYTRS), developed for students in 1st - 10th grades for deeper assessment of disruptive behavior disorders. There were two cohorts of students selected, one cohort of students selected prior to ISF implementation (2015-2016) and one cohort selected after ISF implementation (2016-2017). <strong>Student ratings:</strong> Student surveys were collected from two cohorts of students starting in 4th grade. The first cohort of students completed the surveys starting in the 2015-2016 school year, prior to the beginning of the data collection. The second cohort completed the surveys starting in 2016-2017, after intervention began. Student survey responses were collected from students in all 24 schools. Like the teacher version, the student SDQ assesses EB problems and prosocial behavior. In addition, students evaluated their satisfaction with the most recent mental health services received using the Youth Satisfaction Questionnaire (YSQ). Student perceptions of school climate were measured using scales from the ASCS, assessing perceptions of fairness and strictness of discipline, supportiveness of teachers, affective engagement, and the extent of bullying and teasing. Students also completed items from the Exposure to Violence Screening Measure (EVSM), reflecting recent exposure to violence (e.g., robbery, assault, shooting). <strong>School record data:</strong> School records were retrieved on all students in grades K-5 for three years prior to implementation, as well as each year of the study. Variables collected included demographic data, attendance, behavior, educational placement/disability status, and course performance/grades. <strong>PBIS team variables:</strong> All PBIS teams (average size of 8 members) for the three conditions were asked to record the following during each team meeting: names/disciplines of participants, meeting duration, and which students received Tier 2 or Tier 3 services and the types of services they received. At the beginning and end of the study intervention years, teams completed the Benchmark for Advanced Tiers (BAT) and the Tiered Fidelity Inventory (TFI) to measure the quality of team processes and delivery and refinement of evidence-based practices in each tier. <strong>Fidelity and cost data:</strong> In ISF schools, implementation fidelity was evaluated through data collected for each team meeting and use of the ISF Implementation Inventory (ISF-II) in the fall and spring of intervention years. The cost data reflects the costs of implementation for the academic year June 2017-2018 and are based on actual expenditures. Cost data were assembled from study financial records and interviews with study investigators.
description: Not applicable.
description: Response Rates: Due to the systemic level intervention, which is considered a routine educational practice, all students who required an intervention condition received the one assigned to their respective schools. The subset of students completing questionnaires were recruited by an opt-out procedure. The opt-out rate ranged between 4 percent and 10 percent depending on cohort and school district. For Cohort 1 students who completed the baseline assessment in Spring 2016, 17 percent did not complete measures in Spring 2017 (post-assessment), and 22 percent did not complete measures in Spring 2018 (follow-up assessment). For Cohort 2 students who completed the baseline assessment in Spring 2017, 26 percent did not complete measures in Spring 2018, and 22 percent did not complete measures in Spring 2019.
description: Presence of Common Scales: Authoritative School Climate Survey (ASCS) Behavioral and Emotional Screening System (BESS) Benchmark for Advanced Tiers (BAT) Bully-Victimization Scale (BVS) Effective School Battery (ESB): Teacher version, Safety scale Exposure to Violence Screening Measure (EVSM) Interconnected Systems Framework Implementation Inventory (ISF-II) New York Teacher Rating Scale (NYTRS): Physical Aggression subscale Receipt and Satisfaction with Services Scale Racial/Ethnic Bullying Scale Strengths and Difficulties Questionnaire (SDQ) Tiered Fidelity Inventory (TFI) Youth Satisfaction Questionnaire (YSQ)
description: Two large school districts in the Southeastern United States (South Carolina and Florida) were recruited for this study, with 12 elementary schools from each participating. Participating schools were selected by meeting the following criteria: served students kindergarten through 5th grade, implemented PBIS with fidelity the previous school year, and did not have SMH presence prior to the study. In each district, four schools were randomly assigned to one of three conditions: PBIS only, PBIS+SMH, or ISF. The intervention phase was implemented for two consecutive years, involving two cohorts of students. All students attending schools receiving an intervention condition received the intervention assigned.
description: Datasets: DS0: Study-Level Files DS1: 2015-2016 Student Report Data DS2: 2016-2017 Student Report Data DS3: 2017-2018 Student Report Data DS4: 2018-2019 Student Report Data DS5: 2015-2016 Teacher Report Data DS6: 2016-2017 Teacher Report Data DS7: 2017-2018 Teacher Report Data DS8: 2018-2019 Teacher Report Data DS9: 2013-2014 School Records Data DS10: 2014-2015 School Records Data DS11: 2015-2016 School Records Data DS12: 2016-2017 School Records Data DS13: 2017-2018 School Records Data DS14: 2018-2019 School Records Data DS15: 2016-2017 Behavioral and Emotional Screening System Data DS16: 2017-2018 Behavioral and Emotional Screening System Data DS17: 2016-2017 Team Meeting Form Data DS18: 2017-2018 Team Meeting Form Data DS19: 2015-2016 Staff Survey Data DS20: 2016-2017 Staff Survey Data DS21: 2017-2018 Staff Survey Data DS22: 2016-2017 Intervention Receipt Form Data DS23: 2017-2018 Intervention Receipt Form Data DS24: 2015-2016 Benchmarks of Quality Data DS25: 2016-2017 Benchmarks of Quality Data DS26: 2017-2018 Benchmarks of Quality Data DS27: 2016-2017 Tiered Fidelity Inventory Data DS28: 2017-2018 Tiered Fidelity Inventory Data DS29: Implementation Costs Data DS30: Interconnected Systems Frameworks-Implementation Inventory Fall 2016 Data DS31: Interconnected Systems Frameworks-Implementation Inventory Spring 2017 Data DS32: Interconnected Systems Frameworks-Implementation Inventory 2017-2018 Data
description: Elementary and middle school aged youth in South Carolina and Florida and school staff members who support them. Smallest Geographic Unit: State
description: on-site questionnaire; web-based surveyThis study is also known to researchers and collaborators as the Project about School Safety or PASS Project.The collection includes a zipped package available with restricted access that contains 21 SPSS syntax files. Please see the Secondary Data Analyst User Guide and ICPSR README for Syntax Files for additional information.The qualitative data from focus group and key informant interviews collected for this study are not available as part of the data collection at this time.