DOI: 10.3886/icpsr38572.v1
title: Positive Behavior Interventions and Supports (PBIS) in Challenging Contexts: Evaluating a Replicable Implementation Approach in Philadelphia, Pennsylvania, 2018-2021
title: Version 1

description: This study aimed to validate an approach to implementing Positive Behavior Interventions and Supports (PBIS), a school-climate improvement program, in a partnering school district in an attempt to address pressing issues impacting students' success. The study was a collaboration between the Consortium for Policy Research in Education (CPRE), the partnering school district, and several additional partners, who collaboratively designed and supported the PBIS implementation approach funded from National Institute of Justice (NIJ) and supplemental funds from the participating school district.
The impacts of trauma training for school officers were assessed via an PBIS of all eligible district K-8 schools, including schools that did and did not use school-wide PBIS. The random controlled trial (RCT) used a blocked design to assign schools to treatment and control conditions within PBIS status (i.e., PBIS school, non-PBIS school).
Impacts of the implementation model were assessed via a cluster randomized experiment. To select schools to receive the expanded supports being provided through this project, the research team collaborated with the partnering school district's school climate administrators and project partners who had been supporting PBIS implementation in the district for several years prior to the start of the project. Criteria were established to determine eligibility for the implementation model, including that eligible schools must have: 1) strong PBIS implementation at Tier 1, 2) attempted implementing some form of Tier 2 supports, and 3) leadership buy-in. Applying these criteria, nine potential schools were identified as eligible. From this list, four schools were randomly selected to receive the implementation model and the remaining five schools served as Comparison schools and conducted business-as-usual PBIS.
description: The goals of the study were to assess the effectiveness of two different interventions with different intensities and resource requirements:
Brief trauma trainings for school safety officers. This series of trainings was delivered to school officers in the treatment group during the 2018-2019 and 2019-2020 school years. The impacts of this intervention were assessed via an RCT, where officers from eligible schools were randomly assigned to either receive the trauma trainings or to continue with business-as-usual. Approximately half of the schools in each condition were participants in the partnering school district's PBIS initiative.
Expanded climate supports, including intensive whole-staff training and ongoing coaching in trauma and schoolwide PBIS; universal screening for students; and expanded support for Tier 2 implementation. The impacts of this implementation model were assessed via a cluster randomized experiment, with the intervention delivered to classrooms nested in 4 PBIS schools randomly selected from a list of 9 eligible schools. The remaining 5 PBIS schools served as a comparison group.
The study addressed the following objectives:Assess the impacts of trauma-awareness trainings for school safety officers in K-8 schools in the partnering school district on school officers': 1) knowledge and attitudes regarding trauma and trauma-informed practices, and 2) reported behaviors (i.e., use of trauma-informed practices) in their interactions with students.
Assess the implementation (including acceptability, appropriateness, and feasibility) and impacts of an expanded, trauma-informed PBIS implementation model in 4 K-5 or K-8 schools on teacher outcomes and student outcomes in the domains of attendance, discipline, and academics, relative to a business-as-usual comparison group of schools.Assess the costs of the expanded, trauma-informed PBIS implementation model components.
description: The impacts of trauma training for school officers were assessed via an random controlled trial (RCT) of all eligible district K-8 schools, including schools that did and did not use school-wide Positive Behavior Interventions and Supports (PBIS). The RCT used a blocked design to assign schools to treatment and control conditions within PBIS status (i.e., PBIS school, non-PBIS school). Impacts of the implementation model were assessed via a cluster randomized experiment. To select schools to receive the expanded supports being provided through this project, the research team collaborated with the partnering school district's school climate administrators and project partners who had been supporting PBIS implementation in the district for several years prior to the start of the project. Criteria were established to determine eligibility for the implementation model, including that eligible schools must have: 1) strong PBIS implementation at Tier 1, 2) attempted implementing some form of Tier 2 supports, and 3) leadership buy-in. Applying these criteria, nine potential schools were identified as eligible. From this list, four schools were randomly selected to receive the implementation model and the remaining five schools served as Comparison schools and conducted business-as-usual PBIS.The research team collected survey data from K-5 teachers at Demonstration and
Comparison schools in the Fall and Spring of the 2019-20 and 2020-21 school years, as well as
conducted semi-structured interviews with Demonstration school stakeholders in Spring 2020
and Spring 2021. Survey and interview data from Demonstration school teachers collected each
Spring were used to assess the implementation of the expanded, trauma-informed PBIS
implementation model. Additionally, survey data from Demonstration and Comparison
school teachers across time-points were used to examine changes in teachers' beliefs, self-efficacy,
intentions, and self-reported behaviors
Administrative data was also requested and received from the partnering school district at the classroom level for the four Demonstration and five Comparison schools. These data were used to make comparisons between the two sets of schools on a range of student demographic, attendance, climate, disciplinary, and academic outcomes
description: Response Rates: School Safety Officer Survey Dataset: N/A
Administration Dataset: Spring 2020 - 55.4%; Spring 2021 - 70.5%
Teacher Survey Dataset: Fall 2019 - 62.3%; Spring 2020 - 70.8%; Fall 2020 - 70.8%; Spring 2021 - 68.2%
description: School Safety Officer Survey Dataset: Participants (N = 97) most often identified as male (54%), Black or African
American (65%), and between the ages of 30 and 60 years (82%).
Teacher Survey Dataset: The teacher survey was administered to teachers in both Demonstration and Comparison schools in Fall and Spring of the 2019-20 and 2020-21 school years. At each time point, all K-5 teachers in the Demonstration and Comparison schools were invited by email to participate in the voluntary, online survey. Across both timepoints, 301 surveys were sent, and we received a total of 180 responses, a rate of 59.8%.
Administration Dataset: Administrative data was provided by the school district for all K-5 students,
aggregated at the classroom level, within the nine participating schools (4 Demonstration, 5
Comparison) for the academic years 2016-17, 2017-18, 2018-19, 2019-20, and 2020-21.
description: Datasets: 
DS0: Study-Level Files
DS1: School Safety Officer Survey Dataset
DS2: Administrative Dataset
DS3: Teacher Survey Dataset
description: School safety officer random controlled trial (RCT) surveys: Any officer assigned to a study school (treatment or control) between 2018 and 2020. Implementation model RCT surveys: All K-5 teachers in treatment and comparison schools.
Implementation model RCT interviews: All principals and coaches in treatment schools; teachers from treatment schools representing different grade levels..
description: face-to-face interview; on-site questionnaire; web-based surveyFor additional information on the Positive Behavior Interventions and Supports (PBIS) in Challenging Contexts Study, please visit the Positive Behavior Interventions and Supports (PBIS) in Challenging Contexts: Evaluating a Replicable Implementation Approach in Philadelphia, Pennsylvania, 2018-2021 website.