DOI: 10.3886/icpsr37896
title: Project SOARS: Student Ownership, Accountability, and Responsibility for School Safety, Illinois and Oregon, 2016-2020
title: Archival Version

description: Project SOARS (Student Ownership, Accountability, and Responsibility for School Safety) utilized a mixed-methods study design to develop and test a student-centered and technology-driven school safety framework to address peer victimization, violent behavior, and student reluctance to share critical safety information within high school settings. SOARS was a project of IRIS Media, Inc. and consisted of 4 phases implemented between 2016 and 2020. Activities for each phase were carried out in Oregon and Illinois high schools in order to facilitate inter-site replication of outcomes. Phase 1 was conducted in 2016 and consisted of focus groups and key informant interviews with students, school personnel, and parents to gather perceptions of current school safety practices. Phase 2 was undertaken in 2017 and asked students, school personnel, and parents to assess the acceptability and usability of prototypes of the SOARS framework. The SOARS framework consisted of (a) the <em>Advocatr</em> mobile app, which allowed students to report positive and negative behaviors in their school environment; (b) a 9-week curriculum engaging students with the concepts of student ownership of school safety, advocacy/self-advocacy, physical and emotional safety, and restorative conflict resolution; (c) informational briefs for school personnel and parents about the framework components and their rationale; and (d) guidelines for a student-led school-wide safety campaign. Phase 3 was rolled out in 2018 and 2019 and consisted of feasibility testing conducted with a small subset of teachers and students in those teachers' classrooms. Participants were surveyed before and after implementation of the SOARS framework. The focus of the feasibility test was on student access and use of the <em>Advocatr</em> app and the accompanying curriculum. During Phase 4 implementation in 2019 and 2020, researchers conducted a pilot test with students, school personnel, and parents from 4 high schools, 2 assigned to the intervention and 2 to the control condition. The focus of the pilot was to test the effectiveness of the SOARS framework.
Demographic information was collected from all informants and includes gender (sex male or female; transgender identification), ethnicity, and race. Additional demographic information about students includes sexual orientation, approximate age (over/under 18 years), primary language, GPA, and grade. Parent demographics also include education level and student's grade, while school personnel (teachers and staff) also provided information regarding education level, school role, job title, years in current position, grades taught, and subjects taught.
Users should note that qualitative data collected during phase 1 focus groups and phase 2 user acceptance tests are not included in version 1 of the ICPSR release. Additionally, in the quantitative datasets, character variables featuring open-ended string responses have been masked by ICPSR. This study will be updated at a later date to include qualitative data files and character variables in the quantitative datasets.
description: The purpose of Project SOARS was to develop and test the SOARS framework components in authentic high school contexts and to answer the following research questions:
<strong>Focus Groups</strong>

What are the primary student, school personnel, parent, and community stakeholder concerns regarding school safety?
What barriers and facilitators are present to address the stakeholder concerns? And how can these concerns be addressed with existing resources and practices?
<strong>User Acceptance and Feasibility Field Tests</strong>

Is the SOARS framework perceived as useful and feasible for promoting a safe school environment by all stake holders (students, teachers, administrators, and parents)?
Is SOARS perceived as easy to implement by all stakeholders (students, teachers, administrators, and parents)?
<strong>Pilot Tests</strong>

Does SOARS implementation result in reduced bullying/harassment, increased students' ownership of school safety, increased student responsibility for reporting behaviors of concern, and increased student accountability for problem-solving behavioral incidents?
Does SOARS implementation result in increased teacher and parent perceptions of school safety?
description: This project employed a mixed method study design. Monetary incentives were offered to participating schools and individuals at various points during project implementation.
<strong>Focus Groups:</strong> Researchers conducted 2 waves of focus groups in 2016 with students, school personnel, and parents. Focus groups met for about 2 hours and were facilitated by a project team member. The sessions were audio recorded with the consent of each participant and detailed field notes were taken. Focus group audio recordings were transcribed by undergraduate research assistants.
<strong>User Acceptance Tests:</strong> Researchers conducted 3 waves of user acceptance tests in fall 2017, winter 2018, and spring 2018 to receive feedback from users on all SOARS framework components. Separate sessions were conducted for each participant group and lasted about 2 hours. The session facilitator introduced the project and provided participants with the prototype materials. Participants completed a paper and pencil survey and were asked to score the usability and acceptability of each framework component.
<strong>Feasibility Field Tests:</strong> Researchers utilized a quasi-experimental design to collect measures at pre and post with participants serving as their own controls. At the beginning of the year, teachers attended a 2-hour orientation during which project personnel familiarized them with the project and the consent and data collection procedures.
<strong>Pilot Tests:</strong> At each site, 1 of the participating schools was assigned to receive the intervention and the other school served as control. Schools in the intervention condition received access to all SOARS framework components, while schools in the control condition conducted business as usual. After researchers collected pre data at the beginning of the school year, all students in the intervention schools whose parents did not opt them out of the study were onboarded into the <em>Advocatr</em> app and received log in information from the school-based and project-supported SOARS coordinator. The participating teachers implemented the curriculum and encouraged students to participate in the school-wide safety campaign. Students mounted campaigns under the supervision of school staff. Parents, all staff, and all students had access to the informational materials posted to the <em>Advocatr</em> website.
<em>Note:</em> The 2019-2020 school year was affected by school closures related to the Covid-19 pandemic. All participating pilot schools, intervention as well as control, were affected by the school closure. At both sites, schools closed in March and students and teachers engaged in remote instruction for the remainder of the school year. All participating schools remained in the study and completed post data collection on schedule. Response rates at the end of the year were likely affected by the school closures.
description: <strong>Focus Groups:</strong> Participant data was collected during 2 waves of focus groups to inform the development of the <em>Advocatr</em> app and the implementation of the SOARS school safety framework. Focus group questions were structured around school safety concerns and understanding how students feel most comfortable reporting. Student data collected were read multiple times with the goal of identifying similar phrases and themes that were later grouped to form themes and meaning. The majority of themes and respective codes were derived from the data (e.g., definitions of snitching vs. reporting) and a few themes were derived from the existing snitching literature (e.g., community characteristics and police activity). Parent and Staff data were analyzed using open and selective coding. During selective coding, the emergent themes were organized in the overarching categories of: 1) teacher social emotional competence and 2) barriers or facilitators to physical and emotional safety.
<strong>User Acceptance Tests:</strong> In Wave 1, researchers tested the usability of the <em>Advocatr</em> app and the content of the scripts for the instructional videos with students, school personnel, and parents. In Wave 2, researchers tested the usability of the instructional activities and accompanying 1-page "Did-You-Know" informational briefs with school personnel. In Wave 3, researchers tested the usability of the instructional materials, the accompanying "Did-You-Know" informational briefs, and components of the safety campaign materials with students, teachers, and parents. Wave 1 surveys focused on <em>Advocatr</em>'s technology/user interface, content, and implementation feasibility, and the video scripts' informational value, clarity, and relevance. Wave 2 surveys focused on the contextual fit, relevance, and engagement value of the curricular materials and informational briefs. Wave 3 surveys focused on the engagement value of the videos and informational briefs as well as the contextual fit and relevance of the safety campaign guidelines. All survey items were scored on a 4-point Likert-type scale ranging from 1 = "strongly disagree" to 4 = "strongly agree." Participants could also choose "not applicable" and had the option to provide additional comments.
<strong>Feasibility Field Tests:</strong> Participating teachers completed the <em>Teacher and Staff School Environment Survey</em> (Espelage et al., 2014). The survey consists of 35 items measuring six domains: (1) Student Likely to Intervene (5 items; e.g., "A student would intervene if another student is making fun of and teasing another student who is obviously weaker"; response options are 1 (Very Unlikely) through 4 (Very Likely); (2) Staff Likely to Intervene (5 items; e.g., "A staff member would intervene if another student is making fun of and teasing another student who is obviously weaker"; response options are 1 (Very Unlikely) through 4 (Very Likely); (3) Aggression as a Problem in the School (5 items; e.g., "How much of a problem is 'Students picking fights with other students'"; response options are 1 (Very Rarely) through 4 (Very Much); (4) Administration Commitment to Bully Prevention (8 items; e.g., "Your school is developing policies or programs to prevent bullying"; response options are 1 (Not Much) through 4 (Quite a Lot); (5) Positive Teacher-Staff Interactions (7 items; e.g., "Teachers and staff in this school usually get along with students."; response options are 1 (Strongly Disagree) through 4 (Strongly Agree); and (6) Gender Equity/Intolerance of Sexual Harassment (5 items; e.g., "Sexual harassment is not tolerated at this school"; response options are 1 (Strongly Disagree) through 4 (Strongly Agree). Teachers also completed the <em>Behavior Problem Index (BPI)</em> (Moore et al., 2002) for each participating student measuring the frequency with which students engaged in negative or disruptive behaviors such as bullying others, cheating and lying, or having difficulty concentrating. Response options were 0 (Never True), 1 (Sometimes True) (1), and 2 (Often true) on a 3-point Likert type scale. Participating students completed the <em>Safe School Survey</em> (Skiba et al. 2006). The survey consists of 42 items that assess student perceptions across 4 scales: (1) Personal Safety, (2) Incivility/Disruption, (3) Delinquency/Major Safety, and (4) Connection to school/Positive School Climate. Students were asked to indicate the extent to which they agreed with each statement. Response options ranged from 1 (Strongly Disagree) through 5 (Strongly Agree) on a 5-point Likert-type scale. Students also completed the <em>Peer Experiences Questionnaire</em> (Vernberg et al., 1999). The questionnaire consists of 18 items that assess students' experiences as both victims and perpetrators of different types of anti-social behaviors at school (e.g., hitting, rumor spreading, intimidation, threats, social exclusion). 9 items assessed perpetration (e.g., student engaging in these behaviors towards peers) and 9 items assess victimization (e.g., student experiencing these behaviors from peers). Parents completed the BPI for their participating child. After pre-data collection was complete, the school-based and project-supported SOARS coordinator onboarded all consented students into <em>Advocatr</em> and provided students with their login information. Teachers then taught 1 weekly lesson of the curriculum. At the end of the school year, researchers asked students, teachers, and parents to complete the same measures as at the beginning of the school year in addition to a consumer satisfaction questionnaire. Researchers also collected <em>Advocatr</em> app usage data to assess the extent to which students availed themselves of the reporting tool. Differences in student outcomes were examined across grade levels, gender, race, and sexual orientation.
<strong>Pilot Tests:</strong> As in the field test, students completed the <em>Safe School Survey</em> and the <em>Peer Experiences Questionnaire</em>. All school staff completed the <em>Teacher and Staff School Environment Survey</em>. Parents completed the <em>Parent School Environment Survey</em>, which was identical to the teacher measure but focused on the perspective of parents. Researchers examined <em>Advocatr</em> use by overall frequency as well as across student race and gender.
Please see the published Final Summary Overview for a full list of references.
description: Not applicable.
description: Response Rates: Not available.
description: Presence of Common Scales: Several Likert-type scales were used.
description: <strong>Focus Groups:</strong> Participants for each focus group were selected using convenience and purposive sampling to recruit a diverse pool of students from all grade levels and representing a diverse group of demographics from 4 participating schools.
<strong>User Acceptance Tests:</strong> Researchers recruited students, school personnel, and parents from 2 high schools, 1 in Oregon and 1 in Illinois for a total of 3 waves of user acceptance tests. Students and parents participated in Wave 1 and Wave 3 testing and school personnel participated in all 3 waves.
<strong>Feasibility Field Tests:</strong> In collaboration with school administrators, researchers recruited classroom teachers to select 1 of their classrooms for participation in the study. Teachers reached out to students in the participating classroom and their parents to obtain parental consent for students to participate and to recruit parent participants.
<strong>Pilot Tests:</strong> All students in the intervention schools who provided passive parental consent obtained access to <em>Advocatr</em>. Researchers recruited 10 staff from each intervention school to deliver the curriculum to students. In Oregon, those staff included 3 classroom teachers and the Positive Behavior Interventions and Supports (PBIS) coordinators, dean of students, campus monitors, leadership advisor, and in-school-suspension room supervisor. In Illinois, 10 English teachers delivered the curriculum to students. All students and staff from intervention and control schools were encouraged to complete the student and staff surveys.
description: Datasets: 
DS0: Study-Level Files
DS1: Students Feasibility Field Test
DS2: Teachers Feasibility Field Test: Behavior Problem Index
DS3: Teachers Feasibility Field Test: School Climate
DS4: Students Pilot Test
DS5: Staff Pilot Test
DS6: Parents Pilot Test
description: Students, teachers, school staff, administrators, and parents at high schools in Illinois and Oregon. Smallest Geographic Unit: State
description: coded on-site observation; mixed mode; on-site questionnaireQualitative data collected during phase 1 focus groups and key informant interviews and phase 2 user acceptance tests are not included in version 1 of the ICPSR release. Additionally, in the quantitative datasets, character variables featuring open-ended string responses have been masked by ICPSR. This study will be updated at a later date to include qualitative data files and character variables in the quantitative datasets.There are several discrepancies between the number of
participants reported across phases in the published Final Summary Overview and the actual number of cases in the datasets received by ICPSR. No additional information was provided.Datasets feature data collected at different times. Users should note that both "wave" and "time" are used to differentiate collection times in the study documentation and variable names. For instance, variable names in datasets 1, 4, 5, and 6 use wave (W1, W2), dataset 2 uses time (T1, T2), and dataset 3 uses both wave (W1, W3) and time (T2). Please refer to variable names for wave/time distinctions.