DOI: 10.3886/icpsr38431.v1
title: Student and Staff Comprehension of Emergency Operations Plans, United States, 2018-2021
title: Version 1

description: In this study, the research team investigated emergency operations plans (EOPs) content and comprehension in a purposive sample of 10 U.S. schools using a phased, mixed-methods study design. The four primary goals of the study were to:

Gain access to EOPs for 10 schools and examine their
appearance, layout, and content, and empirically document the comprehensiveness
of EOP materials according to federal guidelines;
Assess access to emergency planning efforts and
perceptions of emergency preparedness, including to what extent different types
of staff members have read and received training on their school's EOP, serve
on emergency planning or school crisis response teams, and believe that their
school has prepared them for a violent event (e.g., an armed intruder incident);
Assess staff and student comprehension of emergency
concepts and protocols and identify areas of high and low comprehension and
respondent- and school-level correlates of comprehension; and
Understand from the perspectives of staff,
students, district representatives, local law enforcement officials, and other
key stakeholders how EOPs and school emergency preparedness more broadly could
be improved and what are the most pervasive challenges and vulnerabilities in
school emergency preparedness efforts.
The study was conducted in four phases. In Phase 1, the team recruited 10 schools and gained privileged access to their EOPs. A comprehensive rubric was developed based largely on guidance put forth in 2013 by six federal agencies, including the United States Department of Education, Department of Health and Human Services, Department of Homeland Security, Department of Justice, Federal Bureau of Investigation, and the Federal Emergency Management Agency. Each EOP was systematically reviewed using this rubric; school-specific and aggregated analyses were conducted to identify common strengths and limitations of the plans. In Phase 2, leveraging insights gained from school-specific analyses of EOPs, the team developed and administered comprehension surveys for staff and students to evaluate the extent to which each school community was knowledgeable of the concepts, protocols, and other details described in their plans. Following survey data collection, the team conducted extensive analyses to identify areas with high and low levels of comprehension and uncover statistical associations between comprehension and respondent characteristics (e.g., staff type, years employed at the school, perceptions of preparedness).
In Phase 3, for a subset of schools, the team conducted site visits and group interviews with students and different types of staff regarding their perceptions of their school's EOP, their school's vulnerability to extreme violence, and how emergency planning and preparedness could be improved. Finally, in Phase 4, the team analyzed and synthesized the results from each data collection activity to draw conclusions about EOP development and emergency preparedness and develop actionable recommendations for enhancing safety efforts in K-12 educational settings.
<b>Qualitative data (interviews, focus groups, drill observations) are not currently available for this collection.</b>
description: The purpose of this study was to build a knowledge base on school emergency operations plans (EOPs) in the United States and to inform where additional efforts may be needed to ensure students and staff are ready to respond to emergency situations.
description: This study used a mixed-methods phased data collection. Each phase's design is detailed below.
<b>Recruitment. </b>The current study's recruitment strategy leveraged a sample of districts recruited in a prior study conducted by the same project team (see ICPSR 37222 for sampling and methodology). All respondents of the previous study were asked if there were schools in their district with model emergency operations plans (EOPs). Respondents from these model EOP schools were contacted and asked to provide contact information for head administrators. Study team members sent introductory emails about the study asking for participation, offering a $250 check to the school, as well as a $250 gift card for a staff member willing to serve as a coordinator, in compensation. If the administrators expressed interest, the team conducted a screening call. Seven schools were recruited using this method. To recruit additional schools, districts that were not classified as having model EOPs were asked for help in recruiting a school to participate in the study. Recruitment letters were sent to an additional random sample of 500 districts who did not respond to the original district survey in the previous study, prioritizing those who had recently enacted an emergency protocol. Three schools were recruited in this fashion.
<b>EOP reviews and assessment. </b>The project team developed a rubric to review each school's EOP, which was reviewed by project consultants for expert feedback. The rubric was based on federal guidance due to significant differences in how states approach school emergency preparedness. A research team member with extensive experience in school safety research, qualitative coding, and document reviews conducted primary reviews of each EOP, and the P.I. conducted a secondary evaluation to verify the results. Each school received a comprehensive EOP review report with their scores for each section.
<b>Staff surveys. </b>Customized surveys for each school based on their EOP were developed, prioritizing items that were applicable to all EOPs (e.g., lockdown, evacuation, shelter-in-place). Surveys were programmed and administered as web surveys to all school staff, via email. School coordinators and administrators sent emails requesting staff participation and posted recruitment flyers at the school. A total of 585 staff completed surveys. Participants were compensated with a $20 gift card.
<b>Student surveys. </b>The research team worked with a coordinator from each school to randomly select 6-8 distinct classrooms that would provide a sample of at least 100 students per school. Paper surveys were administered to students in person on a prearranged date. A total of 1,326 students completed surveys.
<b>Site visits: Individual interviews, focus group interviews, and observations. </b>In 2019 and early 2020, study team members conducted 1-day to 2-day site visits with 4 schools. The team observed 1 evacuation drill and 1 lockdown drill in total during these visits. Student focus groups were organized by grade level. Staff focus groups were organized by job role (e.g., teachers, health services, administration, support staff, food service, custodians, school resource officers, district safety officers), including officers from local police and fire departments. Interviews were conducted in private rooms, audio-recorded, and lasted between 45 and 60 minutes. Due to the COVID-19 pandemic, plans for site visits were modified to virtual interviews in 2021, but only 1 school was able to accommodate this request. Three schools requested to terminate their study participation due to external challenges. 40 separate interview sessions were conducted (32 in-person, 8 virtual), with 29 focus groups and 11 individual interviews. 162 total respondents participated: 58 students, 6 district-level safety officers, 5 school resource officers, 6 police department staff, 12 fire department staff, and 75 school staff. Staff members were compensated with a $30 gift card. Students were compensated with a $10 gift card.
description: Quantitative datasets contain the same school ID (<b>SCHOOLID</b>) for data merging.
<b>Survey data. </b>School-level items include indicators if the school is considered rural, if it is a middle or high school, school size based on students enrolled, and number of days elapsed since an emergency drill. Demographic items include gender, race, ethnicity, grade level (students), years of experience, and job title (staff). Student surveys included indices gauging school safety (e.g., "I feel safe at my school") and belonging (e.g., "I feel proud of belonging to my school"). Staff surveys included number of safety teams participant worked on, training completed on the school's emergency operations plan (EOP), and perception of readiness for a violent event. EOP comprehension grades are aggregated by subsection and as total.
<b>EOP assessment data. </b>For each plan component, items indicate whether the component was partially/fully satisfied or not satisfied according to the study rubric.
<b>Individual interviews and focus groups. </b>Staff interview topics covered participants' involvement in emergency planning, their knowledge of and perceptions of their school's EOP (including strengths and weaknesses), the school's effectiveness in training/educating staff and students on emergency procedures, level of preparedness to respond to a violent event, and lessons learned from other incidents. Student interview topics included safety measures taken by the school, perceptions of school safety, involvement in emergency planning, and trainings for emergency situations.
description: The Staff Survey Data (DS2) contains the weight variable <b>TOTALSAMPLEWEIGHT</b>. Raking ratio estimation was used to create weights to adjust for differences due to overrepresentation of teachers and administrators in the staff data.
description: Response Rates: Out of 922 total staff across all schools, 585 surveys were completed (63% response rate). Out of 1,684 total students across all schools, 1,326 surveys were completed (79% response rate).
description: The research team purposively sampled schools varying by enrollment size, levels of urbanicity, state, if the emergency operations plan (EOP) was considered a model, and whether the school had conducted a lockdown, lockout, or evacuation in the past few years in response to a threat (real or perceived). Schools that only served elementary school-aged children (K-5) were excluded. The final sample consisted of 10 schools with 9 separate EOPs located in 7 states; 2 schools shared a campus and an EOP.
description: Datasets: 
DS0: Study-Level Files
DS1: Student Survey Data
DS2: Staff Survey Data
DS3: Emergency Operations Plan (EOP) Assessment Data
description: 10 secondary schools located across 8 school districts and 7 states within the United States.
description: coded on-site observation; face-to-face interview; on-site questionnaire; telephone interview; web-based surveyThis study is a companion to ICPSR 37222, School Emergency Preparedness Study, United States, 2018. The recruitment strategy used in the previous study (ICPSR 37222) was also used to sample several schools in the current study.