NCEE 2010-4008 U.S. DEPARTMENT OF EDUCATION

Impacts of a Violence Prevention 

Program for Middle Schools 

Findings From the First Year of Implementation 

Executive Summary 

Impacts of a Violence Prevention 

Program for Middle Schools 

Findings From the First Year of Implementation 

Executive Summary 

April 2010

Authors 

Suyapa Silvia 

Jonathan Blitstein 

Jason Williams 

RTI International 

Chris Ringwalt 

Pacific Institute for Research and Evaluation (PIRE) 

Linda Dusenbury 

William Hansen 

Tanglewood Research, Inc. 

John Rice 

Project Officer 

Institute of Education Sciences 

NCEE 2010-4008

U.S. DEPARTMENT OF EDUCATION 
U.S. Department of Education 

Arne Duncan 

Secretary 

Institute of Education Sciences 

John Q. Easton 

Director 

National Center for Education Evaluation and Regional Assistance 

John Q. Easton 

Acting Commissioner 

April 2010

This report was prepared for the National Center for Education Evaluation and Regional Assistance, Institute of 

Education Sciences, under contract no. ED01CO0052/0015 with RTI International, the Pacific Institute for Research 

and Evaluation (PIRE), and Tanglewood Research, Inc. 

This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While permission to 

reprint this publication is not necessary, the citation should read: Silvia, S., Blitstein, J., Williams, J., Ringwalt, C., 

Dusenbury, L., and Hansen, W. (2010). Impacts of a Violence Prevention Program for Middle Schools: Findings From the First Year 

of Implementation – Executive Summary (NCEE 2010-4008). National Center for Education Evaluation and Regional 

Assistance, Institute of Education Sciences, U.S. Department of Education. Washington, DC. 

IES evaluation reports present objective information on the conditions of implementation and impacts of the programs 

being evaluated. IES evaluation reports do not include conclusions or recommendations or views with regard to actions 

policymakers or practitioners should take in light of the findings in the report. 

To order copies of this report, 

• Write to ED Pubs, Education Publications Center, U.S. Department of Education, P.O. Box 22207, Alexandria, 

VA 22304. 

• Call in your request toll free to 1-877-4ED-Pubs. If 877 service is not yet available in your area, call 800-872-5327 

(800-USA-LEARN). Those who use a telecommunications device for the deaf (TDD) or a teletypewriter (TTY) 

should call 800-437-0833.

• Fax your request to 703-605-6794 or order online at www.edpubs.gov. 

This report is also available on the IES website at http://ncee.ed.gov. 

Alternate Formats 

Upon request, this report is available in alternate formats, such as Braille, large print, audiotape, or computer diskette. 

For more information, call the Alternate Format Center at 202-205-8113. 
Impacts of a Violence Prevention Program for Middle Schools iii

Acknowledgments 

This study represents a collaborative effort among participating schools; staff from RTI 

International (RTI); our colleagues at Pacific Institute for Research and Evaluation (PIRE); 

researchers at Tanglewood Research, Inc.; and the program developers for Responding in Peaceful 

and Positive Ways (RiPP), Wendy Bauers Northup and Aleta Meyer, and the program developer for 

Best Behavior at the University of Oregon, Jeff Sprague. We are especially grateful to the school 

principals, faculty, and students of the participating schools for their dedication to implementing the 

programs and for graciously participating in all data collection. 

At Tanglewood Research, Bill Hansen and Linda Dusenbury and a group of dedicated site 

liaisons supported, coached, and monitored teachers and faculty during the implementation of the 

programs; they also coordinated training and technical assistance for large numbers of staff. The 

program developers were true partners in this effort and were extremely flexible and cooperative 

with the needs of the study as we worked to implement an intervention formed from the two 

programs. 

At RTI, the evaluation was supported by a team of devoted school recruiters, site 

coordinators, and site visitors, including Becky Durocher, Linda Bailey-Stone, Betty Burton, Terri 

Dempsey, Connie Hobbs, Farley Bernholz, Kimrey Millar, Lisa McCaskill, Elizabeth Parish, and 

Linda Pucci. The team oversaw locally based field data collectors who worked tirelessly to 

coordinate evaluation activities and achieve high response rates for the student survey. Data 

collection logistics and training were ably supported by Margaret Searle and Lori Hill, under the 

leadership of Lisa McCaskill and Linda Bailey-Stone. Jon Blitstein led the analyses; Jason Williams 

and Gayle Bieler provided additional support. Early drafts of this report were reviewed by Gayle 

Bieler, Jeremy Bray, Gordon Brown, and Dennis Wallace. 

The study’s Technical Working Group provided helpful advice and guidance on the 

evaluation’s design and analysis. Three panel members also provided expert advice on the selection 

of the programs and conducted extremely helpful reviews of the revised curriculum. We appreciate 

the contributions of Robert Boruch, Thomas Cook, David Cordray, Daniel Flannery, Dennis 

Gorman, Nancy Guerra, Jeffrey Smith, Pamela Orpinas, and William Shadish. 

Impacts of a Violence Prevention Program for Middle Schools v 

Disclosure of Potential Conflicts of Interest 

The research team for this evaluation consists of a prime contractor, RTI International 

(RTI), and two subcontractors, Pacific Institute for Research and Evaluation (PIRE) and 

Tanglewood Research, Inc. RTI and PIRE formed the evaluation team, while Tanglewood Research 

oversaw implementation of the two programs. None of these organizations or their key staff 

members has financial interests that could be affected by findings from the evaluation of the two 

school-based violence prevention programs considered in this report. No one on the Technical 

Working Group, convened by the research team to provide advice and guidance, has financial 

interests that could be affected by findings from the evaluation. 

Impacts of a Violence Prevention Program for Middle Schools vii 

Executive Summary 

This is the first of two reports that summarize the findings from an impact evaluation of a 

violence prevention intervention for middle schools. This report discusses findings after 1 year of 

implementation. A forthcoming report will discuss the findings after 2 years and 3 years of 

implementation. In 2004, the U.S. Department of Education (ED) contracted with RTI 

International (RTI) and its subcontractors, Pacific Institute for Research and Evaluation (PIRE) and 

Tanglewood Research, Inc., to conduct an evaluation of a hybrid intervention model that combines 

a curriculum-based program, Responding in Peaceful and Positive Ways (RiPP [Meyer and Northup 

2002a, 2002b, 2006]), and a whole-school approach, Best Behavior (Sprague and Golly 2005). The 

combined intervention was administered over the course of 3 successive years. Using a randomized 

control trial design (with entire schools randomly assigned either to receive the intervention or not), 

the evaluation assesses the intervention’s effects on student violence and victimization and whether 

these effects vary by levels of student risk. Tanglewood Research, which assisted in the process by 

which the programs under study were selected, provided implementation oversight along with sitebased liaisons and coordinated training and technical assistance for staff in intervention schools. The 

developers of the two programs that constitute the intervention—Prevention Opportunities and 

University of Oregon—provided the program materials and conducted staff training. 

Study Background 

The Office of Safe and Drug-Free Schools (OSDFS), through the Safe and Drug-Free 

Schools and Communities Act (SDFSCA, Title IV-A), supports both state and national programs 

intended to prevent and reduce the levels of drug use and violence in and around schools. State 

grants from SDFSCA provide funding to approximately 97 percent of all school districts for this 

purpose. A study conducted in 2000 found that 90 percent of districts that receive SDFSCA funding 

implemented curricula that target youth violence (Hantman and Crosse 2000). While there is now a 

lengthy set of school-based drug prevention curricula that have been evaluated using rigorous 

designs, much less evidence is available concerning effective violence prevention strategies in school 

settings. 

The need for evidence-based violence prevention programs is particularly critical for middle 

schools, whose students experience the highest rate of school-based violence relative to students in 

other grades. Data from a recent National Center for Education Statistics report indicate that the 

rate of victimization for students aged 12 through 14 was 35 incidents per 1,000 students, which was 

higher than the rate for students aged 15 through 18, which was 23 incidents per 1,000 students 

(Dinkes, Kemp, and Baum 2009). Similarly, data from the School Survey on Crime and Safety 

(SSOCS) for the 2005–06 school year reveal that students were more likely to experience a violent 

event in middle schools (52 per 1,000) than in elementary (25 per 1,000) or secondary (26 per 1,000) 

schools (Nolle, Guerino, and Dinkes 2007). National Crime Victimization Survey (NCVS) data also 

indicate that bullying is a significant problem. In 2005, some 36 percent of students in grades 6 and 

7, and 29 percent of students in grades 8 and 9, reported having been bullied in the previous 

6 months (Dinkes, Cataldi, and Lin-Kelly 2007). 

Among the violence prevention strategies for middle schools which have been evaluated 

through rigorous methods are at least eight curriculum-based programs, including RiPP (Meyer and 

Northup 2002a, 2002b, 2006), one of the two prevention programs selected for this study. Four 

evaluations of RiPP have been conducted by the program’s developers in schools serving students in 
Executive Summary 

viii Impacts of a Violence Prevention Program for Middle Schools 

grades 6 through 8. The evaluations found statistically significant decreases in students’ approval of 

violent behavior, levels of peer provocation, physical aggression, victimization, and discipline 

violations, although results have not been consistently maintained at follow-up time points (Farrell, 

Valois, and Meyer 2002; Farrell, Meyer, and White 2001; Farrell, Meyer, et al. 2003; Farrell, Valois, et 

al. 2003). While evaluations of RiPP and similar curriculum programs have yielded significant results, 

their effect sizes tended to be modest. A meta-analysis of school-based violence prevention 

evaluations from a mix of experimental and quasi-experimental designs reported an average effect 

size of 0.10 (Cohen’s d) for classroom-based social skills programs (Wilson and Lipsey 2005). 

A few other school-based programs have sought to prevent violence using whole-school 

strategies that seek to influence the school environment, for example, through clarifying rules for 

behavior, increasing supervision of the school grounds, or setting up positive behavior reward 

systems. Some of these programs have been evaluated using middle school–age students, but not 

with the same methodological rigor to which the curriculum-based programs have been subjected. 

Included among these is Positive Behavior Support (PBS [Sugai and Horner 1994; Sprague, Sugai, 

and Walker 1998]), whose components and strategies form the basis for Best Behavior (Sprague and 

Golly 2005), the second of the prevention programs selected for this evaluation. Two evaluations of 

PBS conducted with elementary and middle school students found reductions in students’ aggressive 

social behavior and discipline violations and also found that students had increased knowledge of 

social skills (Metzler et al. 2001; Sprague et al. 2001). A few other programs combine both 

curriculum and whole-school approaches, but their evaluations, while yielding promising results, 

have not been very rigorous. More research is clearly indicated, particularly to determine the effects 

of programs that use a combination of classroom-based curricula and whole-school approaches to 

prevent school violence, which experts in the field of violence prevention suggest may help boost 

impacts. This study will yield information regarding the effects of a combined curriculum and 

whole-school intervention for middle school students. 

The study’s research questions for impacts and implementation in year one are provided in 

table ES-1.

Table ES-1. Research questions 

Intervention implementation

 Is delivery of the violence prevention program consistent with its design and intended implementation?

 With the goal of decreasing disruptive, aggressive, violent, and other delinquent behaviors, what other 

interventions or prevention programs do the treatment and control schools implement, other than the violence 

prevention program under study?

Intervention impacts

 Does the degree of violence differ in schools that implement the violence prevention program, relative to schools 

that do not implement it?

 What is the impact of the violence prevention program on students who are at elevated risk for violence and 

aggression?1

 

1

 The two programs that make up the intervention are universal programs and, as such, are not designed specifically for students who 

are already exhibiting serious violent behavior in school. However, the study addressed this research question because the impacts of 

universal programs on high-risk students are still of interest to practitioners. 
Executive Summary 

Impacts of a Violence Prevention Program for Middle Schools ix

Implementation Findings 

The key descriptive findings regarding the first year of implementation of the curriculum 

portion of the program include the following: 

 The first-year curriculum was delivered in its entirety to a majority of assigned 

classrooms. Seventy percent of schools delivered all 16 lessons to all classrooms where the 

curriculum had been assigned, while another 15 percent of schools delivered all lessons in at 

least three-fourths of these classrooms. 

 Teachers in the majority of treatment schools followed the curriculum lesson scripts 

and adhered to the prescribed teaching strategies. Teachers in 65 percent of the 

intervention schools were observed to deliver lessons with few or no deviations from the 

written lesson plan (e.g., adding or modifying activities or changing the activity sequence), 

according to classroom observations by the evaluation team. In addition, teachers in 

55 percent of the intervention schools delivered lessons with few or no deviations from the 

prescribed teaching strategies (e.g., using role plays or small group discussions). 

Observations made by the liaisons were comparable for alignment with the teaching 

techniques (65 percent for liaisons, as compared with 55 percent for the evaluation team) but 

less so for deviations from the lesson plan (40 percent for liaisons, as compared with 

65 percent for the evaluation team). 

 In a majority of schools, students were engaged with the curriculum. Based on 

evaluation team observations, students were found to be engaged during the lesson activities, 

exercises, and discussions in 85 percent of the intervention schools. Observations made by 

the liaisons were comparable and indicated that students were engaged with the curriculum 

in 60 percent of the intervention schools. 

The main implementation findings for the whole-school portion of the program (Best Behavior) 

include the following: 

 Half of the treatment schools had principals who were supportive of the wholeschool portion of the program. Principals at 50 percent of the treatment schools were 

rated as supportive on three out of four indicators, according to liaisons who helped 

implement the program. 

 The school management teams, charged with developing and disseminating the 

school rules and reward systems, met less frequently than stipulated by the program.

Although school management teams are to meet monthly during the school year, the average 

team met on five occasions during the first year of implementation. 

 By the end of the first year, the majority of treatment schools had instituted 

behavioral rules and rewards, which are key steps during the first year of Best 

Behavior. By the end of the first year, 75 percent of treatment schools had school rules 

posted in the school, 75 percent had instituted a token reward system for adhering to school 

rules, and 50 percent had the school rules taught in classrooms. 

 The majority of teachers agreed that the rules were well defined and that they were 

clear with regard to the behaviors being targeted. Eighty-four percent of teachers at 

intervention schools agreed or strongly agreed that school rules were clearly defined, 

79 percent agreed or strongly agreed that rules emphasized rewarding desired behaviors, and 
Executive Summary 

x Impacts of a Violence Prevention Program for Middle Schools 

69 percent agreed or strongly agreed that rules emphasized consequences for undesired 

behaviors. 

 The primary treatment contrast between the treatment and control schools was the 

violence prevention program implemented by the study. While no control school was 

implementing Best Behavior, an equal number of control and intervention schools used a 

discipline tracking system, one of the strategies encouraged by Best Behavior. In addition, an 

equal number of treatment and control schools implemented peer mediation programs. 

While no control schools implemented RiPP in the first year, slightly more treatment schools 

than control schools implemented stand-alone violence prevention curricula, other than 

RiPP. In addition, slightly more control schools than treatment schools were using security 

measures such as cameras and law enforcement officers. 

Impact Findings 

The main findings regarding intervention impacts after year one are the following:

 There were no statistically significant differences between intervention and control 

schools on self-reported student violence or victimization. On average, 6th-graders in 

the intervention schools reported engaging in 2.91 violent acts at school in the past 30 days, 

compared with 6th-graders in control schools, who reported engaging in 2.88 violent acts at 

school in the past 30 days. On average, 6th-graders in both intervention and control schools 

reported being victimized 4.97 times in the past 30 days. 

 There were no statistically significant impacts on self-reported violence and 

victimization between high-risk youth at treatment schools and high-risk youth at 

control schools. On average, 6th-graders in the intervention schools who were categorized 

as being at a high risk for violence reported at follow-up that they had engaged in 5.16 

(change from baseline of 0.13) violent acts at school in the past 30 days. This is compared 

with high-risk 6th-graders in the control schools who reported at follow-up that they had 

engaged in 4.78 (change from baseline of 0.11) violent acts at school in the past 30 days. In 

addition, high-risk 6th-graders in the intervention schools reported being victimized an 

average of 6.28 (change from baseline of 0.47) times at school in the past 30 days. This is 

compared with high-risk 6th-graders in the control schools who reported being victimized 

an average of 6.23 (change from baseline of 0.30) times at school in the past 30 days. 

 There were no statistically significant impacts on either secondary or intermediate 

outcomes. In addition, after 1 year of exposure to the RiPP and Best Behavior intervention, 

student measures for secondary outcomes—including student safety concerns, teacher 

victimization and safety concerns, and student prosocial behaviors—did not differ between 

students in intervention schools and students in control schools. Also, there were no 

statistically significant differences on intermediate outcomes—that is, where the program 

logic model predicts change would be observed before it would be observed on the outcome 

measures. These include student self-reported coping strategies, student perceptions of 

behavior expectations, and student attitudes toward aggression. 

 There were no statistically significant impacts for either boys or girls on violence or 

victimization. An exploratory subgroup analysis indicated that there were no statistically 

significant differences between boys in the treatment schools and boys in control schools for 

either self-reported violence or victimization. Likewise, there were no statistically significant 
Executive Summary 

Impacts of a Violence Prevention Program for Middle Schools xi

differences between girls in the treatment schools and girls in control schools on either selfreported violence or victimization. Also, the difference in impacts between boys and girls 

was not statistically significant for either self-reported violence or victimization. 

As the implementation results document, the programs being evaluated as part of the study 

were not fully implemented with complete fidelity in the first year. This has the potential to limit the 

ability to find statistically significant differences between treatment and control schools. 

The Intervention 

Two research-based programs were selected through an open competition and advice from a 

panel of experts in the field of violence prevention: the RiPP program (Meyer and Northup 2002a, 

2002b, 2006) was chosen as the curriculum-based component of the intervention, and the Best 

Behavior program (Sprague and Golly 2005) (a formalized version of schoolwide PBS [Sugai and 

Horner 1994; Sprague, Sugai, and Walker 1998]) was selected as the whole-school component. 

These two approaches are considered complementary in that they target both individual- and 

school-level change mechanisms. 

While both RiPP and Best Behavior are implemented by school staff, in this study technical 

assistance was made available throughout the implementation period by on-site implementation 

liaisons trained and hired by the implementation subcontractor, Tanglewood Research. Liaisons 

(e.g., former school teachers or administrators) were expected to facilitate, coach, and monitor the 

progress and delivery of both programs. 

The Curriculum: RiPP Program 

RiPP (Meyer and Northup 2002a, 2002b, 2006) is a universal, social-cognitive violence 

prevention program that focuses on the reduction of situational and relationship violence. The goal 

of the curriculum is to promote effective social-cognitive problem-solving skills; motivation and 

self-efficacy for using those skills; and school norms that support those attitudes and skills while 

reducing the appeal and perceived utility of violent behaviors and related attitudes. By targeting these 

attitudes and skills, the program is designed to increase social competence and thereby reduce 

violent behavior. 

The RiPP curriculum consists of 16 lessons (each lasting 50 minutes) per year in grades 6 

through 8. Each lesson builds on the previous lessons in a cumulative fashion. Similarly, each gradelevel curriculum expands on the concepts taught in the previous year. In year one of program 

implementation, all students in the 6th through 8th grades received lessons designed for use in 6th 

grade (RiPP-6) because all students were required to receive the foundational lessons before the 

more advanced RiPP materials. 

The lessons in the first-year RiPP program introduce the problem-solving model. The 

lessons comprise a variety of activities and strategies, including team building, social-cognitive 

problem solving, repetition and mental rehearsal, small group work, role playing, rehearsal of 

specific social skills for preventing violence, and didactic learning. Most lessons contain between 

four and six of these activities and are estimated to take between 5 and 15 minutes per activity. Each 

activity is scripted and tied to a specific objective. Most lessons make use of the student workbook 

as part of the activities. 
Executive Summary 

xii Impacts of a Violence Prevention Program for Middle Schools 

The Whole-School Intervention: Best Behavior Program 

Best Behavior (Sprague and Golly 2005) provides a standardized staff development program 

that is designed to develop and administer effective school rules and discipline policies at both 

schoolwide and classroom levels to decrease school violence and antisocial behavior. The complete 

Best Behavior program is designed to be implemented on an approximately 3-year timeline, as 

individual school capacity dictates. The program is implemented by a school management team 

made up of teachers and administrators. Best Behavior involves intervention strategies at the school 

and classroom levels, including the following: 

 review and refinement of school discipline policies; 

 use of positive reinforcement and recognition for prosocial behaviors, both schoolwide and 

in individual classrooms; 

 clarification and teaching of behavioral expectations for student behaviors; and 

 systematic collection and review of patterns of discipline referrals to guide decisionmaking 

and planning. 

The Best Behavior management team is expected to create a systematic approach to 

developing schoolwide positive behavior supports. This approach includes four broad sets of 

activities, the first three of which should be implemented beginning in the first year. First, the team 

is to conduct a schoolwide needs assessment to identify reasonable goals. Needs assessments are to 

be repeated annually. Second, the team is to define rules and expectations, with general rules (e.g., be 

safe, be respectful, be responsible) supported with expectations for all settings within the school 

environment. Rules and expectations are to be taught on a regular basis by all teachers and staff. 

Third, the team is to develop and support a positive behavior reinforcement system in which 

students are to be given token rewards for obeying rules and meeting expectations. Finally, in the 

second year of program implementation, the team is to develop a data-based decisionmaking process 

for identifying and addressing the needs of high-risk students. During annual training, Best Behavior 

prompts each school to develop its own specific strategies for achieving goals and objectives for 

each year. Also, Best Behavior was adapted to reinforce and complement the RiPP curriculum. 

Evaluation Design 

Designed as a group-randomized control experiment, the study was conducted in 40 middle 

schools serving grades 6 through 8. Twenty schools were randomly assigned to receive the 

combined intervention, and 20 schools were randomly assigned to serve as control schools (with no 

intervention beyond that which schools were already implementing). In year one, the intervention 

was delivered schoolwide by school staff trained annually by the program developers. School staff 

also received technical assistance and were monitored throughout implementation by trained site 

liaisons under the guidance of Tanglewood Research. 

Participating Schools 

Following recruitment of 13 districts and 40 schools, random assignment to condition was 

conducted within district, among pair-matched sets of schools based on the percentage of students 
Executive Summary 

Impacts of a Violence Prevention Program for Middle Schools xiii 

receiving free or reduced-price lunches.2 The sites were geographically dispersed and represented a 

range of district enrollment sizes. A majority of the districts were located in large urban or suburban 

areas with only three sites in rural districts. All participating schools were middle schools that 

included only grades 6 through 8. The average enrollment in these schools is 871 and ranges 

between 462 and 1,404 students. Minority students compose 65 percent of the student body, on 

average, and range between 15 percent and 100 percent. The average percentage of students 

receiving free or reduced-price lunches is 56 percent and ranges between 16 percent and 97 percent. 

There were no statistical differences between the intervention and control groups on these 

characteristics. 

Student Sample 

The impact analysis for the main findings uses a cross-sectional sample in each school. The 

first-year analysis used data collected from a census of the entering 6th-grade class in fall 2006 

(baseline) and from those who remained in the study schools as 6th-graders or were new in spring 

2007 (first follow-up). The research team obtained written parental consent from 6th-grade students 

at baseline and from new students at the first follow-up. These data were used to answer questions 

about the effects of the intervention across the general student population after 1 year of program 

implementation. Over 70 percent of enrolled students received parental consent and completed a 

survey at baseline, and 77 percent completed a survey at the first follow-up. A total of 7,351 students 

provided survey data for this analysis. More than one-half of the students in the sample were either 

Hispanic or Black. About one-half of the students in the sample were male, and more than one-half 

lived in single-adult households. A two-tailed t-test, obtained from multilevel regression models 

using the school as the unit of analysis, indicated that none of the mean demographic characteristics 

was statistically different between students attending intervention schools and those attending 

control schools. 

To address the research question regarding how the program impacts students at high risk 

for violent behaviors, we identified a subset of students at high risk for violent and aggressive 

behaviors, based on student responses to the fall 2006 baseline survey. Unlike the remainder of the 

students in the full sample, this sample was to be followed longitudinally; the research team 

attempted to survey the students in the high-risk subgroup even if they left one of the study schools 

and were attending another school in the same district during spring 2007. If a student went to a 

different district, the student was not followed. Over 90 percent of the students identified as high 

risk at baseline completed a survey at the first follow-up (N = 1,938). 

Teacher Sample 

Secondary outcome data were collected from teachers through an annual survey conducted 

in spring. This survey was administered to a random sample of 24 teachers (stratified by grade) at 

each of the 40 middle schools participating in the study; a new sample was selected each year. 

Eligible teachers included all full-time classroom teachers and could include RiPP teachers in 

intervention schools. Ninety-six percent (N = 917) of the sampled teachers completed a survey. 

 

2

In the two school districts with three recruited schools, the schools first were rank ordered on the percentage of free or reducedprice lunches, from lowest to highest. The first two schools within each district formed a pair. The third school from each district 

formed the last pair, across districts. Schools were then randomly assigned within each pair to treatment or control conditions. 
Executive Summary 

xiv Impacts of a Violence Prevention Program for Middle Schools 

Data Collection and Outcome Measures 

Data for the study’s outcome measures to estimate intervention impacts were collected 

through student surveys in both treatment and control schools. Student surveys provided data to 

address the main impact research questions regarding school violence. Baseline data collection for 

students occurred in fall 2006, prior to the introduction of the intervention; follow-up student data 

collection occurred in spring 2007. Teacher data were collected through a survey administered to a 

random sample of teachers at each school in spring 2007 to assess other program impacts besides 

main outcomes. In addition to outcome data, the study team collected implementation data through 

the teacher survey, class records, annual school prevention coordinator and teacher interviews, and 

classroom observations. 

The primary outcomes are student violence and student victimization, both measured 

through student surveys. For each of these two outcomes, two additional sub-indices were created 

to better understand any differences between intervention and control schools with regard to 

specific types of violence and aggression. 

A second set of indices was created to examine possible secondary effects from the 

intervention (e.g., spillover effects), beyond the primary effects. These were: (1) student safety 

concerns; (2) teacher safety concerns; (3) teacher victimization; and (4) student prosocial behaviors. 

Finally, a third set of indices was created to examine possible intermediate effects from the 

intervention. The theoretical model for the combined intervention predicts that changes in these 

areas would precede changes on the primary outcomes and included: (1) student perceptions of 

behavior expectations; (2) student attitudes toward violence; and (3) student self-reported coping 

strategies. 

Analytic Approach for Estimating Program Impacts 

The study team evaluated program impacts using multiple regression models that predicted 

each outcome’s measure (e.g., aggression, victimization) as a function of condition (treatment vs. 

control) and relevant covariates (e.g., demographic characteristics, school characteristics) using a 

mixed-effects regression model based on multilevel equations. Primary outcomes include selfreported counts of violent behavior and victimization occurring in the past 30 days. 

The full student sample and gender subgroup analyses used a matched nested cross-sectional 

model (matched analysis). Under this model, students are nested in schools; schools are nested in 

pairs and in experimental condition; and pairs are crossed with experimental condition (i.e., each pair 

is represented at each level of condition). The covariate models for students in the full sample 

predicted the average response at follow-up, adjusting for the following covariates: baseline school 

mean of the response, school size, and individual demographic variables (gender, race/ethnicity, and 

number of parents in the household). For the gender subgroup analyses, the adjusted models 

included a gender-by-condition interaction effect. 

The statistical models employed to assess program outcomes among high-risk youth are 

different from those employed to assess program outcomes on the general population of students. 

For the high-risk youth, the interest is in whether or not the RiPP and Best Behavior intervention 

led to individual change across time. To address this question, nested cohort models using 

difference-in-difference estimation were developed to assess changes on self-reported measures of 

aggression and victimization among high-risk youth in treatment schools relative to changes among 

high-risk youth in control schools. These models use data collected on the same sample of students 

at each measurement occasion. The repeated measures models for the high-risk sample contained 
Executive Summary 

Impacts of a Violence Prevention Program for Middle Schools xv

the student’s treatment condition (intervention vs. control), data collection wave, wave-by-condition 

interaction effect, gender, race/ethnicity, number of parents in household, and school size. 

Estimated program impacts reflect the net difference of the within-group change from pretest to 

first follow-up for treatment versus controls. 

To examine teacher outcomes, we employed multivariate models where teachers are nested 

within schools and schools are nested within matched pairs randomized to experimental condition. 

Hierarchical linear models account for the correlation of teachers within schools and for schools 

within matched pairs assigned to condition. The models predicted the average response at follow-up, 

adjusting for school size. 

The results are presented in terms of event rates (ERs) and event rate ratios (ERRs). ERs 

indicate incidence density; this refers to the number of events among a particular group for a given 

period of time. For the measures of violent behavior and victimization in this study, all items 

assessed occurrences in the past 30 days. Accordingly, an ER of 2.5 among students in intervention 

schools indicates that students in these schools reported an average of 2.5 incidences in the past 30 

days. ERRs compare the incidence density among a group of interest (intervention schools) to a group 

used as a reference (control schools). Where ERRs are greater than 1.00, the indicated group reports 

a higher frequency of occurrences than the reference group; where ERRs are less than 1.00, the 

indicated group reports a lower frequency of occurrences than the reference group. An ERR of 2.00 

would indicate that, on average, students in the intervention schools reported twice as many 

incidents in the past 30 days as students in the control schools; similarly, an ERR of 0.50 would 

indicate that, on average, students in control schools reported twice as many incidents in the past 30 

days as students in intervention schools. 

After 1 year of program implementation, there were no significant differences between the 

students in intervention and control schools on violence and victimization, both overall and for 

specific types of violence and victimization. These data are reported in table ES-2. In addition, there 

were no statistically significant program impacts among the subpopulation of high-risk youth, as 

measured by student violence and victimization (table ES-3). 
Executive Summary 

xvi Impacts of a Violence Prevention Program for Middle Schools 

Table ES-2. Main program impacts on self-reported violence and victimization—Year one 

Self-reported student outcome

Model-adjusted follow-up event rates (SE) Estimated impact1

(95% CI)

Wald ChiIntervention group Control group Square p-value

Violence2 (All items) 2.91 (1.03) 2.88 (1.03) 1.01 (0.93, 1.10) 0.79

Without a weapon 2.79 (1.03) 2.76 (1.03) 1.01 (0.93, 1.10) 0.74

With a weapon 0.10 (1.09) 0.11 (1.09) 0.95 (0.78, 1.15) 0.58

Victimization3(All items) 4.97 (1.02) 4.97 (1.02) 1.00 (0.95, 1.06) 0.99

Overt 2.88 (1.02) 2.86 (1.02) 1.00 (0.94, 1.07) 0.90

Relational 2.07 (1.02) 2.11 (1.02) 0.98 (0.93, 1.03) 0.45

Sample size (Schools) 20 20

Sample size3(7,351 students 

clustered within schools) 3,619 3,732

1

 Program impact estimated as a model-adjusted event rate ratio (ERR) for intervention versus controls at follow-up, with 95 percent 

confidence limits. Impact estimates of 1.00 indicate no difference between intervention and control groups. 

2

 Based on count data. 

3

 Student sample sizes used in the analysis vary due to item nonresponse at follow-up, covariate nonresponse, or both. Missing 

data ranged from 1 percent to 4 percent. 

NOTE: Generalized linear mixed models (SAS PROC GLIMMIX, Poisson distribution with log link function) were used to evaluate 

the program impact while accounting for the clustering of students within schools. Covariates in the model included the baseline 

school mean of the response variable, race/ethnicity, gender, number of parents in household, and school size. CI = confidence 

interval. SE = standard error. 

SOURCE: Student survey, fall 2006 (baseline) and spring 2007 (first follow-up).
Executive Summary

Impacts of a Violence Prevention Program for Middle Schools xvii

Table ES-3. Main program impacts on self-reported violence and victimization—Year one: High-risk subgroup (Via repeated 

measures) 

Student self-reported outcome1

Model-adjusted baseline event rates2

(SE)

Model-adjusted follow-up event 

rates2(SE) Estimated impact 

(95% CI)3

Wald ChiSquare 

Intervention group Control group Intervention group Control group p-value

Violence (All items) 5.03 (1.03) 4.67 (1.03) 5.16 (1.03) 4.78 (1.03) 1.00 (0.92, 1.10) 0.95

Without a weapon 4.76 (1.03) 4.40 (1.02) 4.89 (1.03) 4.51 (1.03) 1.00 (0.92, 1.09) 0.95

With a weapon 0.26 (1.10) 0.26 (1.10) 0.26 (1.11) 0.26 (1.10) 0.99 (0.97, 1.33) 0.99

Victimization (All items) 5.81 (1.03) 5.93 (1.03) 6.28 (1.03) 6.23 (1.03) 1.03 (0.97, 1.10) 0.34

Overt 3.52 (1.02) 3.60 (1.02) 3.86 (1.02) 3.81 (1.02) 1.03 (0.96, 1.11) 0.35

Relational 2.28 (1.02) 2.32 (1.02) 2.42 (1.02) 2.41 (1.02) 1.02 (0.95, 1.09) 0.60

Sample size: Schools 20 20 20 20

Sample size:4Students within 40 schools 1,005 1,148 897 1,016

1

 Based on count data. 

2

 Group-by-time specific event rates. 

3

 Program impact (with 95 percent confidence limits) estimated via difference-in-difference models comparing change across time in the intervention versus control groups. Ratios of 

impact estimates of 1.00 indicate no interaction between time and program group (i.e., no program impact). 

4

 Student sample sizes used in the analysis vary due to item nonresponse at baseline, follow-up, or covariate nonresponse. Missing data ranged from 4 percent to 5 percent, with 240 

missing at follow-up.

NOTE: Generalized linear mixed models (SAS PROC GLIMMIX, Poisson distribution with log link function) were used to evaluate the program impact while accounting for the 

clustering of students within schools. Covariates in the model included gender, race/ethnicity, number of parents in household, and school size. CI = confidence interval. SE = standard 

error. 

SOURCE: Student survey, fall 2006 (baseline) and spring 2007 (first follow-up).