> **Paperwork Reduction Act Submission\
> Supporting Statement**
>
> **An Impact Evaluation of a School-Based\
> Violence Prevention Program**
>
> **PART A - Justification**

*\[Request to Extend Data Collection Period for One Year\]*

Prepared for: U.S. Department of Education

Prepared by: RTI International

February 5, 2009

> []{.mark}
>
> []{.mark}
>
> []{.mark}
>
> **Table of Contents**

[]{.mark}

[A. JUSTIFICATION 1](#a.-justification)

[Abstract 1](#abstract)

[1. Circumstances Making the Collection of Information Necessary
2](#circumstances-making-the-collection-of-information-necessary)

[Research Questions 3](#research-questions)

[Overall Approach 3](#overall-approach)

[Program Selection 3](#program-selection)

[Program Implementation 4](#program-implementation)

[Program Evaluation 4](#program-evaluation)

[Intervention 5](#intervention)

[Conceptual Framework and Logic Model for the Intervention
6](#conceptual-framework-and-logic-model-for-the-intervention)

[2. Purpose and Use of the Information Collection
7](#purpose-and-use-of-the-information-collection)

[3. Use of Automated, Electronic, Mechanical, or Other Technological
Collection Techniques
11](#use-of-automated-electronic-mechanical-or-other-technological-collection-techniques)

[4. Efforts to Identify and Avoid Duplication
11](#efforts-to-identify-and-avoid-duplication)

[5. Impact on Small Businesses or Small Entities
12](#impact-on-small-businesses-or-small-entities)

[6. Consequences of Not Conducting the Data Collection or Collecting It
Less Frequently
12](#consequences-of-not-conducting-the-data-collection-or-collecting-it-less-frequently)

[7. Special Circumstances 13](#special-circumstances)

[8. Federal Register Announcement and Consultation
13](#federal-register-announcement-and-consultation)

[9. Explanation of Any Payment or Gift to Respondents
13](#explanation-of-any-payment-or-gift-to-respondents)

[10. Assurance of Confidentiality 15](#assurance-of-confidentiality)

[11. Justification for Sensitive Questions
16](#justification-for-sensitive-questions)

[12. Estimates of Hour Burden 17](#estimates-of-hour-burden)

[13. Estimate for the Total Annual Cost Burden to Respondents or
Recordkeepers
17](#estimate-for-the-total-annual-cost-burden-to-respondents-or-recordkeepers)

[14. Estimates of Annualized Costs to Federal Government
17](#estimates-of-annualized-costs-to-federal-government)

[15. Reasons for Program Change 19](#reasons-for-program-change)

[16. Plans for Tabulation and Publication and Project Time Schedule
19](#plans-for-tabulation-and-publication-and-project-time-schedule)

[Tabulation Plans 19](#tabulation-plans)

[Impact Analyses for the School-Based Violence Prevention Program
20](#impact-analyses-for-the-school-based-violence-prevention-program)

[Mediation Analyses 21](#mediation-analyses)

[Process Evaluation 22](#process-evaluation)

[Statistical Models 22](#statistical-models)

[Project Schedule 24](#project-schedule)

[Reporting and Publications 24](#reporting-and-publications)

[17. Request for Approval to Not Display OMB Approval Expiration Date
25](#request-for-approval-to-not-display-omb-approval-expiration-date)

[18. Exceptions to Certification for Paperwork Reduction Act Submissions
25](#exceptions-to-certification-for-paperwork-reduction-act-submissions)

Appendix A: Student Survey

Appendix B: Teacher Survey

Appendix C: Violence Prevention Coordinator Interview

Appendix D: Violence Prevention Staff Interview Guide

Appendix E: School Management Team Interview Guide

Appendix F: RIPP Implementation Records

Appendix G: Classroom Observations

Appendix H: School Policy Violations and Disciplinary Actions

Appendix I: Individual Student Records

Appendix J: List of Experts Consulted

Appendix K: Teacher Survey Assent Form

Appendix L: Parental Consent Form

[]{.mark}

[]{.mark}

[]{.mark}

**List of Exhibits**

[Exhibit 1. Research Question 3](#__RefHeading___Toc128815451)

[Exhibit 2. Logic Model for Intervention Combining RiPP and Best
Behavior 5](#__RefHeading___Toc128815452)

[Exhibit 3. Data Collection Activities 8](#__RefHeading___Toc128815453)

[Exhibit 4. Burden in Hours to Respondents
18](#__RefHeading___Toc128815454)

[Exhibit 5. Effect of Violence Prevention Intervention on Violent
Behaviors 19](#__RefHeading___Toc128815455)

[Exhibit 6. Effect of Violence Prevention Intervention on Student
Infractions\
(Binary Outcomes) 20](#__RefHeading___Toc128815456)

[Exhibit 7. Proposed Data Collection Points
20](#__RefHeading___Toc128815457)

[Exhibit 8. Project Schedule 24](#__RefHeading___Toc128815458)

[]{.mark}

[]{.mark}

[]{.mark}

> []{.mark}
>
> **PAPERWORK REDUCTION ACT SUBMISSION**
>
> **SUPPORTING STATEMENT**

Agency: Department of Education

> **Title: An Impact Evaluation of a School-Based Violence Prevention
> Program**

Student Survey

Teacher Survey

> Violence Prevention Coordinator Interview
>
> Violence Prevention Staff Interview Guide
>
> School Management Team Interview Guide

RiPP Implementation Records

Classroom Observation Form

School Policy Violations and Disciplinary Actions

Individual Student Records

Form: N/A

# A. JUSTIFICATION {#a.-justification .Style-Heading-1-+-Left:--0\"}

## Abstract {#abstract .Style-Style-Heading-2-+-Left:--0\"-+-Left:--0\"-Hanging:--0.31\"}

In an ongoing effort to reduce the levels of violence and bullying in
our nation's middle schools, the U.S. Department of Education is
sponsoring a national study of a violence prevention intervention. The
purpose of the study is to implement and test an intervention that
combines a classroom-based curriculum with a whole-school approach. The
study will implement and evaluate the program using a rigorous
randomized control trial design.

In 1986, Congress enacted the Drug-Free Schools and Communities Act
(DFSCA) as Subtitle B of Title IV of the Anti-Drug Abuse Act of 1986,
which quickly became the largest federal effort to prevent alcohol and
other drug use among the nation's school-aged youth. As safety in our
schools became a more pressing concern---and after the President and the
nation's governors adopted the National Education Goals for the year
2000, including the goal for safe, drug-free, and disciplined
schools---Congress reauthorized the DFSCA as Title IV, Safe and
Drug-Free Schools and Communities Act of 1994 (SDFSCA). A major change
brought about by this reauthorization was the inclusion of violence
prevention as a supported activity under the Act. State grants from
SDFSCA provide funding to approximately 98% of all school districts to
implement programs that target youth alcohol and other drug use and
school safety.

The No Child Left Behind Act of 2001 (NCLB) reauthorized the Elementary
and Secondary Education Act of 1965. Section 4121 of the NCLB authorizes
the Secretary to carry out programs to prevent the illegal use of drugs
and violence among, and promote safety and discipline for, students,
including the development, demonstration, scientifically based
evaluation, and dissemination of innovative and high quality drug and
violence prevention programs and activities, based on State and local
needs.

RTI International (RTI), under contract with the National Center for
Education Evaluation in the Institute for Education Sciences at the U.S.
Department of Education (ED), is conducting a five-year study, An Impact
Evaluation of a School-Based Violence Prevention Program. Subcontractors
Pacific Institute for Research and Evaluation (PIRE) and Tanglewood
Research, Inc. are collaborating with RTI in this effort. This study
will provide rigorous and systematic information about the effectiveness
of a hybrid model of middle school violence prevention combining a
curriculum based approach and a whole-school based approach, a model
which is hypothesized to be effective but which has not been rigorously
tested.

## 1. Circumstances Making the Collection of Information Necessary {#circumstances-making-the-collection-of-information-necessary .Style-Style-Heading-2-+-Left:--0\"-+-Left:--0\"-Hanging:--0.25\"}

[This is a request to extend the current data collection period for this
research study by one year to 5/31/10. Nothing in the justification
statement itself has been revised, including the burden
estimates.]{.underline}

Despite its overall decline in the past decade, school crime continues
to be a major concern for many schools and their students. Violent
school crime is a particular concern for middle schools because
middle-school age children are more likely to be victims of school
violence than their high-school-age counterparts (Indicators of School
Crime and Safety, 2003). However, the violence prevention programs
available for middle schools lack rigorous evaluation. The Department
has been advised by a group of experts in school violence to evaluate a
violence-prevention program for middle schools that combines two
approaches: (1) a curriculum-based model to facilitate students' social
competency, problem solving, and self-control skills, and (2) a
whole-school model that targets school practices and policies usually
through classroom management or teaching strategies, or through systemic
reorganization and modification of school management, disciplinary
policies, and enforcement procedures. Through an open competition and
advice from a panel of experts in the field of violence prevention, the
program *Responding in Peaceful and Positive Ways (RiPP)* was chosen as
the curriculum-based component of the intervention and the *Best
Behavior* program was chosen as the whole-school component of the
intervention.

The study will be conducted in 40 middle schools that include grades 6
through 8, of which half will be randomly assigned to receive the
violence prevention program being tested and half to serve as a control
school (with no intervention). Data will be collected at each middle
school over three years of program delivery, during the 2006-2007,
2007-2008, and 2008-2009 school years.

### Research Questions

+-----------------------------------------------------------------------+
| []{#__RefHeading___Toc128815451 .anchor}Exhibit 1. Research Question  |
+-----------------------------------------------------------------------+
| Student Outcomes and Impacts                                          |
+-----------------------------------------------------------------------+
| -   Are there decreases in violence and aggression in schools that    |
|     implement the violence prevention program compared to schools     |
|     that do not implement it?                                         |
|                                                                       |
| -   What is the impact of the violence prevention program over time   |
|     on students who are at an increased risk for violence and         |
|     aggression?                                                       |
|                                                                       |
| -   What are the outcomes of the violence prevention program on       |
|     students with varying years of program exposure (e.g. dosage)?    |
+-----------------------------------------------------------------------+
| Intervention Implementation                                           |
+-----------------------------------------------------------------------+
| -   What other interventions or prevention programs do the treatment  |
|     and control schools implement other than the violence prevention  |
|     program under study with the goal of decreasing disruptive,       |
|     aggressive, violent, and other delinquent behaviors?              |
|                                                                       |
| -   Is delivery of the violence-prevention consistent with its design |
|     and intended implementation?                                      |
|                                                                       |
| -   What are the costs of the program per school and per student as   |
|     delivered in the study? Also, if the school-based violence        |
|     prevention program is effective, is the decrease in problem       |
|     behaviors large enough to warrant the added financial and         |
|     opportunity costs of the program?                                 |
+-----------------------------------------------------------------------+

The study's primary research question is: Do disruptive, aggressive, and
violent behaviors decrease in schools where the selected violence
prevention intervention is implemented compared to schools that do not
receive the program? The study will include both a process evaluation
(to assess the fidelity of program implementation as well as to provide
contextual information) and an impact evaluation (to assess program
outcomes). Specific research questions, depicted in **Exhibit 1**, will
guide each of these components.

### Overall Approach

There are three major components to the study related to program
selection, implementation and evaluation.

#### Program Selection

RTI and its subcontractors assisted the Department in selecting programs
to test in this study by soliciting proposals from the developers of
school-based violence prevention programs and convening a group of
experts to review these proposals. RTI held an open competition which
was announced through a study website, email distribution lists at the
U.S. Department of Education, relevant conferences, and direct contact
with all known program developers of school violence prevention
programs.

After a careful review of the proposals by the panel, a decision was
made to award the subcontract for the violence prevention curriculum to
the developers of *Responding in Peaceful and Positive Ways* (RiPP).
Selection of this program was recommended based on a number of factors,
including: the program is developmentally appropriate, is designed for
middle school students, and is implemented sequentially by grade level;
the program has the best preliminary evidence of effectiveness of any
violence prevention program; the majority of teaching and training
materials are fully developed; and the program is ready to scale up. The
*Best Behavior* program was selected as the whole-school component
because it is a research-based program that has been paired successfully
with curriculum--based programs similar to RiPP, and incorporates
school-wide activities at the school and classroom levels.

#### Program Implementation

An **implementation team** formed by Tanglewood Research staff and a
cadre of locally based site monitors will be responsible for overseeing
full and faithful implementation of RiPP and Best Behavior.
Implementation activities include staff and/or teacher training, early
program troubleshooting and continued program monitoring, and ongoing
technical assistance. Under contract with RTI, the RiPP and Best
Behavior program developers will assist the implementation team with
these activities. The implementation team will also hire and train
coaches/monitors who will visit each of the treatment schools at least
one day a week during the school year for an entire day in order to meet
with staff who are responsible for program delivery and to observe the
program in action throughout the intervention.

#### Program Evaluation

An **evaluation team** formed by RTI and PIRE staff will perform all
activities needed to rigorously evaluate the intervention. A process
evaluation component will assess fidelity of implementation, level of
student attendance, training of implementers, implementation of
interventions other than the test intervention, and other issues
relevant to effective implementation. An outcome evaluation will assess
intervention and control schools' changes in disruptive, aggressive, and
violent behaviors, other in-school outcomes, behaviors such as
delinquency, and mediating and moderating variables related to the
program's logic model. The evaluation will assess outcome changes in
intervention schools compared to control schools, the extent to which
changes in outcomes are linked to implementation of the intervention,
and whether the effects of the program vary by student risk
characteristics. The evaluation will also examine the costs of the
program.

The study will be conducted in a set of middle schools that include
grades 6^th^ through 8^th^. Schools will be randomly assigned to either
receive the combined violence prevention intervention or to serve as a
control school (with no intervention beyond what is usually
implemented). Students and teachers will be surveyed at each middle
school over three years of program delivery. Data on students'
violations of school policies will also be collected as will data on
these violations for schools as a whole. Also, evaluation team members
will conduct interviews with school staff and observe implementation of
RiPP in classrooms, and RiPP teachers will provide further data on
implementation.

### Intervention 

The intervention combines two approaches: (1) *Responding in Peaceful
and Positive Ways (RiPP),* a curriculum-based model to facilitate
students' social competency, problem solving, and self-control skills,
and (2) *Best Behavior,* a whole-school model that targets school
practices and policies through classroom management or teaching
strategies, or through systemic reorganization and modification of
school management, disciplinary policies, and enforcement procedures.
(See logic model in **Exhibit 2**).

[]{#__RefHeading___Toc128815452 .anchor}Exhibit 2. Logic Model for
Intervention Combining RiPP and Best Behavior

![](media/image1.jpeg){width="4.70625in" height="7.121527777777778in"}

The combined intervention will be implemented in approximately 20 middle
schools for three consecutive school years (i.e., in the 2006-2007,
2007-2008, and 2008-2009 school years). The whole-school component will
begin during spring 2006, prior to the 2006-07 school year.

#### Conceptual Framework and Logic Model for the Intervention

The goal of the violence prevention program is to improve the safety and
climate of the school as well as to decrease violent and aggressive acts
in students (Exhibit 2). These outcomes will be measured in the study.
The program is predicted to have its effects on these behaviors through
a number of intermediate (or mediating) factors. The violence prevention
program is a hybrid of a curriculum (RiPP) and a whole school
intervention (Best Behavior). RiPP is a universal social-cognitive
violence prevention program focused primarily on situational and
relationship violence. The program is designed to increase social
competence and thereby reduce violent behavior and other problem
behaviors and improve school climate and safety. The RiPP curriculum for
this study consists of 16 fifty-minute lessons delivered over the course
of a school year. Grades 6, 7, and 8 each receive 16 lessons that are
designed specifically for each respective grade level. The *Best
Behavior* whole-school component aims to improve school and classroom
discipline in schools and is designed to complement and reinforce the
RiPP concepts taught in the classroom. This component will be
implemented by a school management team made up of teachers and
administrators. The team will meet weekly for the first two months of
the program and monthly thereafter. The whole-school component will
involve intervention strategies at the school and classroom levels,
including:

-   review and refinement of school discipline policies;

-   systematic collection of and review of patterns of discipline
    referrals to guide decision making and planning;

-   instruction on classroom organization and management techniques;

-   use of positive reinforcement and recognition for prosocial behavior
    in classrooms and school wide; and

-   clarifying and teaching behavioral expectations for student
    behaviors

In addition, teams will be taught highly effective procedures for
defusing angry and aggressive behavior in the classroom. For the
purposes of this study, Best Behavior will be adapted to reinforce and
coordinate with the RIPP curriculum.

## 2. Purpose and Use of the Information Collection {#purpose-and-use-of-the-information-collection .Style-Style-Style-Heading-2-+-Left:--0\"-+-Left:--0\"-Hanging:--0.25\"...}

The purpose of the evaluation is to determine the overall effectiveness
of a violence prevention intervention that combines two approaches: (1)
a curriculum-based model to facilitate students' social competency,
problem solving, and self-control skills (Responding in Peaceful and
Positive Ways -- RiPP), and (2) a whole-school model that targets school
practices and policies through classroom management and teaching
strategies, and through systemic reorganization and modification of
school management, disciplinary policies, and enforcement procedures
(Best Behavior). The evaluation will provide important and useful
information by helping to determine if the intervention decreases
problem behaviors and improves school climate and safety.

Reducing school violence and bullying and improving school and classroom
climate are important to the Department of Education---particularly in
middle schools, which often experience more problems of this nature than
elementary or high schools. This study will provide valuable evaluative
information regarding the RiPP and Best Behavior programs, which are
very promising violence prevention programs designed for middle schools.
This study will be of great utility to the Department in its efforts to
decrease school violence and improve school and classroom climate. We
will present our findings to the Department via reports and briefings.
We will also provide the Department a data file of all data collected in
the study, along with supporting documentation.

RTI and its subcontractors will faithfully implement a school-based
violence prevention program and conduct an independent and rigorous
evaluation using a randomized control trial (RCT) design. The evaluation
will assess outcome changes in intervention schools compared with
control schools, the extent to which changes in outcomes are linked to
implementation of the intervention, and whether the effects of the
program vary by various profiles of student risk. To evaluate the
effectiveness of the intervention, the evaluation will collect data from
middle schools in both treatment and control conditions. Because the
data collected at each school will be combined with, and compared
against, those collected from other schools, it is critical that data
collection procedures and elements be uniform across all the schools.
RTI has designed a complementary set of data collection tools to capture
information on key components of the intervention, outcomes the
intervention is intended to influence, and potential mediators through
which such influence is expected to occur. **Exhibit 3** summarizes the
surveys, survey format, data collection schedule, and respondents. In
the section below, we describe the key topic areas to be addressed by
each of the data collection instruments.

[]{#__RefHeading___Toc128815453 .anchor}Exhibit 3. Data Collection
Activities

+----------------+------------+--------------+---+---+---+---+---+---+
| Instrument     | Source     | Condition    | Y |   | Y |   | Y |   |
|                |            |              | e |   | e |   | e |   |
|                |            |              | a |   | a |   | a |   |
|                |            |              | r |   | r |   | r |   |
|                |            |              | 1 |   | 2 |   | 3 |   |
+----------------+------------+--------------+---+---+---+---+---+---+
|                |            |              | > | > | > | > | > | > |
|                |            |              |   |   |   |   |   |   |
|                |            |              | F | S | F | S | F | S |
|                |            |              | a | p | a | p | a | p |
|                |            |              | l | r | l | r | l | r |
|                |            |              | l | i | l | i | l | i |
|                |            |              | > | n | > | n | > | n |
|                |            |              |   | g |   | g |   | g |
|                |            |              | 0 | > | 0 | > | 0 | > |
|                |            |              | 6 |   | 7 |   | 8 |   |
|                |            |              |   | 0 |   | 0 |   | 0 |
|                |            |              |   | 7 |   | 8 |   | 9 |
+----------------+------------+--------------+---+---+---+---+---+---+
| Student        | Students   | Treatment    |   |   |   |   |   |   |
| Self-report    |            | and Control  |   |   |   |   |   |   |
| Surveys        |            | Schools      |   |   |   |   |   |   |
+----------------+------------+--------------+---+---+---+---+---+---+
| Teacher        | Teachers   | Treatment    |   |   |   |   |   |   |
| Surveys        |            | and Control  |   |   |   |   |   |   |
|                |            | Schools      |   |   |   |   |   |   |
+----------------+------------+--------------+---+---+---+---+---+---+
| Violence       | Violence   | Treatment    | [ |   | [ |   | [ |   |
| Prevention     | Prevention | and Control  | ] |   | ] |   | ] |   |
| Coordinator    | C          | Schools      | { |   | { |   | { |   |
| Interview      | oordinator |              | . |   | . |   | . |   |
| Guide          |            |              | m |   | m |   | m |   |
|                |            |              | a |   | a |   | a |   |
|                |            |              | r |   | r |   | r |   |
|                |            |              | k |   | k |   | k |   |
|                |            |              | } |   | } |   | } |   |
+----------------+------------+--------------+---+---+---+---+---+---+
| Violence       | Violence   | Treatment    |   |   |   |   |   |   |
| Prevention     | Prevention | Schools      |   |   |   |   |   |   |
| Staff          | Staff      |              |   |   |   |   |   |   |
| Interview      |            |              |   |   |   |   |   |   |
| Guide          |            |              |   |   |   |   |   |   |
+----------------+------------+--------------+---+---+---+---+---+---+
| School         | School     | Treatment    |   |   |   |   |   |   |
| Management     | Management | Schools      |   |   |   |   |   |   |
| Team Interview | Team       |              |   |   |   |   |   |   |
| Guide          | Members    |              |   |   |   |   |   |   |
+----------------+------------+--------------+---+---+---+---+---+---+
| RiPP           | RiPP       | Treatment    |   |   |   |   |   |   |
| Implementation | Teachers   | Schools      |   |   |   |   |   |   |
| Records        |            |              |   |   |   |   |   |   |
+----------------+------------+--------------+---+---+---+---+---+---+
| Classroom      | Evaluation | Treatment    |   |   |   |   |   |   |
| Observations   | Staff      | Schools      |   |   |   |   |   |   |
+----------------+------------+--------------+---+---+---+---+---+---+
| School Policy  | Principals | Treatment    |   |   |   |   |   |   |
| Violations and |            | and Control  |   |   |   |   |   |   |
| Disciplinary   |            | Schools      |   |   |   |   |   |   |
| Actions        |            |              |   |   |   |   |   |   |
+----------------+------------+--------------+---+---+---+---+---+---+
| Individual     | Principals | Treatment    |   |   |   |   |   |   |
| Student        |            | and Control  |   |   |   |   |   |   |
| Records        |            | Schools      |   |   |   |   |   |   |
+----------------+------------+--------------+---+---+---+---+---+---+

The **Student Survey** (Attachment A) will be conducted in both
treatment and control schools. The student survey will be administered
to all 6^th^ graders in the fall and spring of the 2006-2007 school
year. Respondents who remain in study schools will complete the survey
again in spring 2008 as 7^th^ graders. The survey will be administered
to all 8^th^ graders in the spring of 2009. Questions will ask about
students' social competency skills; aggressive or disruptive conduct;
attitudes toward violence; victimization by violence or bullying; and
perceptions of safety at school and related avoidance behaviors. The
survey will be self-administered in a classroom setting. It will be
available in both English and Spanish and will take approximately 45
minutes to complete.

The **Teacher Survey** (Attachment B) will be completed by teachers in
both intervention schools and comparison schools. Questions will ask
about teachers' perceptions of the level of disruptive behaviors in
class, perceptions of school climate, experience with victimization and
feelings of safety in school. It will take approximately 30 minutes to
complete. The survey will be administered to a random sample of 24
teachers (stratified by grade) at each of the middle schools
participating in the study. Teachers will complete the survey in spring
of 2007, 2008, and 2009, with a new random sample of teachers selected
each year.

The **Violence Prevention Coordinator Interview** (Attachment C) will
provide information on school-wide policies and programs for violence
prevention. The interview will ask about the school's security,
monitoring, and discipline policies and procedures; staff professional
development for violence prevention; information regarding all of the
schools' violence prevention efforts; and, in intervention schools only,
the coordinator's impressions of the teachers' response to RiPP and how
well RiPP and *Best Behavior* work together. Each interview will take
approximately 45 minutes in control schools and 60 minutes in treatment
schools. Data will be collected once each school year during the
2006/2007, 2007/2008, and 2008/2009 school years. We anticipate that we
will conduct the interviews in approximately February so that
respondents will be able to describe activities that have been conducted
or are ongoing. We will revisit this schedule if necessary.

The **Violence Prevention Staff Interview** (Attachment D) will be
conducted to obtain information on implementation of RiPP. Interviews
will be conducted with teachers and other violence prevention staff
implementing RiPP in treatment schools. The semi-structured interview
will ask about RiPP implementation experiences, challenges, and
impressions; the time taken by teachers to prepare, deliver, and
follow-up the intervention; fidelity and adaptation; training and
technical assistance; and how RiPP and *Best Behavior* fit together.
Each interview will take approximately 50 minutes. Data will be
collected once each school year during the 2006-2007, 2007-2008, and
2008-2009 school years. We anticipate conducting these interviews as
part of the same site visit during which the Violence Prevention
Coordinator interview will be conducted.

The **School Management Team (SMT) Interview** (Attachment E) will
provide information on the implementation of the *Best Behavior*
whole-school approach to violence prevention. The interview will ask
about the SMT member's background and violence prevention roles; learn
how staff in each school have implemented *Best Behavior*; estimate the
time taken by each member in SMT activities; assess fidelity of
implementation; gather staff impressions on training and technical
assistance received; and gather staff impressions on how well *Best
Behavior* and RiPP have fit together and with other programs. Each
interview will take approximately 50 minutes. Data will be collected
once each school year during the 2006-2007, 2007-2008, and 2008-2009
school years. We anticipate conducting these interviews as part of the
same site visit during which the Violence Prevention Coordinator
interview will be conducted.

**RiPP Implementation Records** (Attachment F) will provide information
on implementation of the RiPP curriculum and will be collected in
treatment schools only. We will provide RiPP teachers standard reporting
forms to record implementation information including program attendance,
number of participants, and sessions covered or topics addressed. The
data recording will take approximately 240 minutes over the course of
the school year. Reports will be collected monthly during implementation
in each treatment school, in the 2006-2007, 2007-2008, and 2008-2009
school years.

**Classroom Observations** (Attachment G) will provide information on
tangible features of the violence prevention program such as adherence
to program design, consistent delivery, and level of student
participation. Evaluation staff members will be trained in the use of a
standardized observation form and protocol to ensure consistency of
observations across classrooms and schools. During annual site visits
held each spring during program implementation, evaluation staff members
will conduct observations in treatment school classrooms in which RiPP
is implemented. Observations will be designed to minimize disruption to
classroom implementation of RiPP.

**School- Policy Violations and Disciplinary Actions** (Attachment H)
will provide information on important outcomes at the school level, and
will be gathered every 8 weeks in treatment and control schools. We will
provide materials with consistent operational definitions and data
collection procedures for all study schools. Many schools will gather
these types of data for other purposes and will be able to provide it
with little effort. Where more effort is required, we will work with
each school to identify mutually agreeable procedures for collecting the
data. Possibilities include using our trained field data collection or
reimbursing the school for staff time. Information gathered will include
suspensions, expulsions, violent or delinquent infractions and related
disciplinary actions or referrals for counseling. The time required for
gathering the data may vary among schools; we estimate that on average
it will take approximately 95 minutes per school year. Data collection
will be ongoing for each school year to ensure schools' compliance with
the data collection protocol. Data collection for the first year will
begin in the fall of 2006 and continue until the spring of 2007, for the
second year it will begin in fall of 2007 and end in spring of 2008, and
for the third year it will begin in fall of 2008 and end in spring of
2009.

**Individual Student Records** (Attachment I) will provide information
on behavioral outcomes for individual students in treatment and control
schools. We will work with each school to identify mutually agreeable
procedures for collecting the data. We will ask schools to provide
electronic files if possible, in which case RTI will extract the
pertinent data. (Like all project staff, these staff members will be
required to sign a non-disclosure agreement promising to keep
confidential all project data.) If it is necessary to manually abstract
information from student records, our first preference will be to use
our trained field data collection staff. Some schools may not allow that
so as an alternative we will reimburse the school for staff time devoted
to abstracting student records data. Information gathered will include
attendance, policy violations and consequences, and disciplinary actions
(e.g. detention, suspension, expulsion). We estimate that abstraction
will take on average 15 minutes per student. Data will be collected in
the spring of 2007, 2008, and 2009, following the completion of the
school year.

## 3. Use of Automated, Electronic, Mechanical, or Other Technological Collection Techniques {#use-of-automated-electronic-mechanical-or-other-technological-collection-techniques .Style-Heading-2-+-Left:--0\"}

We will collect only the minimum information necessary for the study. We
have designed data collection procedures to minimize respondent burden
and use reporting formats that are best-suited for the type of
information to be gathered. The student surveys and the teacher surveys
will be self-administered paper and pencil surveys using almost entirely
yes/no, Likert scale, or checklist responses to expedite data
collection. Responses to the student and teacher surveys will be
recorded on forms designed to be optically scanned. For collection of
school-wide crime and disciplinary data, student records data, and
implementation records data, we will use existing data, in electronic
format where possible. Where manual collection or abstraction is
required we will use standard forms. We will use interviews with
teachers and other violence prevention staff in order to obtain a
complete and accurate view of violence prevention activities and
policies. These interviews will include structured questions to ease
reporting and maximize comparability of responses.

## 4. Efforts to Identify and Avoid Duplication {#efforts-to-identify-and-avoid-duplication .Style-Heading-2-+-Left:--0\"}

The Department of Education will submit 60-day and 30-day Federal
Register notices intended to solicit public comment on the proposed
information collection.

RTI has reviewed other evaluations of school violence prevention
programs and believes the proposed information collection is unique in
several regards. This data collection is expressly designed to measure
important aspects of the implementation of the unique intervention that
combines a whole-school approach with a student-focused curriculum, and
to measure elements of the intended outcomes of decreased problem
behavior and improved school climate/safety. No previous studies to our
knowledge have assessed the implementation and impact of this type of
combined intervention, which is thought to be a most promising approach
to preventing violence in middle schools.

Moreover, this data collection is unique in its plans for collecting
complementary, non-redundant information from the participants and
stakeholders in this combined intervention. No available studies provide
information from all sources necessary to rigorously evaluate this
intervention, including student and teacher surveys, interviews with
school staff, school records on violence-related infractions and
disciplinary actions, records of participation in intervention
activities, and classroom observations of intervention implementation.
Data from these sources are specific to the intervention and its
intended outcomes are critical to the study and must be gathered in a
uniform and consistent manner.

Extant, ongoing national surveys represent a potential for duplicate
data collection. RTI explored the possibility of relying on data from
surveys such as the Youth Risk Behavior Survey and Monitoring the Future
and identified the following insurmountable problems: (1) these surveys
are not conducted in all sites, (2) site-representative data are not
available from many of the sites where these surveys are conducted, and
(3) the data collected by these surveys do not include measures that
would allow a comprehensive assessment of the impact of RiPP and Best
Behavior.

## 5. Impact on Small Businesses or Small Entities {#impact-on-small-businesses-or-small-entities .Style-Heading-2-+-Left:--0\"}

The respondents for the study are students, teachers, and other school
staff. No small businesses or small entities will be involved in this
data collection.

## 6. Consequences of Not Conducting the Data Collection or Collecting It Less Frequently {#consequences-of-not-conducting-the-data-collection-or-collecting-it-less-frequently .Style-Heading-2-+-Left:--0\"}

Without this data collection, the Department of Education cannot assess
the impact of this unique intervention combining a whole-school approach
with a student-focused curriculum to reduce violent and other delinquent
behaviors among 6^th^, 7^th^ and 8^th^ grade students. In particular, it
is critical to conduct data collection at specific times in schools
participating in the study in order to measure changes in student
behavior and school climate/safety and assess whether those changes are
related to the intervention. Existing data from other schools collected
at other times will [not]{.underline} provide sufficient information to
establish the impact of the intervention. The data collection described
herein and analysis of these data will provide information critical to
the development of effective school violence prevention programs, which
will potentially advance the important field of school violence
prevention. In addition, it is necessary to collect the data as
scheduled to accurately assess the cumulative effects of the violence
prevention program with each successive year of implementation; thus,
data will be collected over the three consecutive years that the program
is implemented.

## 7. Special Circumstances  {#special-circumstances .Style-Heading-2-+-Left:--0\"}

There are no special circumstances involved with this data collection.

## 8. Federal Register Announcement and Consultation {#federal-register-announcement-and-consultation .Style-Heading-2-+-Left:--0\"}

A 60-day notice was published in the Federal Register on December 5,
2005, with an end date of February 3, 2006 to provide the opportunity
for public comment. No substantive comments were received. In addition,
as part of the evaluation design process, a panel of nationally known
experts reviewed and commented on the overall evaluation strategy and
some members provided input on measurement instruments. (See Attachment
J for names of the experts who contributed.) The design and planning of
the data collection effort was completed in collaboration with
representatives from the Department of Education. These representatives
were particularly interested and knowledgeable in school-based violence
prevention programs and made helpful contributions.

## 9. Explanation of Any Payment or Gift to Respondents {#explanation-of-any-payment-or-gift-to-respondents .Style-Heading-2-+-Left:--0\"}

We plan to follow the *Guidelines for Incentives for NCEE Evaluation
Studies* prepared by the Department of Education and dated March 22,
2005. This memo outlines the circumstances in which respondent
incentives are appropriate in NCEE studies and the maximum amounts
permitted. This study meets the criteria for the use of respondent
incentives as outlined in these Guidelines.

[Teacher survey incentives]{.underline}. We will give each teacher an
incentive each time that he or she completes the teacher survey. This
incentive will be \$30 in accordance with the amount stated in the
Guidelines for a survey of medium burden.  This incentive will be
instrumental in obtaining completed teacher surveys for this study
because teachers are the target of numerous requests to complete surveys
on a wide variety of topics from state and district offices, independent
researchers, and the Department of Education (PPSS and NCES). Further,
the teachers' school days are already quite busy, potentially requiring
them to complete surveys outside school time. There are also in some
localities collective bargaining agreements that do not allow teachers
to complete surveys during school time.

[Incentives for consent]{.underline}. In addition, we will provide a
\$25 incentive to classroom teachers or the school survey coordinator to
encourage high rates of return for the parental consent forms needed for
the student survey. Our preferred method of distributing consent forms
is to include the forms in a packet parents receive at the school's open
house during enrollment at the beginning of the school year.
Alternatively, homeroom teachers may distribute consent forms to their
students or the school may wish to have the forms mailed directly to the
students' home. We will work closely with each school to ensure that
parents who do not return the consent form are sent reminder notices
with replacement forms. This process will be repeated, as needed, to
achieve the target active parental consent rate. We will provide an
incentive (\$25 gift card) for each classroom in which at least 90% of
the student survey parental consent forms are returned, whether or not
the parent allows the student to participate in the student survey. The
incentive will be provided either to the classroom teacher or to the
school survey coordinator, based on who is responsible for monitoring
the consent form returns, for the added burden this process will place.
The purpose of the classroom incentive is to encourage the teacher or
the survey coordinator to monitor consent returns carefully and follow
up with students for whom a form has not been returned, encouraging
students to return a signed form. We have used similar approaches
successfully in previous studies. For example, in the cross-site
evaluation of the Safe Schools/Healthy Students Initiative we told
teachers and staff that we would provide each school an additional \$25
for each classroom in which a minimum of 70% of active parental consent
forms were completed by parents and returned to the school. In schools
where we used this approach we achieved return rates of parental consent
forms of 78%, on average. We will make this incentive opportunity
available for each Grade 6 classroom participating in the survey in the
2006/2007 school year, and in the 8^th^ grade classrooms participating
in the 2008/2009 school year.

[Control schools incentive.]{.underline} In accordance with the
Guidelines, we will provide a stipend of \$1500 to each control group
school to partially offset costs related to participating in the study.
Control group schools will receive this stipend each year that they
participate in the study and will be offered the opportunity to use
their stipends to purchase the violence prevention program and training
at a discounted rate, following the third year of the study. Treatment
schools will not receive this stipend because they are receiving the
intervention. 

The burden for control schools will be fairly high in this study and
there will be no inherent benefit to the control schools resulting from
their participation. Control schools will not receive the intervention
under study for three years [and]{.underline} will be asked to not
implement any violence prevention programs similar to the intervention
under study for three years. For control schools in this evaluation,
there will be a student survey in the fall and spring, a teacher survey
in the spring and interviews with multiple staff for multiple days
during the year. In addition, control schools will provide records about
school policy violations committed by students, and distribute and
collect parental consent forms. The data collection will be spread
throughout the school year and will occur over three consecutive years.
This is a lengthy and intense fielding period with no direct benefits to
the control schools, so it is important to use the incentives outlined
in the Guidelines to maintain a commitment from these schools so that
they will continue participation throughout the three years and provide
us with all the data needed for the study.

Control schools that participate in the study will not necessarily
receive Department grants from the Office of Safe and Drug Free Schools;
thus, they will not be obliged to participate in a study of violence
prevention for the Department. Partially off-setting the costs of
participation of control schools will be important in order to maintain
their continued participation and cooperation. Because random assignment
takes place at the school level, not the student level, the loss of a
single control school will result in the loss of a significant portion
of the study sample. It is imperative to keep all control schools in the
study or the result of the whole study could be vitiated. Also, this
study is a random assignment impact study to which substantial
Department of Education resources have been committed. Without high
respondent completion rates for both the program and control groups, the
investment of Department funds will not produce valid findings.

## 10. Assurance of Confidentiality  {#assurance-of-confidentiality .Style-Heading-2-+-Left:--0\"}

All data collection activities will comply with the Privacy Act of 1974,
P.L. 93-579, 5 USC 552 a; the "Buckley Amendment," Family Educational
and Privacy Act of 1974, 20 USC 1232 g; the Freedom of Information Act,
5 USC 522; and related regulations, including but not limited to: 41 CFR
Part 1-1, 45 CFR Part 5b, and 40 FR 44502 (September 26, 1975); and, as
appropriate, the Federal common rule or Department final regulations on
protection of human research subjects. RTI International conducts all
research involving human subjects in accordance with federal regulations
(45 CFR 46 and 21 CFR 50 and 56). It is the responsibility of RTI\'s
Office of Research Protection and Ethics (ORPE), as well as the research
staff, to ensure that these regulations are followed. One of the
functions of ORPE is to oversee RTI's three Institutional Review Boards
(IRBs), which review all research involving human subjects that is
conducted by RTI researchers. RTI holds a Federalwide Assurance (FWA)
from the Department of Health and Human Services\' Office for Human
Research Protections (OHRP). RTI assumes full responsibility for
performing all research involving human subjects (regardless of funding
agency) in accordance with that Assurance, including compliance with all
federal, state, and local laws as they may relate to the research.
RTI\'s IRB procedures are updated as needed and are available for review
by any federal agency for whom RTI does human subjects research.

All data collection activities will be conducted in full compliance with
Department of Education regulations to maintain the confidentiality of
data obtained on private persons and to protect the rights and welfare
of human research subjects as contained in Department of Education
regulations. Research participants (students, teachers, and school
staff) will sign written consent (or assent) forms (Attachments K and
L). The consent materials will inform respondents about the nature of
the information that will be requested and confidentiality protection,
and they will be assured that information will be reported only in
aggregate statistical form. The consent materials will also inform
respondents that the data will be used only for research purposes by
researchers who have signed a confidentiality agreement.

In addition to the consent forms, each self-administered instrument will
include a reminder of the protection of confidentiality. Where data are
collected through in-person interviews or group surveys, interviewers or
survey administrators will remind respondents of the confidentiality
protections provided, as well as their right to refuse to answer
questions to which they object. During the group administration of the
student survey report, desks will be arranged in classroom style to
ensure that children cannot see the responses provided by classmates.
All data collectors and interviewers will be knowledgeable about
confidentiality procedures and will be prepared to describe them in full
detail, if necessary, or to answer any related questions raised by
respondents.

RTI has a long history of protecting confidentiality and privacy of
records, and considers such practice a critical aspect of the scientific
and legal integrity of any data collection. The integrity RTI brings to
protecting data confidentiality and privacy will extend to every aspect
of data collection and data handling for this study. RTI plans to use
its ongoing, long-standing techniques that have proven effective in the
past. Every interviewer will be required to sign a pledge to protect the
confidentiality of respondent data. The pledge indicates that any
violation or unauthorized disclosure may result in legal action or other
sanctions by the national evaluator.

In addition, the following safeguards are routinely employed by RTI to
carry out confidentiality assurances:

-   Access to sample selection data is limited to those who have direct
    responsibility for providing the sample. At the conclusion of the
    research, these data are destroyed.

-   Identifying information is maintained on separate forms, which are
    linked to the interviews only by a sample identification number.
    These forms are separated from the interviews as soon as possible.

-   Access to the file linking sample identification numbers with the
    respondents' identification and contact information is limited to a
    small number of individuals who have a need to know this
    information.

-   Access to the hard copy documents is strictly limited. Documents are
    stored in locked files and cabinets. Discarded material is shredded.

-   Computer data files are protected with passwords and access is
    limited to specific users.

## 11. Justification for Sensitive Questions {#justification-for-sensitive-questions .Style-Heading-2-+-Left:--0\"}

Only the data collection proposed for students and teacher surveys
contain questions that could be deemed sensitive; the data collection
instruments for other participants do not contain sensitive questions.
Some of the questions on the Student Survey could be deemed
sensitive---most notably those related to violent behavior or
victimization. Voluntary student self-reports are needed because
administrative records do not contain the information needed to assess
the impact of this intervention on these sensitive behaviors.
Specifically, administrative records (either school or police records)
will miss many incidents, particularly, for example, less serious but
potentially damaging violent incidents (e.g., bullying). Also, accurate
records of incidents of behaviors outside school cannot be obtained from
administrative records.

Many of the questions that could be considered sensitive are drawn from
or are consistent with widely used national surveys administered to
similar populations in school settings. Questions addressing violent
behavior and victimization are crucial to assessing the impact of the
intervention.

Students and teachers will be advised during the informed consent
process of the voluntary nature of participation and their right to
refuse to answer any question. They will also be assured that the
responses are completely confidential. Moreover, the development and
conduct of the evaluation, including the administration of the student
survey, is overseen by one of RTI's three Institutional Review Boards,
each of which has HHS Multiple Project Assurance.

## 12. Estimates of Hour Burden  {#estimates-of-hour-burden .Style-Heading-2-+-Left:--0\"}

**Exhibit 4** provides our estimate of time burden. Data collection
among the sixth graders will take place in the 2006-2007 school year.
Students from the 6^th^ grade will be followed into the 7^th^ grade with
follow up data collection occurring in the spring of 2008. A refreshed
census will be collected from 8^th^ graders in the 2008-2009 school
years.

The estimated burden for Individual Student Records and School Policy
Violations and Disciplinary Actions are upper bound estimates because
they are based on school staff abstracting information from records
while we will actually have evaluation staff do the abstraction wherever
possible.

## 13. Estimate for the Total Annual Cost Burden to Respondents or Recordkeepers  {#estimate-for-the-total-annual-cost-burden-to-respondents-or-recordkeepers .Style-Heading-2-+-Left:--0\"}

There are no direct costs to individual participants other than their
time to participate in the study. We have assumed that there are no
financial costs to student respondents because the surveys will be
completed during regular school hours; however, we recognize that
participants will each miss approximately 45 minutes from their classes.
To the extent possible we will administer the surveys during students'
study halls or other elective classes in order not to interfere with
learning activities in core academic courses.

## 14. **Estimates of Annualized Costs to Federal Government** {#estimates-of-annualized-costs-to-federal-government .Style-Heading-2-+-Left:--0\"}

The Department of Education conducted an open competition to select an
organization to conduct this study*.* RTI was awarded a 5-year contract
that is being funded incrementally. The estimated cost of the Federal
government contract is \$10,291,690 with an average annual cost of
\$2,000,000.

[]{#__RefHeading___Toc128815454 .anchor}Exhibit 4. Burden in Hours to
Respondents

  ------------------------------------------------------------------------------------------------------
                   **Grade   **Grade   **Grade   **Number of    **Responses/   **Avg.       **Total
                   6**       7**       8**       Respondents\   Respondent**   Burden       Burden
                                                 Total**                       Hours/       Hours**
                                                                               Response**   
  ---------------- --------- --------- --------- -------------- -------------- ------------ ------------
  **Student                                                                                 
  Survey**                                                                                  

  Fall 2006        9,720                         9,720          1              0.75         7290

  Spring 2007      9,720                         9,720          1              0.75         7290

  Fall 2007                                      0              0              0            0

  Spring 2008                7,760               7,760          1              0.75         5820

  Spring 2009                          9,720     9,720          1              0.75         7290

                                                                                            

  **Teacher                                                                                 
  Survey**                                                                                  

  Spring 2007                                    960            1              0.50         480

  Spring 2008                                    960            1              0.50         480

  Spring 2009                                    960            1              0.50         480

                                                                                            

  **Violence                                                                                
  Prevention\                                                                               
  Coordinator                                                                               
  Interview**                                                                               

  Spring 2006                                    40             1              .875         35

  Spring 2007                                    40             1              .875         35

  Spring 2008                                    40             1              .875         35

                                                                                            

  **Violence                                                                                
  Prevention Staff                                                                          
  Interview**                                                                               

  Spring 2006                                    120            1              .833         100

  Spring 2007                                    120            1              .833         100

  Spring 2008                                    120            1              .833         100

                                                                                            

  **School                                                                                  
  Management Team                                                                           
  Interview**                                                                               

  Spring 2006                                    120            1              .833         100

  Spring 2007                                    120            1              .833         100

  Spring 2008                                    120            1              .833         100

                                                                                            

  **RiPP                                                                                    
  Implementation                                                                            
  Records**                                                                                 

  Fall 2007                                      240            1              4            960

  Fall 2008                                      240            1              4            960

  Fall 2009                                      240            1              4            960

                                                                                            

  **School Policy                                                                           
  Violations and                                                                            
  Disciplinary                                                                              
  Actions**                                                                                 

  2006-2007 school                               40             1              1.5          60
  year                                                                                      

  2007 -- 2008                                   40             1              1.5          60
  school year                                                                               

  2008 -- 2009                                   40             1              1.5          60
  school year                                                                               

                                                                                            

  **Individual                                                                              
  Student                                                                                   
  Records**                                                                                 

  Spring 2007                                    40             1              69.5         2780

  Spring 2008                                    40             1              139          5560

  Spring 2009                                    40             1              139          5560

                                                                                            

  **TOTALS**                                     **41,600**                                 **46,795**
  ------------------------------------------------------------------------------------------------------

## 15. Reasons for Program Change {#reasons-for-program-change .Style-Heading-2-+-Left:--0\"}

The program change of 15,599 average annual hours of burden is because
this is a new collection.

## 16. Plans for Tabulation and Publication and Project Time Schedule {#plans-for-tabulation-and-publication-and-project-time-schedule .Style-Heading-2-+-Left:--0\"}

The RTI evaluation team will conduct data analyses that address the key
impact and process evaluation questions. In this section we first
provide sample templates of tables we will use to report data. Next we
provide an overview of our plans for analysis of outcomes, mediators,
and process measures. Then we describe our planned statistical models.
Last, we present the project schedule and our plans for reporting and
publications. Technical details on sampling and power are provided in
Section B.

### Tabulation Plans

**Exhibit 5** shows a sample template of a table that will provide
means and standard errors by treatment conditions for student outcomes
of primary interest.  Information is given for the spring data
collection period, following intervention activities. A comparison of
treatment and control means for each outcome is provided and a
standardized effect size estimate is given. 

[]{#__RefHeading___Toc128815455 .anchor}Exhibit 5. Effect of Violence
Prevention Intervention on Violent Behaviors

  ---------------- ------------- ---------- ----------- ---------- ------------ ----------
                                                                                

  **Outcomes**     **Treatment              **Control                           
                   Schools**                Schools**                           

                   **N**         **Mean     **N**       **Mean     **Impact**   **Effect
                                 (SE)                   (SE)                    Size**
                                 Spring**               Spring**                

  **Student        20                       20                                  
  reports of their                                                              
  own behavior**                                                                

  **Student        20                       20                                  
  reports of                                                                    
  others'                                                                       
  behaviors toward                                                              
  them**                                                                        
  ---------------- ------------- ---------- ----------- ---------- ------------ ----------

\* Difference is statistically significant at the .05 level

Effect size = Impact expressed in standard deviation units

*Sources: Student Survey*

**Exhibit 6** shows a sample template of a table that will provide mean
percentages and standard errors by schools.  A comparison of treatment
and control percentages for each outcome is provided, and a standardized
effect size estimate is given. 

[]{#__RefHeading___Toc128815456 .anchor}Exhibit 6. Effect of Violence
Prevention Intervention on Student Infractions (Binary Outcomes)

  ------------------- -------------- ----------------- ------------ ----------
  **Outcome**         **Mean                           **Model      
                      Percentages\                     Results**    
                      (standard                                     
                      error)**                                      

                      **Treatment    **Control                      
                      Schools (n     Schools\                       
                      =20)**         (n = 20)**                     

                      **%**          **%**             **Impact**   **Effect
                                                                    Size**

  **Bullying**                                                      

  **Robbery**                                                       

  **Physical attack                                                 
  or fight**                                                        

  **Threat of                                                       
  physical attack**                                                 

  **Theft/larceny**                                                 

  **Possession of                                                   
  firearm or                                                        
  explosive device**                                                

  **Possession of                                                   
  knife or sharp                                                    
  object**                                                          
  ------------------- -------------- ----------------- ------------ ----------

\* Difference is statistically significant at the .05 level

Notes: Odds ratio = odds of occurrence in the treatment group divided by
odds of occurrence in the control group.

*Sources: School Policy Violations and Disciplinary Actions*

### 

#### Impact Analyses for the School-Based Violence Prevention Program

The impact analysis is based on a group-randomized control experiment.
We will evaluate program impact using multiple regression models that
predict each outcome measure (e.g., aggression, victimization) as a
function of condition (treatment versus control), and relevant
covariates (e.g., baseline measures, demographic characteristics). The
inclusion of covariates related to the outcomes but unrelated to program
exposure will improve the precision of the test of the program effect by
reducing unwanted variation. Because students will be nested within
schools and schools nested within condition, we will estimate these
effects using multilevel regression equations and software. Exhibit 7
explains the planned data collection

[]{#__RefHeading___Toc128815457 .anchor}Exhibit 7. Proposed Data
Collection Points^a^

+-----------------+----------------+----------------+----------------+
| Data Collection | Grade Level    |                |                |
| Year            |                |                |                |
+-----------------+----------------+----------------+----------------+
|                 | 6^th^ grade    | 7^th^ grade    | 8^th^ grade    |
+-----------------+----------------+----------------+----------------+
| Year 1          | **Census of    |                |                |
|                 | 6th grade fall |                |                |
|                 | & spring**     |                |                |
+-----------------+----------------+----------------+----------------+
| Year 2          |                | **7th grade    |                |
|                 |                | stayers & ITT  |                |
|                 |                | high-risk^b^** |                |
|                 |                |                |                |
|                 |                | **spring**     |                |
+-----------------+----------------+----------------+----------------+
| Year 3          |                |                | **Census of    |
|                 |                |                | 8th grade &    |
|                 |                |                | ITT            |
|                 |                |                | high-risk^b^   |
|                 |                |                | spring**       |
+-----------------+----------------+----------------+----------------+

^a^Data will be collected at both treatment and control schools.

^b^Intent-to-treat sample of students at high risk for aggressive and
violent behaviors.

Question 1: Are there decreases in violence and aggression in schools
that implement the violence prevention program compared to schools that
do not implement it?

These analyses address the primary issue of program impact and will
include an overall impact evaluation after three years of implementation
as well as an interim evaluation of the effects of the program after one
year of implementation. The overall impact analysis (i.e., impact of
three years of program implementation) will be based on data collected
from a census of students in the eighth grade class in the spring of
2009 in treatment and control schools. An interim analysis of program
impact after a single year of program implementation will be based on a
census of data collected from students in the sixth grade in spring of
2007 in treatment and control schools. The difference in follow-up
between the sixth graders in control and treatment schools will then be
compared.

Question 2: What is the impact of the violence prevention program over
time on students who are at an increased risk for violence and
aggression?

Similar to the analysis of main effects, analysis of program impacts on
the high-risk group will occur in implementation years one, two, and
three. First, using information from the aggression measures in the
6^th^ grade baseline survey in year one, we will identify a high-risk
sample in both control and treatment schools before the program begins.
Impacts will be estimated for this intent-to-treat subgroup for all
three years of the data collection. Impacts from the first year will
appear in the interim report and the longer term effects from years two
and three will appear in the final report.

Question 3: What are the outcomes of the violence prevention program on
students with varying years of program exposure?

The current study design does not permit an experimental examination of
dosage and such a study would be cost prohibitive. But the current
design will permit a correlational analysis of dosage using data from
treatment students with varying years of exposure to the program and
comparing the data to students in the control schools. This analysis
will take place after the third year of program implementation when
there will be varying degrees of exposure to the program among the
eighth-grade students in the treatment group.

#### Mediation Analyses

We will conduct a series of **mediation analyses** (Baron and Kenny,
1986; MacKinnon, 2002; MacKinnon and Dwyer, 1993) to identify mediators
of program effects. Our mediation analysis will evaluate the indirect
effects of the program on student outcomes by positing two intermediate
variables in the analytic causal pathway: student attitudes towards
violence and aggression, and student strategies for coping with anger.
By examining these intermediate factors that are hypothetically related
to the outcomes of interest, our mediation analysis will begin to
address important questions about *how* the program achieves its effects
(MacKinnon and Dwyer, 1993). We will build a model that examines how
participating in the intervention affects these mediators and how they
in turn relate to the student outcomes. Path models, estimated with
Mplus statistical software, will be used to conduct all mediation
analyses. Mediated effects will be estimated as the product of two
regression coefficients obtained from the path model: (1) the
coefficient relating treatment group to the mediators at follow-up and
(2) the coefficient relating the mediator at follow-up to the outcome at
follow-up. This estimate, together with an estimate of the standard
error of this effect (Sobel, 1982), provides a *z*-score test of each
mediated effect (MacKinnon et al., 2002).

#### Process Evaluation

The purpose of the process evaluation will be twofold. First, it will be
used to determine if the violence prevention program was implemented as
designed and intended. Second, it will be used to describe any violence
prevention activities taking place in control schools or treatment
schools (other than the program being evaluated), and to determine the
extent to which these activities are similar to the violence prevention
program used in the treatment condition. We will determine the fidelity
of the violence prevention program in treatment schools by analyzing
data from interviews with violence prevention staff, implementation
records, and classroom observations. We will determine the nature of any
additional violence-prevention activities in the treatment schools (and
determine the treatment contrast between treatment and control schools)
also through these measures. In addition, we will examine treatment
contrast through the responses of the violence coordinator interviews as
well as several of the responses from surveys of teachers.

### Statistical Models

The primary research questions will be addressed using statistical
models that properly account for the complexities of the study design.
Both nested cross-sectional and nested longitudinal models will be
employed. To address question one, student population outcomes will be
assessed with a cross-sectional model. For this class of model, the
analytical unit is the schools, not individuals within the schools. For
question two, outcomes for high-risk students will be assessed with a
longitudinal experimental design with repeated measures taken on
students nested in schools and schools nested within experimental
conditions. For question three, a dosage analysis will compare results
from students in treatment schools who receive varying years of program
exposure to students at control schools. The standard errors estimated
and significance tests conducted will account for the fact that schools
(not students) are the units of random assignment.

Question 1: Are there decreases in violence and aggression in schools
that implement the violence prevention program compared to schools that
do not implement it?

The nested cross-sectional model is appropriate to assess the primary
research question as well as for interim analyses that evaluate
short-term effects of the program. We assume a census of each grade will
provide approximately 243 students in the censuses of both grades six
and eight in each school. We assume balance across experimental
conditions and express the model as follows:

$Y_{i:k:l} = \mu + C_{l} + \left( \sum_{o = 1}^{x}{\beta_{o}\left( X_{\text{oi}:k:l} - {\overline{X}}_{o\text{.}\text{.}\text{.}\text{.}} \right)} \right) + G_{\text{k:l}} + \varepsilon_{\text{i:k:l}}$
(1)

In this model, Y~i:k:l~ represents the response of the i^th^ person
nested in the k^th^ group in the l^th^ condition. The four terms on the
right hand side of the equation represent the grand mean (µ); the effect
of the l^th^ condition (C~l~); the realized value of the k^th^ group
(G~k:l~**);** any difference between this predicted value and the
observed value is allocated to residual error (e~ij:k:l~). If any of
random components are excluded, the model will be misspecified. The
inclusion of covariates $\left( X_{\text{oi}:k:l} \right)$ can improve
the test of the intervention effect (i.e., increase the precision of the
model) to the extent that the covariates explain a portion of the
residual variation.

Question 2: What is the impact of the violence prevention program over
time on students who are at an increased risk for violence and
aggression?

Repeated measures nested cohort models will be employed to address the
impact of the program over time on a high-risk sample of students.
Estimation of treatment effects will be accomplished with a model that
includes repeated measures on an intent-to-treat sample of students
within groups and on groups within conditions. With additional levels of
nesting, this model will have five components of random variation and
can be expressed as:

$Y_{\text{ij}:k:l} = \mu + C_{l} + T_{j} + \text{TC}_{\text{jl}} + \left( \sum_{o = 1}^{x}{\beta_{o}\left( X_{\text{oij}:k:l} - {\overline{X}}_{o\text{.}\text{.}\text{.}\text{.}} \right)} \right) + G_{\text{k:l}} + M_{\text{i:k:l}} + \text{TG}_{\text{jk:l}} + \text{MT}_{\text{ij:k:l}} + \varepsilon_{\text{ij:k:l}}$
(2)

In this model, Y~ij:k:l~ represents the response of the i^th^ person
measured at time j, nested in the k^th^ group of the l^th^ condition.
The first four terms (normal text) represent the fixed-effects portion
of the model, and the last five terms (bold text) represent the
random-effects portion of the model. The fixed effects include: the
grand mean (µ); the effect of the l^th^ condition (C~l~); the effect of
the jth time (T~j~); and the joint effect of the jth time and the lth
condition (TC~jl~). The random components include: the realized value of
the ith person **(M~i:k:l~**); the realized value of the kth group
(**G~k:l~);** the realized value of the kth group at time j
(**TG~jk:l~**); and the realized value of the combination of the ith
member at the time j (**MT~ij:k:l~**). Any difference between this
predicted value and the observed value is allocated to residual error
(**e~ij:k:l~**). If any of random components are excluded the model will
be misspecified. The inclusion of covariates
$\left( X_{\text{oij}:k:l} \right)$ can improve the test of the
intervention effect (i.e., increase the precision of the model) to the
degree that the covariates represent variables that are related to the
endpoint and unevenly distributed across conditions, these variables
would confound the observed value of the intervention effect if ignored.
In this model, the null hypothesis is no difference across the adjusted
condition means over time. Any variation associated with the
intervention effect will be captured in the adjusted time by condition
means (TC~jl~) and evaluated against the variation among the adjusted
time by group means (TG~jk:l~). Once again, the standard errors
estimated and significance tests conducted will account for the fact
that schools (not students) are the units of random assignment.

### Project Schedule

Data collection will begin in 2006 and continue through 2009, as noted
in **Exhibit 8**.

[]{#__RefHeading___Toc128815458 .anchor}Exhibit 8. Project Schedule

+-----------------------+---------------+---------------+--------------+
| **Activity**          | **Schedule**  |               |              |
+=======================+===============+===============+==============+
|                       | **2006-2007\  | **2007-2008\  | **2008-2009\ |
|                       | School Year** | School Year** | School       |
|                       |               |               | Year**       |
+-----------------------+---------------+---------------+--------------+
| Consent letters sent  | August --     |               | August --    |
| to parents^1^         | September     |               | September    |
|                       |               |               |              |
|                       | April-May     |               | April-May    |
+-----------------------+---------------+---------------+--------------+
| Student Survey        | August        | April-May     | April-May    |
|                       | --September\  |               |              |
|                       | April-May     |               |              |
+-----------------------+---------------+---------------+--------------+
| Teacher Surveys       | April-May     | April-May     | April-May    |
+-----------------------+---------------+---------------+--------------+
| Violence Prevention   | January --    | January --    | January --   |
| Coordinator Interview | April         | April         | April        |
+-----------------------+---------------+---------------+--------------+
| Violence Prevention   | January --    | January --    | January --   |
| Staff Interviews      | April         | April         | April        |
+-----------------------+---------------+---------------+--------------+
| School Management     | January --    | January --    | January --   |
| Team Interviews       | April         | April         | April        |
+-----------------------+---------------+---------------+--------------+
| RiPP Implementation   | September --  | September --  | September -- |
| Records               | June          | June          | June l       |
+-----------------------+---------------+---------------+--------------+
| Classroom Observation | January --    | January --    | January --   |
|                       | April         | April         | April        |
+-----------------------+---------------+---------------+--------------+
| School Policy         | September --  | September --  | September -- |
| Violations and        | June          | June          | June         |
| Disciplinary Data     |               |               |              |
+-----------------------+---------------+---------------+--------------+
| Individual Student    | May 2007 -    | May 2008 --   | May 2009 --  |
| Records               | November 2007 | November 2008 | November     |
|                       |               |               | 2009         |
+-----------------------+---------------+---------------+--------------+
| Reports               | February      |               |              |
|                       | 2008\         |               |              |
| First\                | \             |               |              |
| \                     | August 2010   |               |              |
| Final                 |               |               |              |
+-----------------------+---------------+---------------+--------------+

^1^Consent letters in spring 2007 and spring 2009 will be needed only
for students who are new to the study. The spring 2008 survey will
include only students who participated in the fall 2006 survey; no
additional consent letters will be needed.

### Reporting and Publications

There will be two reports of evaluation results as shown in **Exhibit
8** above. These reports will present findings from the analyses
described above, including descriptive analyses of violence prevention
in study schools and implementation of the violence prevention program
in treatment schools; changes in outcomes and differential change in
treatment and control schools; whether changes in outcomes are mediated
or moderated; and the cost of implementing the violence prevention
program\\. Special reports will be prepared as needed/requested.

## 17. Request for Approval to Not Display OMB Approval Expiration Date {#request-for-approval-to-not-display-omb-approval-expiration-date .Style-Heading-2-+-Left:--0\"}

The present submission does not request such approval. The expiration
date will be displayed along with the OMB approval number.

## 18. Exceptions to Certification for Paperwork Reduction Act Submissions  {#exceptions-to-certification-for-paperwork-reduction-act-submissions .Style-Heading-2-+-Left:--0\"}

No exceptions to the certification statement are requested or required.

> **Paperwork Reduction Act Submission\
> Supporting Statement**
>
> **An Impact Evaluation of a School-Based\
> Violence Prevention Program**
>
> **PART B -- Collection of Information Employing Statistical Methods**

*\[Request to Extend Data Collection Period for One Year\]*

Prepared for: U.S. Department of Education

Prepared by: RTI International

February 5, 2009

> []{.mark}

[]{.mark}

> **Table of Contents**
>
> [B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
> [3](#b.-collection-of-information-employing-statistical-methods)](#b.-collection-of-information-employing-statistical-methods)

[1. Respondent Universe and Sampling Methods
[3](#respondent-universe-and-sampling-methods)](#respondent-universe-and-sampling-methods)

[2. Statistical Methods for Sample Selection and Degree of Accuracy
Needed
[5](#statistical-methods-for-sample-selection-and-degree-of-accuracy-needed)](#statistical-methods-for-sample-selection-and-degree-of-accuracy-needed)

[3. Methods to Maximize Response Rates
[7](#methods-to-maximize-response-rates)](#methods-to-maximize-response-rates)

[4. Tests of Procedures and Methods to be Undertaken
[7](#tests-of-procedures-and-methods-to-be-undertaken)](#tests-of-procedures-and-methods-to-be-undertaken)

[5. Individuals Consulted on Statistical Aspects and Individuals
Analyzing Data
[8](#individuals-consulted-on-statistical-aspects-and-individuals-analyzing-data)](#individuals-consulted-on-statistical-aspects-and-individuals-analyzing-data)

[C. REFERENCES [9](#c.-references)](#c.-references)

List of Exhibits

[]{.mark}

[Exhibit 9. Expected Number of Students, for All Students and the
High-Risk Subset 4](#__RefHeading___Toc128815459)

[Exhibit 10. 80% Statistical Power as a Function of Minimal Detectable
Difference\
(Number of Students Per School = 243) 6](#__RefHeading___Toc128815460)

[Exhibit 11. 80% Statistical Power as a Function of Minimal Detectable
Difference\
(Number of High Risk Students Per School = 41)
6](#__RefHeading___Toc128815461)

Exhibit 12. Statute Authoring the
Evaluation.........................................................\...10

[]{.mark}

# B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS {#b.-collection-of-information-employing-statistical-methods .Style-Heading-1-+-Left:--0\"}

## 1. Respondent Universe and Sampling Methods {#respondent-universe-and-sampling-methods .Style-Heading-2-+-Left:--0\"}

In order to obtain the sample for the study, first schools will be
sampled and then a subgroup of students within those schools will
complete the outcomes measures. The following explains the sampling
methods for both these levels of sampling.

[Schools]{.underline}. The sampling frame of schools will be constructed
using publicly available information from the Common Core of Data
regarding poverty (based on the percentage of students in each school
eligible for free and reduced meals), enrollment size, locale, and
race/ethnicity. In order to address the analytical objectives of this
study and manage the many operational challenges, the sampling frame
will be limited to regular (public, non-charter, and non-magnet) schools
with grades 6-8 with a 6^th^-grade population of at least 250 students.
We require at least 250 students in each school in order to obtain a
sufficiently large high-risk subsample.  School districts with fewer
than three eligible schools will be excluded because they will yield
insufficient matched pairs of schools to include in the study. Moreover,
schools in the sampling frame will be selected to be urban, high
poverty, and/or high minority schools, factors that are strongly
correlated with school violence. Because data on actual rates of school
violence are often not collected and, when they are, are notoriously of
poor validity, the aforementioned characteristics will be used as
proxies for identifying schools with elevated levels of violence and
aggression.

Working from the sampling frame, districts will first be screened to
determine their eligibility and identify any approval process required
to contact sampled schools. Districts will be deemed ineligible to
participate if there is a current or planned district mandate for all
middle schools to implement curriculum based or whole-school violence
prevention programs that are similar to RiPP or Best Behavior. Upon
receipt of district approval to contact schools, schools will be mailed
study materials and called to invite participation into the study.
Criteria such as 6^th^ grade enrollment, percentage of minority
students, percentage of students eligible for free and reduced lunch
will be requested to verify school level eligibility. Schools will be
excluded if they are currently implementing or planning to implement
programs similar to the RiPP and or Best Behavior programs or if future
redistricting plans will negatively impact the 6^th^ grade enrollment
levels for the ensuing academic year. Schools must have a class schedule
that can accommodate sixteen 50- minute lessons for all 6^th^, 7^th^,
and 8^th^ graders and identify placement for the RiPP curriculum in
their normal academic day. Schools will agree to be randomly selected
into the intervention or control group. If selected for the control
group, schools must also agree to refrain from implementing similar
violence prevention programs throughout the three years of the study.
Within each school district, matched pairs of schools will be created
based on levels of poverty and ethnicity mix. One school from within
each pair will be randomly assigned to each of the two experimental
conditions.

[Students]{.underline}. We will survey sixth graders in the 2006-2007
school year, seventh graders in the 2007-2008 school year, and eighth
graders in the 2008-2009 school year. **Exhibit 9** shows the number of
students we anticipate will participate in each round of the student
survey. We will obtain consent from parents of sixth-grade students in
fall 2006. We anticipate that 9,720 sixth-grade students (80% of the
sixth-grade population) will receive parental consent and will be able
to participate in the baseline survey. In spring 2007, we will obtain
parental consent for sixth grade students who have moved into the
schools since the baseline survey and will again survey sixth graders at
that time. In spring 2008 we will survey seventh-grade students who
completed the baseline the previous year and who remain in the school;
their prior parental consent will still apply. We estimate that of the
9,720 students in the baseline survey, 7,760 (80%) will remain in the
study schools and complete the survey in spring 2008. In spring 2009 we
will obtain consent for eighth graders who have transferred into the
school since the baseline survey (and for whom the baseline consent
would not apply) and will survey eighth graders at that time. We
anticipate that 9,720 students (80% of the population) will participate
in the spring 2009 survey.

[]{#__RefHeading___Toc128815459 .anchor}Exhibit 9. Expected Number of
Students, for All Students and the High-Risk Subset

  ------------ ---------- ---------- --------------------------------------------
               **Total    **Per      **Comments**
               sample**   school**   

  **All                              
  Students**                         

  Available    12,160     304        

  Fall 2006    9720       243        Census, assuming 80% parental consent

  Spring 2007  9720       243        Census, assuming 80% parental consent

  Spring 2008  7760       194        Students remaining from fall 2006 survey,
                                     assuming 20% attrition

                                     

  Spring 2009  9720       243        Census, assuming 80% parental consent

  **Subset of                        
  High-Risk                          
  Students**                         

  Fall 2006    2560       64         Identified from fall 2006 survey

  Spring 2007  2360       59         Assuming 8% attrition of identified group
                                     (within school year)

  Spring 2008  1960       49         Assuming 16% attrition of remaining group
                                     (one year later)

  Spring 2009  1640       41         Assuming 18% attrition of remaining group
                                     (one year later)
  ------------ ---------- ---------- --------------------------------------------

We will identify a subset of students at high risk for violent and
aggressive behavior, based on responses to the baseline violence and
aggression student self-report in fall 2006. We anticipate identifying
on average 64 high-risk students per school. Throughout the survey
schedule described for all students, we will follow the high-risk
students that remain enrolled in the participating districts. The number
of students in this high-risk group will decrease over time to the
extent that some students leave study schools. We estimate that of the
64 high-risk students per school at baseline, 59 will participate in the
spring 2007 survey, 49 in the spring 2008 survey, and 41 in the spring
2009 survey. Consequently, a total of 1,640 high-risk such students are
anticipated across the 40 schools at the final survey with approximately
820 high-risk students in each of the control and treatment groups,
respectively.

[Teachers.]{.underline} The teacher survey will be administered to a
random sample of 24 teachers (stratified by grade) at each of the middle
schools participating in the study. Teachers will complete the survey in
spring of 2007, 2008, and 2009, with a new random sample of teachers
selected each year.

## 2. Statistical Methods for Sample Selection and Degree of Accuracy Needed {#statistical-methods-for-sample-selection-and-degree-of-accuracy-needed .Style-Heading-2-+-Left:--0\"}

Statistical power indicates the likelihood that an intervention effect
will be judged to be statistically significant, given the assumptions of
the specified model. In other words, power is the likelihood of
observing a statistically significant difference where such a difference
exists. An under-powered study then, is one in which the investigators
risk failing to notice a significant intervention effect. Previous
small-scale, less rigorous evaluations of the RiPP intervention have
reported significant effects for suspensions, violent behaviors,
self-reported frequency of physical aggression, drug use, peer
provocation, non-physical aggression among boys, attitudes for violence
in boys, delinquent behaviors, and victimization (Farrell, Valois,
Meyer, & Tidwell, 2003, Farrell, Meyer, Sullivan, & Kung, 2003; Farrell,
Meyer, & White, 2001; Farrell, Valois, & Meyer, 2002). The current study
is designed to detect an effect size of 0.20, given the effect sizes
found in previous studies of RiPP and the fact that the inclusion of the
Best Behavior program is anticipated to increase program effects. In
addition, experts with whom we consulted regard a minimum detectable
effect (MDE) of 0.20 as reasonable given the estimated monetary and
opportunity costs of a school-based intervention combining curricular
and whole-school approaches.

In order to determine the sample size required for an MDE of 0.20, we
assumed an ICC of 0.045 based on a conservative review of previous
research measuring self-reported violent behavior in students (Janega,
Murray,Varnell, Blitstein, Birnbaum, & Lytle, 2004). We also assume an
alpha-level of 0.05 and a two-tailed significance test. Given these
assumptions, randomly assigning 40 schools to either treatment or
control conditions and surveying 243 students at each school will
provide sufficient power for testing the null hypothesis regarding
treatment effectiveness on schools (**Exhibit 10**). These estimates are
based on the mixed-effect model as appropriate for an analysis that
includes random effects at the individual and school level. Statistical
power could be improved even further by including covariates in the
models such as baseline data.

[]{#__RefHeading___Toc128815460 .anchor}Exhibit 10. 80% Statistical
Power as a Function of Minimal Detectable Difference\
(Number of Students Per School = 243)

  ----------------------------------- -----------------------------------
  **Minimal Detectable Difference**   **Number of Schools Required\
                                      for Each Study Condition**

  **0.24**                            **14**

  **0.22**                            **17**

  **0.20**                            **20**

  **0.18**                            **25**

  **0.16**                            **32**
  ----------------------------------- -----------------------------------

Note: Calculations assume = .05 and a two-tailed test.

Note: Calculations are based on ICC = 0.045

**Exhibit 11** provides a power table for the nested cohort analysis
that will be used to assess the sub-sample of high risk youth. Using
assumptions previously noted regarding attrition rates across the three
years of the program, we anticipate approximately 41 students from each
school will provide data for this analysis. Although fewer students
provide data for the cohort analysis than the cross-sectional analysis,
this potential limitation is offset by a smaller anticipated ICC and a
gain in precision both of which are associated with taking replicate
measures on individuals. In this case, a sample size of 40 schools will
allow us to detect effects of only 0.20 or larger.

[]{#__RefHeading___Toc128815461 .anchor}Exhibit 11. 80% Statistical
Power as a Function of Minimal Detectable Difference\
(Number of High Risk Students Per School = 41)

+----------------------------------+-----------------------------------+
| **Minimal Detectable             | **Number of Schools Required      |
| Difference**                     | for**                             |
|                                  |                                   |
|                                  | **Each Study Condition**          |
+----------------------------------+-----------------------------------+
| **0.24**                         | **15**                            |
+----------------------------------+-----------------------------------+
| **0.22**                         | **17**                            |
+----------------------------------+-----------------------------------+
| **0.20**                         | **20**                            |
+----------------------------------+-----------------------------------+
| **0.19**                         | **21**                            |
+----------------------------------+-----------------------------------+
| **0.18**                         | **26**                            |
+----------------------------------+-----------------------------------+
| **0.16**                         | **33**                            |
+----------------------------------+-----------------------------------+

Note: Calculations assume = .05 and a two-tailed test.

Note: Calculations are based on ICC = 0.019

As in **Exhibit 10**, statistical power could be improved even further
by including covariates in the models such as baseline data. In summary,
Exhibits 10 and 11 demonstrate that a total of 40 schools are required
to detect the target minimal detectable effects.

## 3. Methods to Maximize Response Rates  {#methods-to-maximize-response-rates .Style-Heading-2-+-Left:--0\"}

Different issues concern response rates for student surveys versus other
types of surveys, and our approaches vary accordingly. For the student
survey, response rates at baseline will be largely determined by three
factors: parental consent, student absence at survey administration, and
student willingness/ability to complete the questionnaire. Active
parental consent is required, and we will work closely with each school
to ensure that parents who do not return the consent form are sent
reminder notices with replacement forms. We will provide a \$25
incentive to classroom teachers or the school survey coordinator to
encourage high rates of return for the parental consent forms needed for
the student survey. Follow-up procedures will be repeated, as needed, to
achieve the target active parental consent of 80%. Regarding student
absence, we will track attendance at survey administration for all
students for whom we have parental consent, keeping a list of absentees.
We will schedule one make-up session in each school for students who
missed the regular administration. Regarding students being able and
willing to complete the questionnaire, we will pilot-test the
questionnaire with nine students to determine that they are able to
complete the questionnaire in 45 minutes or less and that the items are
not overly difficult and do not make respondents uncomfortable. During
survey administration, we will bolster students' belief in the
confidentiality of the survey by using two-part identification labels
with peel-away portions that leave only a bar-coded label with no
identifiers on the survey. The surveys will be administered by trained
RTI staff members and we will ask teachers to leave the classroom during
administration. Upon completing the survey, students will seal their
response sheets in a large envelope.

At the follow-up student survey administrations, participation will also
be affected by students transferring out of study schools. We will trace
students and attempt to administer the survey in their new locations, as
long as the student remains within the school district.

For participants other than students, we will facilitate participation
in the interviews by being flexible in scheduling interviews. For the
teacher survey, we will monitor data collection returns and contact
respondents who have not yet returned their completed questionnaire to
encourage their participations. Teachers who complete the teacher survey
will be given a small gift card valued at \$30.

## 4. Tests of Procedures and Methods to be Undertaken {#tests-of-procedures-and-methods-to-be-undertaken .Style-Heading-2-+-Left:--0\"}

We will pilot test the student survey with fewer than nine students in
grades 6, 7, and 8. We will be certain to include 6^th^ graders because
they are relatively more likely to be challenged by completing the
survey and will therefore provide the most diagnostic information. We
will recruit students through personal contacts and, if necessary,
through middle schools in the vicinity of RTI offices in Research
Triangle Park, NC. We will provide pilot test participants a small
incentive for participating. We will instruct participants that their
participation is voluntary and they may skip any questions that they do
not choose to complete. We will also assure them that their responses
are anonymous and confidential and will instruct them not to put their
names on the survey.

We will ask pilot test participants to complete the survey and will
monitor the time required. We will ask them to flag items that are
unclear or difficult, to facilitate later discussion. When all
participants have completed the survey we will lead a group discussion
of their reactions to the survey, with a particular focus on those items
that they found unclear or difficult. As appropriate we will ask their
input on improving the items. We will also invite participants to
provide written comments if they like.

## 5. Individuals Consulted on Statistical Aspects and Individuals Analyzing Data {#individuals-consulted-on-statistical-aspects-and-individuals-analyzing-data .Style-Heading-2-+-Left:--0\"}

  ---------------------- --------------------------- ---------------------
  **Name, Degree**       **Title**                   **Telephone**

  Mansour Fahimi, Ph.D.  Senior Research             301-230-4675
                         Statistician                

  Jonathan L. Blitstein, Health Research             919-541-7313
  Ph.D.                  Methodologist               

  Jason Williams, Ph.D.  Health Research Analyst     919-541-6734
  ---------------------- --------------------------- ---------------------

#  C. REFERENCES {#c.-references .Style-Heading-1-+-Left:--0\"}

Bureau of Labor Statistics, U.S. Department of Labor, *Occupational
Outlook Handbook, 2006-07 Edition*, Teachers---Preschool, Kindergarten,
Elementary, Middle, and Secondary, on the Internet at
**http://www.bls.gov/oco/ocos069.htm** (visited *February 14, 2006*).

Cheong, J., MacKinnon, D.P., & Khoo, S.-T. (2003). Investigation of
mediational processes using parallel process latent growth curve
modeling. *Structural Equation Modeling, 10*(2), 238-262.

Farrell, A. D., Meyer, A. L., & Sullivan, T.N., & Kung, E.M. (2003).
Evaluation of the Responding in Peaceful and Positive Ways (RIPP)
Seventh Grade Violence Prevention Curriculum. Journal of Child and
Family Studies, Vol. 12, No. 1, 101-120.

Farrell, A. D., Valois, R. F., Meyer, A. L., & Tidwell, R. P. (2003).
Impact of the RIPP violence prevention program on rural middle school
students. *The Journal of Primary Prevention , 24*(2), 143-167.

Farrell, A. D., Meyer, A. L., & White, K.S. (2001). Evaluation of
Responding in Peaceful and Positive Ways (RIPP): A School-Based
Prevention Program fro Reducing Violence Among Urban Adolescents.
Journal of Clinical Child Psychology, Vol. 30, No. 4, 451-463.

Farrell, A. D., Valois, R. F., & Meyer, A. L. (2002). Evaluation of the
RIPP-6 Violence Prevention Program at a Rural Middle School. American
Journal of Health Education , 33(3), 167-172.

Gold, M.R., J.E. Siegel, L.B. Russell, M.C. Weinstein (Eds). (1996).
Cost-Effectiveness in Health and Medicine. New York, NY: Oxford
University Press.

Indicators of School Crime and Safety (2003). National Center for
Education Statistics; Bureau of Justice Statistics, Department of
Justice. "School related" is defined as occurring at, or on the way to
or from school.

Janega, J. B., Murray, D. M., Varnell, S. P., Blitstein, J. L.,
Birnbaum, A. S., & Lytle, L. A. (2004) Assessing the most powerful
analysis method for schools intervention studies with alcohol, tobacco,
and other drug outcomes. Addictive Behaviors, 29(3), 595-606.

Murray, D. M., (1998).Design and Analysis of Group Randomized Trials.
New York: Oxford University Press.

Polsky, D., H.A. Glick, R. Willke, and S. Dunmeyer. (1997). Confidence
intervals for cost-effectiveness ratios: A comparison of four methods.
Health Economics 6(3): 243-252.

Shadish, W.R., Cook, T.D., & Campbell, D.T. (2002). Experimental and
Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA:
Houghton Mifflin Company.

Snedecor, G. W., & Cochran, W. M. (1989). Statistical Methods (8th ed.).
Ames, IA: The Iowa State University Press.

Varnell, S., Murray, D. M., Janega, J. B., & Blitstein, J. L. (2004).
Design and analysis of group-randomized trials: A review of recent
practices. American Journal of Public Health, 94(3), 393-399.

Zarkin, G.A., L.J. Dunlap, and G. Homsi. (In Press). The substance abuse
services cost analysis program (SASCAP): a new method for estimating
drug treatment services costs. Evaluation and Program Planning.

### Exhibit 12 --

### Statute Authoring the Evaluation

### 

### SEC. 4121. FEDERAL ACTIVITIES.

> \(a\) PROGRAM AUTHORIZED- From funds made available to carry out this
> subpart under section 4003(2), the Secretary, in consultation with the
> Secretary of Health and Human Services, the Director of the Office of
> National Drug Control Policy, and the Attorney General, shall carry
> out programs to prevent the illegal use of drugs and violence among,
> and promote safety and discipline for, students. The Secretary shall
> carry out such programs directly, or through grants, contracts, or
> cooperative agreements with public and private entities and
> individuals, or through agreements with other Federal agencies, and
> shall coordinate such programs with other appropriate Federal
> activities. Such programs may include ---
>
> \(1\) the development and demonstration of innovative strategies for
> the training of school personnel, parents, and members of the
> community for drug and violence prevention activities based on State
> and local needs;
>
> \(2\) the development, demonstration, scientifically based evaluation,
> and dissemination of innovative and high quality drug and violence
> prevention programs and activities, based on State and local needs,
> which may include ---
>
> \(A\) alternative education models, either established within a school
> or separate and apart from an existing school, that are designed to
> promote drug and violence prevention, reduce disruptive behavior,
> reduce the need for repeat suspensions and expulsions, enable students
> to meet challenging State academic standards, and enable students to
> return to the regular classroom as soon as possible;
>
> \(B\) community service and service-learning projects, designed to
> rebuild safe and healthy neighborhoods and increase students\' sense
> of individual responsibility;
>
> \(C\) video-based projects developed by noncommercial
> telecommunications entities that provide young people with models for
> conflict resolution and responsible decisionmaking; and
>
> \(D\) child abuse education and prevention programs for elementary and
> secondary students;
>
> \(3\) the provision of information on drug abuse education and
> prevention to the Secretary of Health and Human Services for
> dissemination;
>
> \(4\) the provision of information on violence prevention and
> education and school safety to the Department of Justice for
> dissemination;
>
> \(5\) technical assistance to chief executive officers, State
> agencies, local educational agencies, and other recipients of funding
> under this part to build capacity to develop and implement
> high-quality, effective drug and violence prevention programs
> consistent with the principles of effectiveness in section 4115(a);
>
> \(6\) assistance to school systems that have particularly severe drug
> and violence problems, including hiring drug prevention and school
> safety coordinators, or assistance to support appropriate response
> efforts to crisis situations;
>
> \(7\) the development of education and training programs, curricula,
> instructional materials, and professional training and development for
> preventing and reducing the incidence of crimes and conflicts
> motivated by hate in localities most directly affected by hate crimes;
>
> \(8\) activities in communities designated as empowerment zones or
> enterprise communities that will connect schools to community-wide
> efforts to reduce drug and violence problems; and
>
> \(9\) other activities in accordance with the purpose of this part,
> based on State and local needs.
>
> \(b\) PEER REVIEW- The Secretary shall use a peer review process in
> reviewing applications for funds under this section.