Chapter 1: Introduction 


This chapter presents an overview of the Community Prevention Grants Program model. Also discussed 

are the implementation and design of the national evaluation. 

Community Prevention Grants Program Model 

The Community Prevention Grants Program is a federal grants program that funds collaborative, 

community-based delinquency prevention efforts guided by six underlying principles: comprehensive 

and multidisciplinary approaches, research foundation for planning, local control and decisionmaking, 

leveraging of resources and systems, evaluation to monitor success, and a long-term perspective. In the 

Community Prevention Grants Program, these fundamental principles combine to form a strategic 

approach to reducing juvenile delinquency. They also provide a sound framework for the program’s 

practical application. 

The key features of the Community Prevention Grants Program—specifically, its risk- and protectionfocused approach, research-based planning, communitywide interventions, and local flexibility—make it 

a distinctive initiative. The program is intended to help communities support strong and healthy families 

and, ultimately, to result in fewer youth in the juvenile justice system. 

Established by Congress in the 1992 reauthorization of the Juvenile Justice and Delinquency Prevention 

Act of 1974 (subsequently, the Act), the Community Prevention Grants Program is a comprehensive, 

nationwide approach to reducing juvenile delinquency and related adolescent problem behaviors. It is 

based on a risk- and protection-focused model of prevention that asserts that, in order to reduce juvenile 

delinquency, communities must reduce the risk factors identified as predictors of adolescent problem 

behaviors. The model also asserts that communities must enhance known protective factors (e.g., social 

bonding, healthy beliefs and clear standards for behavior, and opportunities to contribute to the 

community) that provide buffers against risk factors (Hawkins and Weis, 1985). The Community 

Prevention Grants Program is unique in that it supports the development and implementation of riskand protection-focused prevention strategies that meet the discrete circumstances and risk conditions of 

local communities throughout the nation. 

The Community Prevention Grants Program’s structure is designed to provide communities with a 

guiding framework for building healthy communities in an objective, systematic, and comprehensive 

manner. As outlined in the Program Guideline (Federal Register, Vol. 59, No. 146), Title V awards 

funding to participating states and territories, which then make subgrants to qualified units of local 

government for delinquency prevention programming. The 50 states, the District of Columbia, and the 

U.S. territories (subsequently, the states) are eligible to apply for program funds, provided they have 

both a state agency designated by the Chief Executive under Section 299 (c) of the Act and a State 

Advisory Group (SAG). State awards are based on a formula determined by the state’s population of 

youth below the maximum age limit for original juvenile court delinquency jurisdiction. With approval 

from the SAG, each state agency awards subgrants to units of local government (e.g., communities) 

through a competitive process. 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 1 
To be eligible to apply for a subgrant from the state, a community must first: 

• Receive SAG certification of compliance with the Act’s core protections established under the Title 

II, Part B, Formula Grants Program. 

• Convene or designate a local prevention policy board, comprising 15 to 21 individuals representing a 

balance of public agencies, nonprofit organizations, private business and industry, and youth and 

parents. 

• Submit a 3-year delinquency prevention plan describing the prevalence of identified community risk 

and protective factors and how they will be addressed. 

• Provide a 50-cents-on-the-dollar match, either in cash or in kind, of the subgrant award amount. 

SAGs are authorized to establish additional eligibility criteria for subgrant awards on the basis of need 

or other program-related criteria. 

Since 1994, federal resources provided through the Community Prevention Grants Program have helped 

more than 1,200 communities in 49 states and territories implement this model of juvenile delinquency 

prevention, reduce risk factors for youth, and enhance protective factors. 

Underlying Principles 

The Community Prevention Grants Program provides states and communities with both funding and the 

guiding framework for reducing risk factors, enhancing protective factors, and decreasing the occurrence 

of juvenile problem behaviors. Specifically, the program integrates the following six principles to form a 

comprehensive and strategic approach to reducing juvenile delinquency: 

• Comprehensive and multidisciplinary approaches. The Community Prevention Grants Program 

requires communities to either designate or form a multidisciplinary prevention policy board that 

includes representatives from across the community (e.g., representatives from human services 

agencies, education, justice, law enforcement, public and mental health agencies, local government, 

religious institutions, and private industry). The program also promotes comprehensive risk 

reduction through the implementation of research-based prevention programs and services that 

address multiple risk factors. 

• Research foundation for planning. The Community Prevention Grants Program requires each 

potential grantee community to conduct a thorough, data-driven assessment of its local community 

to identify risk and protective factors, existing resources, and gaps in services for youth and families. 

Communities use their findings to select proven or promising research-based prevention strategies to 

implement as part of their local Title V initiative. 

• Local control and decisionmaking. The Community Prevention Grants Program allows each state 

to establish its own process for determining the number and amount of grant awards to individual 

communities. In addition, each local community that receives Title V funds is allowed flexibility in 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 2 
planning, developing, and implementing a comprehensive delinquency prevention plan that best 

meets its unique risk- and protection-focused profile. 

• Leveraging of resources and systems. The Community Prevention Grants Program requires local 

grantees to secure a 50-percent match of resources if a match is not provided by the state. This 

requirement provides the incentive communities need to secure additional, much-needed local 

resources. As “seed” money, the awards made by the Community Prevention Grants Program are 

intended to provide communities with a financial base for prevention efforts, one that communities 

can eventually sustain on their own. 

• Evaluation to monitor program success. The Community Prevention Grants Program supports 

communities in conducting evaluation activities. Evaluation activities allow local stakeholders to 

assess progress, refine the community’s prevention strategies over time, and optimize effectiveness. 

• Long-term perspective. The Community Prevention Grants Program does not propose quick-fix 

solutions to longstanding juvenile problem behaviors. Instead, the program’s structure and 

guidelines help communities think about the long term. The 3-year delinquency prevention plan—a 

trademark of the Community Prevention Grants Program—is intended to move communities away 

from historical “hit-and-miss” approaches to problem solving and toward long-term strategic 

community planning and change. 

These program principles are intended to guide collaborative, community-based prevention efforts. 

Implementation 

To help communities make the transition from theory to action, the Community Prevention Grants 

Program is built on four key implementation stages—community mobilization, assessment and 

planning, implementation of prevention strategies, and institutionalization and monitoring—with each 

stage following and building on the previous one (figure 1.1). 

Figure 1.1: Implementation Stages of the Title V Community Prevention Grants Program 

Stage 3: 

Stage 1: Stage 2: Implementation of Stage 4:


Community Assessment Prevention Institutionalization


Mobilization and Planning Strategies and Monitoring


Introduce community 

prevention to key 

officials and leaders 

Form prevention 

policy board 

Participate in 

community prevention 

training 

Identify risk factors, 

existing prevention 

resources, and 

promising delinquency 

prevention programs 

Develop 3-year 

comprehensive 

prevention plan 

Initiate prevention 

services and activities 

Identify and leverage 

other resources for 

prevention 

Monitor ongoing 

program activities, 

indicators of risk, and 

protective factors 

Analyze indicators for 

program planning 

Monitor impacts on 

juvenile delinquency 

and other problem 

behaviors 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 3 
At each successive stage, communities acquire skills and achieve certain goals that are designed 

ultimately to strengthen their capacity to implement and sustain comprehensive delinquency prevention 

strategies. The specific activities and goals of each stage are as follows: 

• Community mobilization. This stage has two phases. In the first phase, key community leaders are 

convened to participate in community team training. Bringing these leaders together gains local 

support for a comprehensive, community-based prevention strategy by introducing them to the 

principles and benefits of risk- and protection-focused delinquency prevention and long-term 

prevention planning. In the second phase, the community’s prevention policy board—designated by 

the local leaders—attends a 3-day workshop. The community data collection training is designed to 

teach community members how to conduct a community risk and resource assessment that includes 

data collection and analysis. 

• Assessment and planning. This stage has two key components. First, applying skills learned during 

the community data collection training workshop, members of the prevention policy board conduct a 

risk and resource assessment in their community to identify local risk and protective factors, existing 

prevention program resources, and resource gaps. Next, using the results of this assessment, board 

members work together to develop a comprehensive, 3-year delinquency prevention plan that 

outlines the community’s risk- and protection-factor profile and identifies the research-based 

strategies the community will implement to reduce risk factors and strengthen protective factors. The 

plan also specifies how the community will obtain and coordinate existing and future financial and 

program resources. To help communities choose effective and research-based prevention strategies, 

board members attend a community delinquency prevention plan development training session. This 

training familiarizes communities with a wide variety of programs and strategies that have 

demonstrated effectiveness in reducing risk factors and enhancing protective factors. Once the 3­

year plan is completed, it will serve as the community’s application to the state for a Community 

Prevention Grant Program subgrant award. 

• Implementation of prevention strategies. After receiving a subgrant award, communities are ready 

to implement their delinquency prevention plans. Because each community assessment yields 

different needs and resources, the type, scope, and combination of programs and services 

implemented varies from community to community. For example, one community’s risk and 

resource assessment may indicate the need for new afterschool recreation services and youth 

leadership development activities, while another community may find a need to improve the 

coordination of its existing resources to serve a target population more effectively. In line with its 

guiding principles, the Community Prevention Grants Program does not emphasize the 

implementation of new services. Instead, it first encourages the integration and coordination of 

existing services and prevention efforts and then, when necessary, the implementation of new, 

promising, and effective programs and services. 

• Institutionalization and monitoring. Once prevention programs, resources, and data collection 

systems are in place and operating, communities are required to monitor program activities to track 

progress toward chosen goals and objectives. Communities must have an evaluation plan that 

includes methods for ongoing assessment of program activities and outcomes and for periodic 

reevaluation of risk and protective factors. 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 4 
Together, the six underlying principles and the four implementation stages give rise to a dynamic, 

national prevention initiative, one that does not restrict participating communities to a prescribed 

intervention, but rather provides a guiding framework and the tools communities need to plan and 

implement risk- and protection-focused prevention strategies. As a result, the more than 1,400 

communities that have received Title V funds in the last 9 years are implementing prevention strategies 

that share a number of common characteristics, but also vary greatly in the characteristics, magnitude, 

scope, and intensity of their interventions. 

Understanding and implementing the Title V model have proved challenging for many communities. 

Title V represents the first time some communities have engaged in a structured, data-driven planning 

process, which can pose a challenge even for communities that have previously engaged in a similar 

process. To help communities implement the Title V model effectively, OJJDP incorporates training and 

technical assistance, described in detail in the following section, into the program model. 

Training and Technical Assistance 

The Title V model assumes that communities receiving substantial training and technical assistance will 

implement more effective, comprehensive prevention planning processes than those that receive less 

support. To facilitate ongoing support to communities for implementing the Title V program model 

effectively, OJJDP has offered training and technical assistance to states and communities across the 

country since 1994, the first year Title V funds became available. This training and technical assistance 

includes pre-award assistance to help potential Title V grantees develop the knowledge and skills 

necessary to negotiate each stage of the comprehensive, risk- and protection-focused planning process 

successfully. 

Additional, individualized training and technical assistance became available to communities in 1998. 

OJJDP supports two training and technical assistance vehicles specific to the Community Prevention 

Grants Program: the Title V Training Curriculum and individualized technical assistance provided under 

a broad-based technical assistance contract offered in coordination with OJJDP state representatives. 

Title V Training Curriculum 

Though training and technical assistance have always been an integral part of the Title V initiative, the 

training model and training provider have changed over the course of the Program. From 1994 through 

2000, trainers from Developmental Research and Programs, Inc., of Seattle, WA, delivered a training 

curriculum called Communities That Care. Grounded in 30 years of research, the curriculum was 

designed to provide communities with the skills necessary to progress successfully through the four 

implementation stages of the Title V model. Although communities were not required to apply the 

Communities That Care approach, the strategy was well suited to implementation of the Title V model. 

Many communities adopted it, including communities in Pennsylvania, Michigan, and Hawaii, three of 

the national evaluation states. 

The training provided by Developmental Research and Programs was initially conducted in two phases. 

The first session, key leader orientation, was a 1-day workshop for the major policymakers, business 

leaders, and high-level agency executives in communities interested in applying for Title V funds. The 

primary purposes of the workshop were to familiarize the community’s leadership with the theoretical 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 5 
basis of risk-focused prevention and to secure the commitment of community leadership to a long-term, 

comprehensive, risk-focused prevention strategy. The second session, risk and resource assessment 

training, was a 3-day, hands-on workshop for local prevention policy board members and staff who were 

to be involved in the development of the local delinquency prevention plans. This training taught 

communities how to apply the research framework of risk and protective factors to the assessment of 

community risks and resources. 

In 1999, OJJDP made a third session, promising approaches, available to the states to help communities 

improve the match between prevention approaches and their unique community risk- and protectivefactor profile. During this session, which was also part of the Communities That Care curriculum, 

community teams learned about prevention programs and system change strategies that had 

demonstrated effectiveness in reducing risk factors. They also learned how to assess the suitability of 

these programs and strategies for their communities and to create action plans for enhancing existing 

resources and implementing new programs. 

In April 2000, OJJDP awarded a contract to a new Title V training and technical assistance provider, 

Developmental Services Group, Inc. Whereas the initial Title V training curriculum had been based 

primarily on a risk-focused approach to delinquency prevention, the new training curriculum, developed 

with input from state juvenile justice specialists and Title V prevention coordinators, was intended to 

present a more integrated, balanced approach to prevention planning and implementation by combining 

risk- and protection-focused prevention. 

Between April and July 2000, OJJDP and Developmental Services Group conducted four regional focus 

groups with more than 30 juvenile justice specialists and state Title V coordinators to gain insight into 

the effectiveness and applicability of past training sessions. The key recommendations were that the 

training curriculum be made more flexible, so it could meet specific needs of participating communities, 

and that the training help community members engage in the Title V process more effectively. As a 

result, the curriculum’s first training session was shortened and, rather than being taught at a regional 

level, was brought to each individual community. 

Focus group participants also made other suggestions. First was that the training curriculum include a 

variety of risk- and protection-focused models, including models based on resiliency and assets. In 

response, the new training curriculum includes asset and resiliency prevention models in addition to the 

risk- and protection-focused approach. Participants also suggested that community members could 

benefit from a tool to assist with the sometimes daunting task of collecting data for the risk and resource 

assessment. In response, an easy-to-use Community Data Collection Manual was developed. Finally, 

participants suggested that more examples of successful, research-based prevention strategies be 

provided. To accommodate this request, Developmental Services Group developed a science-based 

Model Programs Guide that presents program, evaluation, and contact information about more than 250 

programs that meet selection criteria for effectiveness. 

On the basis of these recommendations, a new training curriculum was made available to communities 

in 2001. It includes three training sessions for communities interested in applying for Title V funds: 

• The first session, community team orientation, is conducted in each community interested in 

applying for Title V funds. The goal of this half-day training is to bring together key local leaders 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 6 
and all members of the prevention policy board and give them an overview of the Title V model and 

its components, focusing on topics such as community mobilization, team building, and collection of 

data on state and local risk and protective factors. 

• The second session, community data collection and analysis, is a 2-day training designed to help 

participants review, analyze, interpret, prioritize, and present their collected risk- and protectivefactor data. Community members also learn how to assess their resources, identify gaps in their 

resources, and craft a community profile and an assessment report, which form the basis of their 

Title V delinquency prevention plan. 

• The third training session, community plan and program development, centers on the development 

of the 3-year comprehensive delinquency prevention plan. The 1-day training accommodates 

multiple teams of six to eight participants who represent critical sectors of their communities. It 

focuses specifically on developing a community plan, including selecting research-based effective 

and promising prevention strategies, assessing the suitability of programs for the community, 

developing measurable goals and objectives, and developing a timetable for implementation. 

Of the 11 communities that participated in the national evaluation, 8 had at least one person who was 

associated with the initiative attend training. The two Nebraska communities were not represented in 

training sessions, and one of the Virginia communities, having been trained in a similar community 

planning model under a different initiative, opted not to have a representative attend the Title V training. 

The Hawaii community sent two individuals to the training, the project coordinator and a prevention 

policy board member, but only after the grant had been awarded. Most frequently, those attending 

training were project directors or coordinators, and, in communities where more than one individual 

attended, board members were generally in attendance. Communities who participated in training were 

represented at the key leader orientation, the risk and resource assessment, or both.1 The details 

regarding training participation and its effect on communities’ ability to implement the Title V model 

effectively are presented in the case study presentations (see chapters 2–7). 

Developmental Services Group also provides individualized training to meet states’ and communities’ 

specific needs. For example, when a state or community has specific technical assistance needs, or if the 

series of three training sessions does not fit a state’s particular funding cycle, customized training and 

technical assistance are offered. 

OJJDP also provides other technical assistance to states and communities on an as-needed basis. Title V 

subgrantees can access training in a variety of interest areas and technical topics through their OJJDP 

state representative. Technical assistance activities under Title V include strengthening a community’s 

conceptual understanding of risk- and protection-focused prevention and teaching communities how to 

maintain and build on existing collaborative relationships. 

Training Challenges 

Although the effect that training has had on Title V grantees has not been evaluated empirically, 

anecdotal reports gathered from state juvenile justice specialists, Title V coordinators, and local 

 The national evaluation communities attended training during the period of time when Developmental Research and 

Programs, Inc., was the training contractor for OJJDP. 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 7 

1
community members indicate that training has been effective in teaching community members the skills 

necessary to effectively understand and implement the Title V model. Despite the reported value of 

training, however, some issues pose a challenge to state representatives and community members alike. 

The following paragraphs describe these challenges, and chapters 2 through 7, the case study 

presentations, explore the implications these challenges have for the national evaluation communities. 

Logistics at the State Level 

For training to be effective, it must be offered during times that maximize participation (e.g., not in the 

summer months or during major holiday seasons) and that coincide with states’ funding cycles. 

Unfortunately, in some states the training is offered months before the initiation of the state Title V 

request for proposals. By the time the request for proposals is announced, community members have 

forgotten what they learned in training or must scramble to pull together materials they have not thought 

about for months. In other states, training is not offered each year, even though a Title V request for 

proposals is announced annually. As a result, grantees who apply for funds during the years when 

training is not offered must negotiate the Title V planning process without the benefit of training. Also, 

some Title V grantees attend training after being awarded a grant, though the training information is of 

particular value during the planning phase of the initiative, when assessment activities are undertaken 

and the 3-year plan is developed. 

In addition to the problems with timing, the logistics of training can be overwhelming to a sometimes 

already overburdened juvenile justice specialist or Title V coordinator. Selecting dates, finding a 

convenient location, distributing announcements, handling questions and registrations, and coordinating 

each aspect of the training with the training provider can be very labor intensive, and add to the burden 

of a staff that may already be overseeing several grant programs and also conducting their usual 

administrative duties. 

Mandated Attendance 

States mandating participation in training as a pre-requisite for submitting a Title V grant application is 

another factor that affects whether understanding and implementing the Title V model pose a challenge 

to state representatives and communities. The national evaluation team found that states that mandate 

training are more committed to making sure it is available to all potential grantees, and that they offer at 

least one full training curriculum each year. Of the six states participating in the Title V national 

evaluation, only two—Pennsylvania and Vermont—require participation in training as a prerequisite for 

submitting an application. Although the other national evaluation states encourage participation, they 

permit communities to submit a Title V grant application without having attended training. Applicants 

can generally expect to receive a Title V subgrant as long as their proposals meet the criteria and 

sufficient funds are available. However, communities that do not receive training are often less familiar 

with the Title V model and are more likely to have difficulty implementing it. 

Attendance at Training 

Who attends training is important, as is the consistency in attendance across the three training sessions. 

Because each training session builds on concepts taught in the previous session, having at least some of 

the same individuals attend all three sessions is important. Because training is offered before grants are 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 8 
awarded, however, the individuals who attend the training sessions, such as key community leaders and 

prevention policy board members, are sometimes not the same individuals who are involved in the 

initiative’s later stages, such as implementation and monitoring. When they are not, a disconnect in 

information often exists between the individuals who planned the initiative and those who implement it, 

and the implementers are sometimes distanced from the goals and objectives of the overall initiative. 

The national evaluation team found that some program staff who were implementing prevention 

strategies were not even aware they were part of a larger initiative. Pennsylvania is the only state in the 

national evaluation that requires all communities to have at least one person attend all three training 

sessions. 

Staff and Prevention Policy Board Turnover 

Because they typically involve key community and agency leaders, who tend to be very busy and even 

over-committed, local prevention policy boards sometimes experience turnover. Also, many of the 

national evaluation communities encountered turnover among project directors and coordinators. When 

these are the same individuals who attended training or have become familiar with the Title V initiative, 

their departure can create information gaps that challenge the capacity of the board or project director to 

implement and monitor the Title V model effectively over time. Turnover not only challenges the 

cohesiveness of a group, it also shifts the knowledge base of the effort. 

Training Processes 

Despite the challenges described above, by providing states and communities the training resources 

necessary to develop and implement comprehensive, collaborative prevention efforts to reduce juvenile 

delinquency and related problem behaviors, OJJDP has helped states and communities learn and apply 

new and effective methods for creating and sustaining positive community change. Across the country, 

training and technical assistance have helped thousands of communities, including some of the national 

evaluation communities. More than 1,400 communities have received Title V funds and are 

implementing local delinquency prevention strategies. These communities share a number of 

characteristics, but they also vary greatly in the magnitude, scope, and intensity of their initiatives. The 

effectiveness with which training and technical assistance are able to help different types of 

communities gives Title V grantees the flexibility they need to address their unique risk- and protectivefactor profiles. 

National Evaluation of the Community Prevention Grants 

Program 

The key features of the Community Prevention Grants Program—its risk- and protection-focused 

approach, communitywide interventions, and local flexibility—make it a distinctive initiative aimed at 

establishing communities with strong and healthy families and fewer youth entering the juvenile justice 

system. Implementing Title V on a large scale has created a significant opportunity to evaluate the 

program’s effectiveness in reducing juvenile crime, delinquency, and other adolescent problem 

behaviors. 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 9 
However, many key features of the program (i.e., its comprehensiveness, locally determined prevention 

components, and dynamic planning and programming) and differences among communities also pose 

special challenges for Title V model evaluation, at both the local and national levels. The 

implementation of the national evaluation design and the modifications required so that local 

subgrantees could participate fully in the evaluation process are described below. 

National Evaluation Design 

Traditional experimental and quasi-experimental evaluation methodologies rely on the ability to isolate 

recipients of a well-specified treatment, or intervention, and compare their outcomes to an equivalent 

group that did not receive the treatment or intervention. In a comprehensive, community-based initiative 

like the Community Prevention Grants Program, however, varying members of a community are 

targeted for an overlapping array of prevention strategies and activities that are assumed to be 

interdependent and only yield the desired end result collectively over an extended period of time. The 

difficulties of evaluating such initiatives have received concentrated attention in recent years. 

In their 1995 publication New Approaches to Evaluating Community Initiatives, Connell et al. identify 

six attributes of comprehensive community-based initiatives that make them challenging to evaluate. 

These attributes also characterize the Community Prevention Grants Program evaluation: 

• Horizontal complexity. Interventions cut across multiple agencies, programs, and practices. 

• Vertical complexity. A range of prevention strategies and activities involving multiple target groups 

(individual, family, school, peer groups, communities) is implemented and intended to be 

synergistic. 

• Contextual issues. Local political, economic, and social context variables are expected to have 

major effects on desired outcomes. 

• Flexible and evolving prevention strategies, activities, and programs. Prevention strategies, 

activities, and programs are not pre-specified; they are designed to be flexible to fit each 

community’s unique risk- and protective-factor profile and to respond over time to changes in 

community risk and protective factors, resources, and program effectiveness. 

• Broad range of outcomes. Although delinquency prevention is always the desired long-range 

outcome, short-term and intermediate outcomes include a wide range of risk and protective factors 

for specific prevention strategies, activities, and programs. 

• Absence of comparison or control groups. Appropriate within-community control groups are 

extremely difficult to identify because all residents are targeted. In addition, finding adequate 

comparison communities is highly problematic because of a host of factors, ranging from in/out 

migration and spillover effects to the uncontrollability of contextual effects and the difficulty in 

measuring the degree to which similar strategies, activities, and programs may be occurring in 

comparison sites. 

The grant award procedures in the states that receive Title V funds do not lend themselves to the random 

assignment of communities to program or control conditions. The selection of a matched community 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 10 
comparison group required by a quasi-experimental design would dictate matching communities on 

characteristics relevant to the Community Prevention Grants Program outcomes. This process would be 

extremely complex because of the number of relevant identified characteristics that would be matched 

and the potential for omitting important but unidentified characteristics. 

On the other hand, the Title V program offers several evaluation advantages. First, where the initiative 

has been attempted, a large number of communities are funded each year and across years. In 2002, the 

number of Title V communities nationwide was estimated at more than 1,400. Thus, at any given time, 

numerous opportunities exist to assess how well the Title V model holds up in practice. There is also the 

potential to create a large-enough sample to use communities as the unit of analysis. In addition, the 

Title V model provides a common conceptual framework for assessing implementation and outcomes, 

and thus facilitates the development of common measures across sites. Finally, the Title V program is 

meant to offer considerable support for data collection and other activities directed toward evaluation. 

By definition, the Title V approach requires communities to collect and analyze data on risk and 

protective factors and to use the data to select relevant interventions. It also encourages the use of 

program effectiveness data to select or modify interventions. In addition, at the time the initial design 

was conceived, many states were using funds or helping communities find outside resources to build 

evaluation capacity at the local level. In many states, however, Title V grantees were not aware that 

these resources were available and therefore did not take advantage of them. In some cases, state 

representatives were just beginning to understand the kinds of training issues emerging with 

implementation of the Title V model, including the ongoing analysis of risk factor data, and so were not 

able to respond proactively to local evaluation issues. In fact, little or no evaluation capacity existed at 

the local level, and, in most cases, communities were not fully engaged in tracking risk factors or 

program outcomes. 

Initial Design 

Developed and approved by OJJDP in 1997, the initial evaluation design was based on input from 

leading experts in the design and conduct of evaluations of comprehensive program initiatives. Focused 

on capturing information that would be helpful in testing and refining the Title V model, the evaluation 

was designed to use as much of the data available nationally as possible, but to control expense by 

minimizing the level of data collection required. To facilitate the conceptualization of the small, 

incremental links between Title V activities and outcomes, the collection of qualitative data at a 

relatively small number of sites was proposed. Quantitative methods were proposed to test key links for 

a larger number of sites. 

The national evaluation of the Community Prevention Grants Program was intended to examine the 

comprehensive, locally defined, risk- and protection-focused model’s viability and effectiveness in 

preventing juvenile delinquency. Very broadly, the design was developed to address the following 

research questions: 

• What is the impact of the Community Prevention Grants Program on risk factors, protective factors, 

and juvenile problem behavior? 

• What factors and activities lead to the effective implementation of the Community Prevention Grants 

Program model and to positive program outcomes? 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 11 
As originally conceptualized, the evaluation consisted of three interrelated levels: 

• Level I. A basic profile of Community Prevention Grants Program communities in the participating 

states and territories (e.g., number and amount of awards, participation in training and technical 

assistance). This information provides a general description of Community Prevention Grants 

Program funds and activities nationwide. The sample at Level I comprises all grantees in all states. 

• Level II. An assessment of planning, implementation, and outcome characteristics in all or most of 

the participating Community Prevention Grants Program communities in six states, performed using 

data from the Title V Community Self-Evaluation Workbook. These data are used to analyze 

community-provided impact and outcome information and to provide general implementation 

information, such as selected prevention strategies and program activity dosage. The proposed 

sample at Level II was approximately 60 Title V communities across the 6 states. 

• Level III. An assessment of the efficacy of the Title V model through intensive case studies of the 

implementation processes and the links between activities and outcomes in 12 communities (2 in 

each of the 6 Level II states).2 

Using a mixed-method design that included descriptive (Level I), non-experimental (Level II), and case 

study approaches (Level III), the three-level evaluation design was intended to move from broad 

descriptions of the Title V program in every community to increasingly detailed investigations of 

program implementation and outcomes (figure 1.2). The approach was also intended to build the 

capacity of the sites, especially at Level III, to conduct their own evaluations. 

Figure 1.2: Initial National Evaluation Design 

Methods 

Level III: Case Studies 

6 states, 11 grantees 

Level II: Implementation and


Outcome Evaluation 


6 states (HI, MI, NE, PA, VA, VT), 

All grantees (n=50–60) 

Level I: Grantee Profiles 

All states, All grantees

 Onsite interviews 

 Qualitative analysis 

Cross-site analysis of Workbook data and metaanalysis, as appropriate 

Trend analysis of Workbook data 

Technical assistance on completing Workbook forms 

 Records review 

 Descriptive analysis 

 Twelve sites were initially selected, but one of the Hawaii sites dropped out of the study early in the implementation 

process. A strained relationship between the site and the unit of local government, concern about the burden of data 

collection, and the limited resources available to the site, including financial and manpower, finally proved too much for the 

site, and it withdrew from the evaluation. Given the issues that had already compromised the evaluation and delayed its 

implementation, it was decided that the evaluation would continue with 11 sites (see sidebar 1.1). 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 12 

2
Overall, the initial evaluation design was intended to result in: 

• An ongoing description and characterization of the Community Prevention Grants Program 

subgrantees in all participating states and territories. 

• An assessment of the extent to which communities were able to implement risk-focused prevention, 

including an understanding of what planning processes were undertaken by each community, which 

risk factors were addressed, what prevention strategies were carried out, what target populations 

were served, and the magnitude and intensity of services provided, in addition to the impact of the 

Community Prevention Grants Program on hypothesized outcomes. 

• An increased understanding of the processes involved in the effective implementation of the 

Community Prevention Grants Program model and a test of the theoretical causal links between the 

risk-focused prevention model and its impacts. 

Site Selection 

Six states were recommended for inclusion in the Level II and III evaluations. The number of states 

selected to participate in the national evaluation, and hence the number of subgrantees examined, was 

initially intended to strike a balance between cost and scientific rigor. To capture the variation in levels 

of implementation and the associated variation in outcomes, and to increase the generalizability of this 

assessment to all Title V communities, all subgrants in each selected state would be examined. The 

choice of design required that the number of states be limited. Examining more states by restricting the 

Level II evaluation to exemplary communities or to 

communities that had large grants would not have Sites Participating in the National Evaluation 

captured adequately the diversity in the quality of of the Community Prevention Grants Program 

implementation at the community level. By 

examining this diversity, and by including Hawaii 

communities that had been successful in 	 City and County of Honolulu (Kaneohe community) 

implementing the Title V model, the evaluation team Michigan


would have an opportunity to gather information City of Holland (West Ottawa County) 


about the factors that contributed to or impeded City of Novi 


successful implementation. 
 Nebraska 

City of Norfolk On the basis of information from state juvenile justice City of Valentine 

specialists and the U.S. General Accounting Office 

survey conducted in 1995 and 1996, most of the states Pennsylvania 

could be grouped into two categories: those that Northampton County (city of Easton) 

contributed additional funding in support of the Fayette County (city of Uniontown) 

Community Prevention Grants Program and those Vermont 

that did not. At the time, it was thought that the Town of Middlebury 

contribution of state money to develop long-term, Town of Windsor 

sustainable, local delinquency prevention efforts 

indicated a high level of commitment by the states to Virginia 

Hanover County the Title V approach. These funds were expected to Waynesboro and city of Staunton 

improve the quality of local implementation and 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 13 
outcomes by providing additional support for planning and training at the local level. By stratifying the 

sample of states, the evaluation would be better able to assess the effect of the state context on local 

implementation experiences and outcomes. For the sample selection process, contributing additional 

funds was defined as a state’s providing a monetary supplement to the Community Prevention Grants 

Program from its general fund, Title II funds, or other federal block grant funds. 

After stratification, five criteria were used to reduce the states included in the Level II and III 

evaluations to a core group: 

• Current or planned use of the Title V Community Self-Evaluation Workbook. 

• Availability of good-quality data on risk factors and juvenile problem behaviors at the state and local 

levels. 

• Allocation of state funds for evaluation. 

• Commitment to the Community Prevention Grants Program model. 

• High participation levels in training and technical assistance. 

From the core group, the recommended states were selected in consultation with OJJDP’s State 

Relations and Assistance Division on the basis of reporting history and willingness to participate in the 

evaluation. The six recommended states were Michigan, Nebraska, Pennsylvania, Ohio, Vermont, and 

Washington. Because of their participation in other national evaluations occurring at the same time as 

the Title V evaluation, Ohio and Washington were removed from the list at the request of state 

representatives and replaced by Hawaii and Virginia. 

The number of states selected was to strike a balance between methodological rigor and funding realism. 

Studying grantees in all participating states and territories would have been prohibitively expensive. The 

sample of six states was thought to moderate the cost of the evaluation, while maintaining adequate 

statistical power and including levels of variation in implementation and outcome factors sufficient to 

investigate the research questions fully. 

The Title V national evaluation presented a unique situation for states and communities in that they were 

not given additional resources for participation. However, they would receive a benefit in the form of 

additional evaluation training and technical assistance. Early on in the selection process, finding states 

that met all five of the participation criteria was difficult. Few states were actively using the Workbook 

or had good-quality data on risk factors and juvenile problem behaviors. In addition, at the time, no 

states had set aside money for evaluation for Title V subgrantees. As a result, the federal stakeholders 

and the national evaluation team began to consider states less on the basis of their ability to meet the 

criteria and more on their willingness to participate. Given that the data collection would be somewhat 

labor intensive for local community members, having participants who were committed to the evaluation 

for reasons other than financial incentives was important. 

The evaluation team tried to ensure that the six states met the criteria, at least minimally, but it did not 

always have control over which states were chosen. The lack of adherence to the five criteria in 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 14 
choosing the national evaluation sample ultimately played a role in compromising the initial evaluation 

design, especially for Level II. It also influenced the modifications that were eventually required. 

Modified Evaluation Design 

In early 1998, before the national evaluation design was fully implemented, the evaluation team 

conducted evaluability assessments in each of the six states. An evaluability assessment is a preevaluation analysis that helps ensure that an evaluation will be technically feasible and capable of 

answering the research questions that are important to decisionmakers. Evaluation staff performed 

several assessment tasks, including, in some instances, visiting a state and its Title V communities to 

assess the sites’ capacity to participate in Level II of the national evaluation. This assessment was very 

important because it looked at implementation and outcomes on the basis of common measures across 

sites (i.e., those collected using the Workbook). Specifically, these assessments were conducted to 

determine the extent to which state and community staff could participate fully in data collection 

activities and included measuring the availability and quality of existing state and local data. 

The evaluability assessments were not encouraging. Despite attempts that had been made to ensure that 

the sample was appropriate to the national evaluation design and activities, several issues emerged from 

the evaluability assessment findings that pointed to the contrary: 

• Unfamiliarity with the Title V prevention model. Many community members representing local 

Title V initiatives across the selected states had not participated in Title V training and so were 

unfamiliar with the Title V prevention model. 

• Unfamiliarity with the Workbook. Most community members and several state staff were unaware 

of the existence of the Title V Community Self-Evaluation Workbook, the primary data collection 

tool for the Level II evaluation. In addition, communities that were somewhat familiar with the 

Workbook were not using it to collect data. 

• Lack of local evaluation and data collection plans. Almost none of the communities had local 

evaluation plans. They were not tracking risk-factor data or program-specific process or outcome 

data and had no plans to do so. Some communities were collecting limited process data, for example, 

documenting the numbers of children served by prevention programs. 

• Limited reporting. For all of the states except one, the only required reporting mechanisms for 

grantees were quarterly and final progress reports that were limited in content. The quality of these 

reports varied significantly across states. 

These findings suggested that, in its initial design, Level II could not be fully implemented in the states 

chosen for participation. Before modifying the design or the sample, both of which had taken months to 

finalize and would require additional months to modify, OJJDP program staff, in consultation with the 

evaluation team, turned their attention to the issue of training and technical assistance. The question 

became: “Given enough training and technical assistance, could these communities collect and report 

data, as required by the national evaluation?” 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 15 
To answer this question and identify potential areas for training and technical assistance, the national 

evaluation team, in consultation with OJJDP program staff, contacted the juvenile justice specialists in 

each of the six selected states. The goal was to assess fully the availability, quality, and content of 

existing data in each state, including: 

• The type of information Title V grantees were required to submit to the state on a regular basis (i.e., 

descriptive, quantitative, or qualitative data, or some combination of the three). 

• The quality of the information the Title V grantees had been submitting to date. 

• The evaluation challenges with which grantees were struggling the most. 

• The willingness of state juvenile justice specialist to help communities collect and submit data for 

the national evaluation. 

Findings from these discussions were not encouraging. With regard to reporting, in all six states grantees 

were required to submit quarterly and final reports only. Grantees were held accountable for submitting 

the reports, but not for the information included in them, which most often was limited to descriptive 

and budgetary information. In addition, in five of the six states, the consensus regarding the reports was 

that: 

• The content and quality of the information were not comprehensive (e.g., the information did not 

cover planning, implementation, and evaluation activities) and it rarely included process or outcome 

data. 

• The information was rarely consistent across grantees, and frequently did not include data about 

activities that were presented in the 3-year delinquency prevention plan. 

• The information was often unclear and difficult to understand, and, as initiatives progressed and 

prevention activities evolved away from original plans, it become increasingly difficult to use 

progress reports to monitor activities funded under Title V. 

The six juvenile justice specialists agreed that grantees did not understand evaluation, specifically, they 

thought that the Title V grantees did not comprehend the distinction between tracking risk factors over 

time (impact data) and reporting short- and intermediate-term program outcomes (outcome data), an 

important distinction in the Title V program model. In addition, many of the communities were not 

implementing initiatives on a scale large enough to affect risk factors. For example, some of the 

communities were implementing programs serving less than 100 children or families per year. Such 

numbers are large enough to show small changes in the children and families participating in the 

prevention strategies, but they are not large enough to impact risk-factor data (e.g., the number of 

juvenile arrests for substance use or the rates of child abuse and neglect). 

Several of the juvenile justice specialists admitted that they too were challenged by evaluation, 

especially by the distinction between risk-factor tracking and program evaluation. As a result, they were 

unsure how to help the local grantees collect data, despite their willingness to do so. In the end, it was 

clear that, for Level II to be implemented, frequent, intense, and ongoing training and technical 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 16 
assistance activities would have to take place in each state. Training and technical assistance had been 

built into the program from the beginning, but not to the extent required to bring the states and 

approximately 60 communities to the point of participating fully in data collection activities. 

In response to the findings that emerged from the discussions with the juvenile justice specialists, the 

national evaluation team considered developing a training and technical assistance curriculum that 

would give sufficient attention to evaluation capacity at the local level. It was determined that more than 

26 onsite regional training sessions would be necessary to reach the 60 or so grantees in the six states, 

the sample originally proposed for Level II. In addition, to continue to build evaluation skills and help 

grantees apply them, followup sessions would be necessary in most of the participating communities. 

Finally, it would take at least a year to implement the plan fully, but this would delay the national 

evaluation even further. In the end, OJJDP program staff determined that the resources to finance such 

an intensive plan were unavailable and that they did not want to wait an additional year before 

implementing the national evaluation. It was decided that, rather than develop a training and technical 

assistance curriculum to support the implementation of Level II as it was originally designed, the 

national evaluation team would modify the initial Level II design. 

By the time Level II was to be modified, the six selected states had established relationships with the 

national evaluation team and, to some extent, had already subscribed to the idea of participating in the 

national evaluation. As a result, it was decided that the original sample would remain the same. Because 

Level II was the only component of the initial design that could not be implemented, the team focused 

its efforts on re-conceptualizing Level II while retaining Levels I and III as originally designed. 

Eventually, a decision was made to collapse Levels II and III into a multiple-case study design to be 

implemented with the 11 Level III communities only. Thus, each Level III community became the 

subject of an individual case study design, with the study as a whole using a multiple-case design. The 

original data collection activities would remain the same but would be implemented with the smaller 

sample. The revised plan is illustrated in figure 1.3. 

Figure 1.3: Modified National Evaluation Design 

Methods 

Theories of change approach 

Multiple methods 

Level III: Case Studies 

6 states, 11 grantees Qualitative analysis 

Evaluation, training, and technical assistance 

Level II: Implementation and Cross-site analysis of community-level data 

Outcome Evaluation 

Evaluation, training, and technical assistance 6 states (HI, MI, NE, PA, VA, VT), 

All grantees (n=50–60) 

Level I: Grantee Profiles 

All states, All grantees

 Records review 

 Descriptive analysis 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 17 
The revised plan was acceptable for several reasons. First, the multiple-case study design still allowed 

the team the opportunity to examine the factors and activities leading to effective implementation of the 

program model and to positive program outcomes, one of the primary goals of the national evaluation. 

Second, by having 11 cases in the design, the team could pursue different patterns of replication of the 

model. The findings from the evaluability assessments suggested that Title V sites possessed 

characteristics that were both similar and considerably different in relation to the Title V model. In 

allowing the team to examine both the similarities and the differences, the multiple-case study design 

would help identify both the conditions under which communities could plan and implement the Title V 

model effectively and the factors that facilitated or impeded the communities’ ability to do so. Finally, 

the revised plan still allowed the team opportunities to collect process, outcome, and impact data, but for 

a smaller sample. 

Valid reasons existed for accepting the revised design, but several methodological issues were also 

related to the change. With more than 1,500 Title V communities nationwide, generalizing the findings 

from the 11 case studies to the larger population would be difficult. Case studies are not expected to 

generate findings that can be generalized to a large population, however. Rather, they are used to 

understand the unique experiences of one or several cases in relation to a variable of interest such as the 

Title V model. 

The revised design also posed issues for statistical power, especially in relation to the initial proposal for 

the Level II design, in which implementation and outcome characteristics were to be examined in 

approximately 60 communities across the 6 states. The inclusion of all the Title V communities in the 

six states was designed to capture the variation in levels of implementation and the associated variation 

in outcomes, and to increase the generalizability of the assessment to all Title V communities. Reducing 

the sample to 11 cases would not permit this level of assessment. 

On the positive side, however, the 11 case studies provided the team with ample opportunities to study 

processes and outcomes within sites, leading to greater robustness in conclusions. In addition, with 

Level III intact, the team could still focus on gaining a thorough understanding of the Title V 

implementation process in the 11 communities, helping to refine the Title V model and assess how 

different types of communities can best create the conditions necessary for more effective prevention 

planning and programming. Finally, working with a smaller sample gave the team the opportunity to 

work closely with the community members at the 11 sites, potentially helping influence the quality and 

type of data that could be collected. 

Selection of Case Study Communities 

Once the team had modified the Level II design, it selected the case study sites. With input from OJJDP 

program staff and the state juvenile justice specialists, along with discussions with representatives from 

potential communities to determine their willingness and capacity to participate, the team selected two 

communities from each state. Thus, the team could test the assumptions underlying the Title V model 

while holding one important contextual condition (the state) constant. 

In addition to allowing the national evaluation team to collect process and outcome data at the sites, case 

studies would allow the team to test the program model under varying contextual conditions by 

comparing communities across states. Case studies would also permit the collection of rich, detailed 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 18 
information on the implementation of the initiative, including the processes most needed to develop and 

carry out an effective, comprehensive prevention model. 

Besides input from key federal and state stakeholders and the communities themselves, several other 

criteria were considered in choosing the potential case study sites. One consideration was the relative 

level of funding for the initiative in each community. Having participating communities that had 

sufficient resources to implement an initiative large enough to make a difference and to collect data over 

time was important. To the extent possible, choosing communities that had participated in Title V 

training was desirable because the evaluation was designed to assess the effect of training and technical 

assistance activities on the ability to develop and implement effective prevention initiatives. To be sure 

that communities had comprehensive plans that included the various components of the Title V model, 

the evaluation team reviewed the plans of all proposed communities before making a final decision. 

Finally, the communities had to have demonstrated some initial success in implementing their 

comprehensive plans, as reported by community representatives and supported by the juvenile justice 

specialist and OJJDP program staff. 

Although the team tried to ensure that the proposed communities met the criteria, at least minimally, it 

did not always have control over which communities were ultimately chosen. State-level factors often 

influenced which communities were chosen, regardless of if or to what extent they met the criteria for 

participation. In Vermont, for example, two Title V communities are funded every 3 years, limiting the 

pool of potential sites and eliminating the possibility of using the criteria for site selection. In addition, 

Juvenile Justice Specialists often wanted some part in choosing which sites were selected, if only to 

ensure that those selected were having some success in implementing the model. In the end, the 

communities selected varied significantly in relation to the level to which they met the criteria. In some 

cases, communities had been through training, and in others they had not. Some communities were 

receiving as much as $150,000 over 3 years, whereas others were receiving less than one-fourth that 

amount. Interestingly, the variation proved to be an advantage, as it provided naturally occurring 

comparison groups, allowing the team to identify factors associated with both successful and 

unsuccessful implementation of the Title V model. For example, the team found that communities that 

did not receive training had more difficulty implementing the model than did those that did. Similarly, 

communities that had a thorough comprehensive plan had the most success in implementation. These 

issues and others are discussed in detail for each site in the case study presentations, which are found in 

chapters 2 through 7. 

Implementing the National Evaluation Design 

Originally scheduled for early 1998, the first phase of the national evaluation was delayed by almost a 

year because of issues associated with the initial design and its subsequent modifications. The state 

selection process caused some delays as state representatives (e.g., agency staff and juvenile justice 

specialists) worked with federal program staff to determine what would be required if their state agreed 

to participate, and if and how they could support local communities to facilitate their full participation in 

data collection activities. 

In late 1998, the national evaluation team began to work directly with four communities in Michigan 

and Pennsylvania. Approximately 6 months later, the communities from Vermont and Nebraska were on 

board and ready to begin working with the team to build local evaluation capacity and start data 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 19 
collection activities. Finally, in late 1999, after Hawaii and Virginia were added to the evaluation, two 

communities from each of those two states were chosen for participation. 

By late 1999, 11 of the proposed sites were actively engaged in national evaluation activities; however, 

the one site that was not, one of the Hawaii sites, terminated its participation in late 1999). In general, 

the staggered start dates for the sites were the result of several factors, including the states’ Title V 

funding cycles (some communities were funded earlier than others) and the time required to recruit sites 

for participation. Recruiting frequently required that a member of the national evaluation team conduct a 

site visit to establish contact with community members, introduce the national evaluation, give 

community members the opportunity to ask questions, and talk with them in general terms about the 

nature and extent of their participation. The sites and their funding schedules are presented in table 1.1. 

Table 1.1: Title V National Evaluation Community Funding Timeline 

Funding Years 

State ULG (Target Area) 1997 1998 1999 2000 2001 2002 

Hawaii City and county of 

Honolulu (Kaneohe June x September 

community)

Michigan West Ottawa County (city 

of Holland) April x x x 

City of Novi June x x June 

Madison County (city of 

Norfolk) July x New grant x x 

Cherry County (city of 

Valentine) August x x x 

Pennsylvania Northampton County (city 

of Easton) December x x December 

Fayette County (city of 

Uniontown) January x January 

Vermont Town of Middlebury July x July

 Town of Windsor July x July 

Virginia Hanover County (same) July x x July 

City of Waynesboro (city 

of Waynesboro and city of July x x July 

Staunton) 

Note: Nebraska communities are guaranteed funding for only 1 year at a time, so start and stop dates cannot be anticipated. 

Grants are awarded competitively each year. 

In the first year, national evaluation activities focused primarily on building relationships and 

developing the community members’ capacity to engage fully in data collection activities. In the early 

phases of implementation, the evaluation team visited each community approximately every 6 months, 

sometimes more frequently if necessary. In general, these visits were used to present an overview of the 

national evaluation design; foster a shared understanding of goals, objectives, and outcomes of the 

evaluation; and lay the groundwork for collaboration by the evaluation team and key stakeholders at the 

state and local levels. More specifically, early activities included: 

• Establishing collaborative working relationships at the state and community levels through frequent 

and ongoing contact with state staff, state-level evaluators, and community members. 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 20 
• Building the “state context” (e.g., state support for prevention programs, levels of funding, and 

availability of community technical assistance and training) through interviews with State Advisory 

Group members, juvenile justice specialists, and other key state-level stakeholders. 

• Assessing the community by reviewing relevant materials, such as grant applications and program 

plans; interviewing key players, including prevention policy board members, project directors, and 

program staff; and appraising existing evaluation capacity and future evaluation support needs. 

• Building state and local evaluation capacity by conducting technical assistance and training 

workshops on collecting national evaluation data, the Title V Community Self-Evaluation Workbook, 

and other state-specified evaluation topics. 

• Developing tools for data collection and management, including a data collection guide, interview 

protocols, and databases relevant to evaluation data. 

All these activities were conducted during the first year. In the second and later years, national 

evaluation activities were balanced among data collection activities, evaluation training and technical 

assistance activities, ongoing assessment of the community, and continued collaboration by the members 

of the national evaluation team with stakeholders at the state and local levels. To support the 

communities throughout the evaluation process, evaluation team members had frequent and ongoing 

communication with site representatives by telephone and e-mail, and in person during site visits. 

Because of their frequency and intensity, these contacts were instrumental in building positive and 

lasting relationships between community and national evaluation team members and helped facilitate 

capacity building, especially in relation to evaluation. Data collection varied across sites in terms of 

quality and quantity, but the commitment to participating did not. Regardless of the challenges they 

faced, all 11 communities remained fully committed to providing the data they could, and to 

participating in activities to increase their evaluation capacity to the best of their ability. However, data 

collection remained a challenge throughout the evaluation. 

Because the evaluation capacity of the sites was limited throughout the national evaluation, the 

collection of data varied across sites in terms of quality and quantity, and they were submitted 

sporadically. Despite the commitment of the communities and the evaluation team’s efforts, timely data 

collection and submission activities were an issue throughout the evaluation. In addition, because 

participation was voluntary and sites were not receiving additional resources to participate in the 

evaluation, the communities had no external incentive for engaging in timely data collection and 

submission, and the team did not have the authority to impose consequences on sites that did not fulfill 

data collection requirements. 

To encourage participation to the fullest extent possible, the team adopted a participatory, or 

collaborative, approach to the evaluation early on in the implementation process: 

One of the negative connotations associated with evaluation is that it is done to people. Participatory 

evaluation involves working with people. Instead of being research subjects, the people in the research 

settings (e.g., the community members) become co-researchers. The process is facilitated by the 

researcher or evaluator, but is controlled by the people in the community. (Patton, 2002:183) 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 21 
This type of approach shifts the framework of the evaluation. Instead of presenting the national 

evaluation as something being done to the sites, the team presented it as something the sites and the 

team would do together. This approach is less intimidating than more traditional evaluation approaches 

and generates much more buy-in from program personnel and other key stakeholders. This leads to more 

complete and accurate data collection; more clarity about program goals, objectives, and strategies; and 

increased likelihood that evaluation results will be used to improve current or future programs. 

The basic principles of a participatory approach are presented in sidebar 1.2. They were used to guide 

the following evaluation activities: 

• Identifying and planning the evaluation with key decisionmakers and stakeholders who will use the 

evaluation information or be affected by it. 

• Using logic models and other tools jointly to map out the key elements of the program, the expected 

outcomes, and the theory underlying the initiative as a foundation for the evaluation.3 

• Involving stakeholders in the identification and design of local data collection instruments, 

collection plans, and reporting formats. 

• Developing the community’s capacity to collect and interpret data by conducting training sessions 

and hands-on workshops. 

• Discussing evaluation findings regularly with 

stakeholders to identify problems, modify plans, Principles of a Participatory Approach 

and determine how to use evaluation findings to to Evaluation 

strengthen programs and policies. 

♦ The community knows best. The evaluator facilitates 

During the evaluation, these activities most frequently and supports the activities of the local program and 

evaluation. took place during site visits, which were also used as 

opportunities for conducting interviews, gathering ♦ The community is responsible for the evaluation, 

relevant data, and observing program activities. These exercises self-determination and autonomy, and 

visits helped build local evaluation capacity while monitors its own progress. 

strengthening the relationships between the national 

♦ The evaluation is a collaborative effort between the evaluation team and community members. Site visits evaluator and the community.

are discussed in greater detail below. 

♦ The intended users of the evaluation results (the 

In addition to securing buy-in and increasing the community) are directly involved in the evaluation. 

quantity and quality of data, participatory approaches ♦ Evaluation information is used primarily for program often have an impact beyond the findings or report improvement and decisionmaking. 


that may be generated from the evaluation. 


Participants are exposed to and acquire skills such as ♦ The evaluation recognizes the uniqueness of the 


conceptualizing an evaluation plan and data 	 community and its local context (e.g., social, political, 

collection, analysis, and interpretation. 	 economic, and cultural characteristics). 

Logic models are useful because, as graphic representations of programs, they identify and lay out clearly the relationships 

among program conditions, activities, outcomes, and impacts. 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 22 

3 
To promote data collection, the national evaluation team first worked very closely with community 

members to develop logic models of the local initiatives. The logic models were then used to plan and 

implement a local evaluation or, in most cases, a local data collection plan. Most communities reported 

that working with logic models was helpful to them when thinking about and managing their local data 

collection plan. 

The national evaluation team worked with community 

members to develop data collection plans that met the 

information needs at the local level but also resulted in the 

collection of data that the national evaluation team could 

use in the national evaluation. Participation in data 

collection for the national evaluation was moderate at best, 

but, without the participatory approach it used, the team 

feels confident that the data collection would have been 

compromised even further. 

To enhance the participatory, and collaborative, nature of 

the evaluation, the team was structured to encourage 

relationship building. Each site was assigned two national 

team members who worked with the community throughout 

the evaluation. These relationships provided communities 

with the support they needed to participate in data 

collection activities as fully as possible. 

Data Collection 

To address research questions, the evaluation team 

developed a comprehensive, multimethod data collection 

plan for each community. The plan was designed to 

examine the four key stages of program implementation: 

planning, implementation, outcome characteristics, and the 

efficacy of the program model. The data collection plan 

included qualitative and quantitative data sources, as 

illustrated in sidebar 1.3. Together, these data provide a 

framework for understanding both the process and the 

progress of the Title V initiative in each community. 

Data were collected over a 4-year period, during the 3 years 

the communities were receiving Title V funds and then for 

1 year after funding had ended. The quantitative data were 

collected quarterly for only the first 3 years of the 

evaluation. The interviews for qualitative data were 

conducted three times per year for 3 years. Interviews were 

conducted on site visits, and by telephone at the end of each 

funding year. In the fourth year of the evaluation, 

interviews were conducted with project directors and 

Data Collection 

Qualitative Data Sources 

♦ Ongoing individual interviews with prevention 

policy board (PPB) members, community 

coordinators, and other project staff in the early 

stages of the initiative. 

♦ Annual end-of-year process interviews with 

community coordinators, project directors, and 

some PPB members. 

♦ One-time individual end-of-grant interviews with 

project coordinators or directors and PPB 

members. 

♦ Meeting minutes. 

Quantitative Data Sources (data collected quarterly 

via data collection forms) 

The Initiative 

♦ Grant and funding information. 

♦ Target community description. 

Background 

♦ Past and present 


♦ Training and other planning activities. 


Organizational Structure and PPB 

♦ PPB roster. 

♦ PPB member information. 

♦ Diagrams of PPB. 

♦ PPB meetings. 

Vision 

♦ Vision beyond Title V. 

Budget Information 

Programs and Projects 

♦ Service delivery program description, 

implementation, and process and outcome 

assessment. 

♦ Systems change project description, 

implementation, and process and outcome 

assessment. 

♦ Special event project description, 

implementation, and process and outcome 

assessment. 

♦ Community coordinator project description, 

implementation, and process and outcome 

assessment. 

Lessons Learned 

Risk Factor Tracking 

♦ Tracking risk factors and indicators. 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 23 
coordinators and, in some cases, prevention policy board members at 3, 6, and 12 months following the 

termination of the Title V grant. 

Site Visits 

Visits to each site were made at least twice per year by two members of the evaluation team. Several 

sites required more frequent visits because of technical assistance needs. The purposes of the site visits 

were to conduct interviews, collect and review documents and records, and observe key events (e.g., 

prevention policy board meetings, significant prevention activities). Before each site visit, the team 

contacted the project director or coordinator to discuss the nature and extent of the visit and to identify 

the board members, project staff, local stakeholders, and members of the community who would 

participate in the interviews. In collaboration with local stakeholders, the team identified a core group of 

individuals at each site who would be interviewed during each site visit. Generally, this group comprised 

three to five individuals, mostly prevention policy board members, who were actively involved with the 

initiative and had been for its duration. Program staff were also interviewed periodically regarding 

program implementation, including the number of children or families being served, the types of 

activities taking place, and their assessment of program effectiveness. 

In addition to this core group, other individuals were frequently identified to participate in interviews 

because of their unique perspective, experience, or knowledge of the community. For example, to 

understand the context for prevention at the local level, in some communities, team members 

interviewed the state’s attorney or other high-level officials regarding their perspective on juvenile 

delinquency prevention. Similarly, the team might meet with the chief of police or, as was the case in 

one community, with the local pastor. These individuals were interviewed because they had information 

about the community that was interesting or important to the local initiative. Generally, because these 

individuals were not involved directly with the local Title V initiative, they were interviewed only once 

or twice. 

In addition to ongoing interviews, the team conducted end-of-year interviews with local Title V project 

directors and coordinators. The purpose of the interviews was to assess progress to date; identify 

challenges to the implementation process, review evaluation activities and plans, and identify future 

goals, objectives, and challenges. In sites with a particularly involved or knowledgeable board member, 

that individual might also participate in these interviews. The end-of-year interviews provided the team 

with a process reflection on each site’s Title V experiences. 

Submission of Quantitative Data 

The national evaluation team requested quantitative data on a quarterly basis, although such data were 

generally received less frequently. The Title V Community Self-Evaluation Workbook was intended to 

serve as one of the primary data collection tools for process and outcome data. Because many of the 

national evaluation communities were unfamiliar with the Workbook, and the team could not mandate its 

use, the national evaluation team developed a tool specifically for the national evaluation communities. 

Based on the design and structure of the Workbook, the Title V Evaluation Data Collection Guide was 

designed to provide detailed instructions on the data collection process for the national evaluation, 

including timetables for submitting data and other helpful resources. It also includes data collection 

forms for use in gathering and submitting local data to the national evaluation team and examples of the 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 24 
types of data required by the national evaluation. The Guide was made available to communities 

electronically so they could submit their local data via e-mail. 

In addition to data collection tools, the team developed data tracking forms for each community to use to 

track the data they had submitted and identify data that were still needed. The team sent these forms to 

communities approximately every 3 to 6 months in an attempt to remind them that the data were due. 

The Guide and the data tracking forms were helpful, but data collection remained a challenge throughout 

the evaluation. 

Despite the best of efforts of the evaluation team, local evaluation plans and data collection posed a 

significant challenge throughout the national evaluation. Factors related to this issue included the need 

to train staff to collect data, local priorities that often did not include evaluation, and the lack of financial 

resources for data collection and analysis. These problems are discussed in detail in the case study 

presentations (chapters 2–7) and again in chapter 8. 

Despite the challenges, however, the national evaluation yielded some very interesting findings that can 

help other Title V communities and OJJDP understand how the Title V model can be applied 

successfully at the local level (see chapter 8). 

Conclusion 

Though initially conceptualized as a project designed to assess the process and impact of the Title V 

Community Prevention Grants Program, the national evaluation of the program also provided 

opportunities for learning. 

First and foremost, the evaluation provided opportunities to learn about how communities plan and 

implement local delinquency prevention initiatives. The recorded experiences of the national evaluation 

communities, presented in chapters 2 through 7, describe in detail how each community negotiated the 

four implementation stages of the Title V model, including the factors that facilitated and hindered 

success. These experiences add to the general knowledge about what works in community prevention 

planning, and also provide information about how different communities embrace the same model and 

adapt it to their unique circumstances. 

Second, the national evaluation provided opportunities, somewhat unanticipated, to learn about 

evaluation. These opportunities were certainly related to evaluation at the local level, but also to 

evaluation and related issues at the federal and state levels. For example, before the national evaluation, 

how much evaluation capacity existed at the state level was not clear. During the evaluation, the team 

learned that evaluation capacity was limited at the state level and that this limitation often compromised 

capacity at the local level. 

These challenges forced the team, in collaboration with its OJJDP partners, to examine carefully the 

factors contributing to these challenges and to find ways to resolve them. Progress was made on a 

number of levels. For example, on the basis of input from the national evaluation team’s experiences 

with state-level staff, OJJDP included, in its regional training meetings in the summer of 2000, sessions 

on how to write a comprehensive Title V request for proposals, including sections on conducting an 

evaluation, building state capacity to support local Title V prevention efforts, and enhancing the ability 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 25 
of juvenile justice specialists and other state-level staff to support local Title V communities in 

developing and implementing local evaluation plans. 

Finally, the national evaluation provided opportunities to learn firsthand about the challenges of 

evaluating comprehensive, community-based initiatives like Title V. In addition, as the evaluation 

progressed, so did other national evaluations of comprehensive, community-based initiatives, such as the 

Drug-Free Communities Support Program and the Comprehensive Strategy for Serious, Violent, and 

Chronic Juvenile Offenders. In combination, these national evaluation team experiences can help inform 

future national evaluations of programs like Title V by identifying what works in terms of methodology, 

design, and data collection activities, and how best to support communities to participate fully in large 

evaluation projects. 

The remaining chapters of this Report present the experiences of the 11 national evaluation communities 

and lessons learned from the evaluation. Chapters 2 through 7 present in detail each community’s Title 

V experiences and the state context in which the program operates. The discussion of each community’s 

experiences analyzes the extent to which it was successful in implementing the model and the factors 

that contributed to its success. Chapter 8 summarizes the lessons learned from the national evaluation. 

Together, these chapters provide an indepth look at the process by which these 11 communities came to 

understand and implement the Title V program model, the federal- and state-level factors that both 

facilitated and hindered their progress, and the lessons that were learned by all throughout the process. 

National Evaluation of the Title V Community Prevention Grants Program—Introduction 26