NJJEC Bulletin May 2012

The NJJEC Bulletin

NJJEC is a project of the 

Justice Research and 

Statistics Association 

funded by the Office of 

Juvenile Justice and 

Delinquency Prevention

(OJJDP). NJJEC’s purpose 

is to improve the 

evaluation capacity of 

states, localities, and tribes 

and facilitate the use of 

evidence-based programs 

and practices in juvenile 

justice. The NJJEC Bulletin

is disseminated directly to 

subscribers and posted on 

the NJJEC website, and 

provides information 

related to juvenile justice 

evaluation that is of 

interest to state, local, and 

tribal juvenile justice 

professionals. 

We welcome your 

participation! If you would 

like to submit an article, 

suggest a news item for 

the Bulletin, or ask a 

question about evaluation, 

e-mail njjec@jrsa.org. For 

more information about 

the resources available 

through the National 

Juvenile Justice Evaluation 

Center, visit the NJJEC 

website. 

What’s Going On at 

NJJEC?

In conjunction with OJJDP, 

NJJEC held a webinar entitled 

Data Collection and Analysis on 

March 22 for 413 attendees. This 

webinar was the fourth and final 

webinar in the series Assessing 

Project Performance: Building 

Blocks of Evaluation and 

Performance Measurement. 

Topics included techniques for 

collecting accurate, high quality 

data, best practices for recording 

and storing data, techniques for 

using data and sharing results, 

and the use of data collection 

and analysis for program 

development and sustainability. 

Slides, audio, and other 

materials from the four-part 

webinar series are available on 

the NJJEC website. Thank you 

to everyone who participated! If 

you have ideas for future 

webinar topics, please e-mail

njjec@jrsa.org. 

On May 2, an NJJEC staff 

member participated in a 

webinar for tribal grantees 

entitled You Can’t Do Something 

With Nothing: Getting Your Data 

and Using It, hosted by OJJDP 

and Education Development 

Center’s Tribal Youth Program. 

The webinar focused on the 

basics of performance 

measurement and evaluation, 

and included an extensive 

discussion of data collection, 

storage, and reporting 

techniques. Audio and slides 

are available on the Tribal 

Youth Program website. 

Upcoming Events of 

Interest

Coalition for Juvenile Justice 

(CJJ) Annual Conference

June 2012

The Coalition for Juvenile 

Justice will hold its annual 

conference in Washington, DC 

June 21 to 24. The theme of the 

conference is “Improving 

Justice Outcomes for Youth 

and Families: Uniting Science, 

Policy, and Practice.” Check 

the CJJ website for the agenda 

and to register.

Office of Justice Programs 

Financial Management 

Seminar

June 2012

The Department of Justice’s 

Office of Justice Programs will 

host a series of seminars on the 

financial management of 

discretionary and formula 

grants. The live seminars will 

be held in Washington, DC 

June 20 to 21. While 

registration is already at 

capacity, OJP offers a free 

training online for direct award 

recipients with a current grant 

or cooperative agreement.
NJJEC Bulletin Page 2

Featured Resources: Does “What 

Works” Work for You?

There are a number of databases that provide 

information about “what works” to criminal 

justice practitioners, based on rigorous 

evaluation that has found programs to be 

effective. Examples of these resources include 

Crimesolutions.gov, the Model Programs 

Guide, and Blueprints for Violence Prevention. 

Each database reviews and rates the evidence 

for specific programs, and provides a profile for 

each program that includes the type of service 

or intervention involved, information about the 

data collected and research that has been 

conducted, a description of the target 

population, and the implementation process.

These resources are valuable for juvenile justice 

practitioners, but do not necessarily provide 

program options that address every situation. 

It may be necessary to make changes to an 

existing program so that the program can be 

effective in a specific context. 

Consider the nature of the problem you are 

trying to address, as well as the characteristics 

of your target population and the location in 

which you are operating the program. For 

example, an anti-bullying program might need 

to be altered to account for racial or cultural 

tensions. An alcohol abuse prevention program 

demonstrated effective for middle school youth 

might have no impact on high school seniors 

without accounting for age differences. An 

anti-drug program that has been successful in 

urban areas may need to be changed in order to 

be effectively implemented in a rural area.

This has become a source of confusion for 

many in the juvenile justice community, 

because we know fidelity in implementation, or 

following the program plan, is an important 

part of evidence-based practice. 

How can each and every jurisdiction use 

research evidence to support its efforts in

juvenile justice? How are we able to make 

informed decisions about program innovation? 

What elements are essential for program 

effectiveness, and what components can be

changed to meet local needs?

One possible solution to this problem is to 

consider the use of evidence-based practices or 

components. Such evidence-based strategies

often cross model or promising programs—that 

is, a similar approach has been found to be 

effective in different programs and for varying 

target populations. This approach might be 

likened to deciding to eat healthfully rather 

than following a strict diet—eating well is 

important for everyone, but certain diets might 

not be effective depending on personal 

characteristics such as age. Towards this end, 

we need to consider the elements that are 

critical to effectiveness in different contexts. 

Safety, Fairness, Stability: Repositioning Juvenile 

Justice and Child Welfare to Engage Families 

and Communities, a paper by Dr. Joan Pennell, 

Carol Shapiro, and Dr. Carol Spigner published 

by the Center for Juvenile Justice Reform,

provides an example of this approach: engaging 

the components of a system in using the “traits 

of effective practice” (p. vi). The paper 

discusses the use of family engagement across 

child welfare and juvenile justice agencies. 

Family engagement is defined as directly 

involving families in social services affecting 

children. Principles of family engagement are 

found in model programs such as MultiSystemic Therapy (MST) and Functional Family 

Therapy (FFT), which include family members 

in the delivery of therapy for violent, disruptive, 

or substance-abusing youth. Family 

engagement is the basic strategy used by 

evidence-based programs like MST and FFT to 

address problem behaviors in youth.
NJJEC Bulletin Page 3

The paper notes an important consideration in 

using evidence-based components or strategies: 

how do these strategies work? In the case of 

family engagement, the authors explain that 

family participation can help hold youth 

accountable, press family members into being 

involved in the youth’s behavioral 

improvement, and reinforce the youth’s 

support system. 

Understanding the “how” rather than just 

knowing the “what” helps us to consider ways 

this strategy might be applied in a specific 

context. For example, a juvenile justice 

programmer in a rural area is still able to use 

family engagement without necessarily 

employing a psychologist. The frequency and 

type of interaction between the youth and 

his/her family is the critical component. 

Awareness of program characteristics and the 

mechanisms by which they effect change can 

assist juvenile justice practitioners in the 

creation of new strategies or modification of 

existing interventions. This ensures that these 

new strategies employ findings from research 

evidence, even if the program does not adhere 

to a specific model program plan.

Evaluation Feature: 

How Can I Measure Something 

That Doesn’t Occur?

Collecting Outcome Data for 

Prevention Programs

Many juvenile programs focus on preventing 

negative behaviors by youth. For example, 

programs try to prevent youth from bullying 

other youth, using drugs and alcohol, or 

becoming a victim of an Internet predator. 

Developing good performance measures to 

assess outcomes for these programs can be 

difficult because we are trying to measure 

something that, if the intervention is successful, 

will not occur. However, if we understand 

program logic—the “how” and “why” of the 

program’s expected effect on the problem—we 

can use this information to assess behavior 

prevention.

One way to assess prevention is to administer 

pre- and post-tests to youth who participate in 

a program to determine if their attitudes and/or 

actions have changed over time. Collect 

information on the outcome of interest at the 

beginning of the program (this is called baseline 

data) and again once the program has been 

completed. While an evaluation is needed to 

link program activities to any observed changes 

in attitudes or actions, simply comparing this 

information before and after the program is a 

good way to see if expected changes have 

occurred. For prevention programs, this 

expected change typically involves 

improvement in attitudes or reduction in 

negative behaviors.

SAMHSA’s National Registry of Evidence-Based 

Programs and Practices provides information 

about substance abuse and mental health 

interventions that have been demonstrated 

effective. One program reviewed by NREPP is 

the SANKOFA Youth Violence Prevention 

Program, a product of The Center for Healthy 

Schools, Families, and Communities. The goal 

of the program is to prevent violence and 

victimization among 13- to 19-year-old African 

American youth.

The program, measures three different types of 

outcome data: fighting and bullying behaviors, 

violence-related bystander behaviors, and 

personal victimization. These data provide 

information on the youth’s behavior towards 

other youth, victimization of youth, and the 

youth’s responses to viewing violence (e.g. 

watching a fight as a bystander rather than 

walking away). Because this information is 

collected for program youth before and after 

they have participated in the program, it can be 

used to examine changes over time. 
It is important for juvenile justice practitioners 

to review research evidence or databases of 

evidence-based programs such as NREPP to 

understand what types of data are typically 

collected to assess the effectiveness of 

intervention and prevention programs. These 

resources provide information and ideas on 

ways to measure program activities and 

outcomes, and possible sources from which 

these data might be collected. 

State Spotlight: Georgia

The ability to report quality data across 

agencies serving youth is vital to planning and 

assessing juvenile justice initiatives. Data can 

be used to describe the nature and extent of 

problems, demonstrate progress, and provide 

information valuable for forming strategies that 

address at-risk or justice-involved youth. 

Accurate data are useful to state residents, 

educators, funding agencies, community 

programs and services, and the juvenile justice 

system. However, collecting this information 

and reporting it in an accessible and systematic 

way can be difficult.

Georgia has met this challenge by creating the

Georgia Juvenile Justice Data Clearinghouse. 

The goal of the project is to provide current and 

accurate juvenile justice data. Visitors to the 

project website are able to obtain information 

about juvenile offending and legal responses 

such as referrals, diversions, and commitments. 

The data are available statewide, and can be 

viewed by county or jurisdiction. The site 

includes a valuable but often-neglected element

in data reporting: definitions of the terms and 

sources. This information promotes uniformity 

in data reporting—ensuring that each county 

and agency reports the same type of 

information derived from the same sources. 

The website is maintained by The Georgia 

Governor’s Office for Children and Families

(GOCF). The project is a result of the

cooperative efforts of Georgia’s Council of 

Juvenile Court Judges, Department of Juvenile 

Justice, Georgia Bureau of Investigation, and 

the Georgia Public Defender Standards Council.

NJJEC Bulletin Page 4

NJJEC Bulletin is the newsletter of the National Juvenile Justice Evaluation Center. It provides information on NJJEC 

project activities, evaluation-related activities of OJJDP grantees, and upcoming trainings and conferences of interest.

To access NJJEC project information and resources: http://www.jrsa.org/njjec/

To subscribe: http://www.jrsa.org/njjec/newsletter.htm

To unsubscribe: e-mail njjec@jrsa.org

NJJEC is a Project of the Justice Research and Statistics Association. 

Notice of Federal Funding and Federal Disclaimer: This document was prepared under Grant Number 2010-JF-FX-0063 from the 

Office of Juvenile Justice and Delinquency Prevention, Office of Justice Programs, U.S. Department of Justice. Points of view or 

opinions in this document are those of the authors and do not necessarily represent the official position or policies of the 

U.S. Department of Justice.