DOI: 10.3886/icpsr34934
title: A Multi-site Comparison of Risk Assessments within the Juvenile Justice System, 2007-2013 [UNITED STATES]
title: Archival Version

description: These data are part of NACJD's Fast Track Release and are distributed as they were received from the data depositor. The files have been zipped by NACJD for release, but not checked or processed except for the removal of direct identifiers. Users should refer to the accompanying readme file for a brief description of the files available with this collection and consult the investigator(s) if further information is needed.


This study examined the validity, reliability, equity and cost of nine juvenile justice risk assessment instruments.  It was designed to provide a comprehensive examination of how several risk assessments perform in practice.
This study posed the following questions:

Is each risk assessment instrument sufficiently reliable (i.e., inter-rater reliability) to ensure that decisions regarding level of risk and identified service needs are consistent across the organization?; 
What specific risk assessment items are associated with less reliability? What items are rated reliably by staff?; 
 Is each risk assessment instrument valid? Specifically, what degree of discrimination is attained between assigned risk levels? Could the instrument be improved by adding or deleting specific factors and/or altering cut-off scores? ; 
 Is each risk assessment instrument valid for population subgroups: White/Caucasian, Black/African American, Hispanic/Latino, females, probationers, and youth in aftercare status? Could equity be improved by adding or deleting specific factors or altering cut-off scores?; 
 What costs are associated with each risk assessment instrument?; 



The study collection includes 31 SPSS data files



all_jais_combined.sav (n=1,141; 6 variables); 
ar_fire_final_file_ojjdp-ICPSR.sav (n=119; 205 variables); 
AR_yls_irr_FINAL-ICPSR.sav (n=155; 136 variables); 
azaoc_FINALFILE-ICPSR.sav (n=7,589; 438 variables); 
AZAOC_irr_FINAL-ICPSR.sav (n=458; 101 variables); 
AZDJC_FINAL_FILE-ICPSR.sav (n=1,265; 1,290 variables); 
AZDJC_irr_FINAL-ICPSR.sav (n=55; 120 variables); 
COMMITMENT_FINAL_SAMPLE2-ICPSR.sav (n=11,154; 719 variables); 
FinalDJJReleasesWithRecid_BothYears2-ICPSR.sav (n=90,818; 31 variables); 
FIRE_NE_COMM_FINAL_FILE_OJJDP-ICPSR.sav (n=597; 174 variables); 
fire_ne_probation_final-ICPSR.sav (n=1,077; 237 variables); 
FL_irr_FINAL-ICPSR.sav (n=519; 140 variables); 
GA_irr_FINAL-ICPSR.sav (n=509; 263 variables); 
gafire_boyscommunityALL_FINAL-ICPSR.sav (n=5,009; 781 variables); 
gafire_communityALLforretrofit2-ICPSR.sav (n=6,943; 666 variables); 
gafire_finalsampforanalysis_all-ICPSR.sav (n=7,412; 642 variables); 
gafire_finalsampforanalysis_girls-ICPSR.sav (n=2,005; 768 variables); 
jais_boys_wk_1-ICPSR.sav (n=1,989; 484 variables); 
jais_girls_wk_1-ICPSR.sav (n=745; 484 variables); 
NE_irr_FINAL-ICPSR.sav (n=727; 160 variables); 
OR_irr_FINAL-ICPSR.sav (n=477; 112 variables); 
ORE_FIRE_final-ICPSR.sav (n=12,370; 340 variables); 
PROBATION_FINAL_BOYS_ALL-ICPSR.sav (n=20,621; 837 variables); 
PROBATION_FINAL_GIRLS_ALL-ICPSR.sav (n=6,748; 849 variables); 
va_boyssample-ICPSR.sav (n=1,106; 1,273 variables); 
va_final_sample_fullscreen-ICPSR.sav (n=1,439; 1,237 variables); 
va_girlssample-ICPSR.sav (n=333; 1,256 variables); 
VA_irr_expert_FINAL-ICPSR.sav (n=10; 308 variables); 
VA_irr_worker_FINAL-ICPSR.sav (n=685; 308 variables); 
vafinalsample-ICPSR.sav (n=1,919; 1,200 variables); 
workersurveyfinal-ICPSR.sav (n=400; 69 variables);
description: The goal was to provide an objective study of the validity, reliability, and equity of different approaches to risk assessment and to review methods currently used to evaluate validity and reliability. In addition, the study sought to identify each instrument's capacity to estimate risk across different race/ethnicity groups and to identify basic cost parameters about each of the risk assessment tools.
description: The study design measured inter-rater reliability, validity and equity.  Inter-rater reliability of risk assessment was measured by asking a sample of officers/caseworkers to review case files for ten youth, observe a videotaped interview of each youth,
and score a risk assessment (or risk/needs assessment) for each youth. Validity and Equity was assessed by testing the predictive validity and equity of the risk assessments involved sampling a cohort of youth on probation or released from a facility (i.e., post-commitment). Recidivism was tracked over a 12-month follow-up period for all sites but one (where only nine months of outcomes were available). Outcome measures were obtained from agency databases and include subsequent arrests, subsequent adjudications, and subsequent placement in a juvenile facility. An advisory board consisting of researchers, a former head of a juvenile corrections department, and purveyors of the various risk assessment instruments examined in the study helped oversee study design and completion. The advisory board met seven times during the course of the project to review and approve all proposed methods of data collection and analysis, all materials used to conduct reliability testing, and all findings and results.
description: Data file all_jais_combined.sav (n=1,141; 6 variables) contains an ID variable, dichotomous variables related to assessments, and a risk level variable.


Data file ar_fire_final_file_ojjdp-ICPSR.sav (n=119; 205 variables) contains variables pertaining to prior and current offenses, family circumstances, parenting, education, employment, peer relations, substance abuse, leisure/recreation, personality/behavior, and attitudes/orientation.


Data file AR_yls_irr_FINAL-ICPSR.sav (n=155; 136 variables) contains variables related to prior offenses, behavior, substance use, attitudes, and parents.


Data file azaoc_FINALFILE-ICPSR.sav (n=7,589; 438 variables) contains variables related to risk assessment and criminal complaints, petitions, and dispositions.


Data file AZAOC_irr_FINAL-ICPSR.sav (n=458; 101 variables) contains variables related to complaints, family relationships, drug use, school attendance, behavior, friends, and running away.


Data file AZDJC_FINAL_FILE-ICPSR.sav (n=1,265; 1,290 variables) contains variables related to maniuplative behavior, empathy, respect, respect, and accountability.  There are also variables regarding conflicts in the home, education, community activities, and parents.  Additional variables relate to suicide, employment, drug use, and exposure to violence.


Data file AZDJC_irr_FINAL-ICPSR.sav (n=55; 120 variables) contains variables relate to family conflict, community involvement, family history of drug use, incarceration history of family members, and history of violent behavior.


Data file COMMITMENT_FINAL_SAMPLE2-ICPSR.sav (n=11,154; 719 variables) contains variables related to offenses, confinement, school, recreational activities, employment, family relations, drug use, exposure to violence, mental health, and behavior.


Data file FinalDJJReleasesWithRecid_BothYears2-ICPSR.sav (n=90,818; 31 variables) contains variables related to prison admission and delinquency.


Data file FIRE_NE_COMM_FINAL_FILE_OJJDP-ICPSR.sav (n=597; 174 variables) contains variables pertaining to attributes of juvenile offenders and their situations relevant to decisions regarding level of service, supervision, and programming.  They include assessment and summary of risks and needs such as prior and current offenses , peer relations, and substance abuse.


Data file fire_ne_probation_final-ICPSR.sav (n=1,077; 237 variables) contains variables related to arrests, peers, substance abuse, family, education, and behavior.


Data file FL_irr_FINAL-ICPSR.sav (n=519; 140 variables) contains variables related to mental health history, family imprisonment history, substance use history, and violence history.


Data file GA_irr_FINAL-ICPSR.sav (n=509; 263 variables) contains variables related to usual behavior and peers, personality, substance abuse and sexual behavior, school and education, and family and socialization.


Data file gafire_boyscommunityALL_FINAL-ICPSR.sav (n=5,009; 781 variables) contains variables related to arrests, sentencing, adjudications, usual behavior and peers, personality, substance abuse and sexual behavior, school and education, and family and socialization.


Data file gafire_communityALLforretrofit2-ICPSR.sav (n=6,943; 666 variables) contains variables related to arrests, adjudication, usual behavior and peers, personality, substance abuse and sexual behavior, school and education, and family and socialization.


Data file gafire_finalsampforanalysis_all-ICPSR.sav (n=7,412; 642 variables) contains variables pertaining to arrests, adjudication, sentencing, social activities, criminal associates, behaviors and attitudes, substance use, promiscuity, school, and family.


Data file gafire_finalsampforanalysis_girls-ICPSR.sav (n=2,005; 768 variables) contains variables related to arrests, adjudication, sentencing, usual behavior and peers, personality, substance abuse and sexual behavior, school and education, and family and socialization.


Data files jais_boys_wk_1-ICPSR.sav (n=1,989; 484 variables) and jais_girls_wk_1-ICPSR.sav (n=745; 484 variables) contain variables related to offenses, schools, peers, substance use, behavior, and attitudes.


Data file NE_irr_FINAL-ICPSR.sav (n=727; 160 variables) contains variables related to school, peers, behavior, and family.


Data file OR_irr_FINAL-ICPSR.sav (n=477; 112 variables) contains variables related to school, relationships, behavior, family, substance use, and attitudes.


Data file ORE_FIRE_final-ICPSR.sav (n=12,370; 340 variables) contains variables pertaining to demographics such as language and ethnicity, school issues, peer relationships, behavior issues, family functioning, substance use, attitudes, values, and beliefs, and mental health indicators.


Data files PROBATION_FINAL_BOYS_ALL-ICPSR.sav (n=20,621; 837 variables) and PROBATION_FINAL_GIRLS_ALL-ICPSR.sav (n=6,748; 849 variables) contain variables related to criminal history, school, employment, relationships, family, substance use, behaviors, and attitudes.


Data files va_boyssample-ICPSR.sav (n=1,106; 1,273 variables), va_final_sample_fullscreen-ICPSR.sav (n=1,439; 1,237 variables), va_girlssample-ICPSR.sav (n=333; 1,256 variables), and vafinalsample-ICPSR.sav (n=1,919; 1,200 variables) contain variables related to legal history, family, school, community and peers, alcohol and drugs, mental health, aggression, attitudes, skills, and employment and free time.


Data files VA_irr_expert_FINAL-ICPSR.sav (n=10; 308 variables) and VA_irr_worker_FINAL-ICPSR.sav (n=685; 308 variables) contain variables related to family, substance use, mental health, victimization, and violence.


Data file workersurveyfinal-ICPSR.sav (n=400; 69 variables) contains variables related to the opinions of juvenile justice staff who participated in the study. Respondents were asked their opinions about the risk assessment instrument design, how often they agreed with the risk level assigned by the risk assessment instrument, and whether they believed the risk assessment instrument was effective. In addition, staff were asked to provide feedback on the training they received since the assessment instrument was implemented.
description: Presence of Common Scales: 
					
The following risk assessment instruments were used in the study:

Georgia Comprehensive Risk/Needs Assessment (CRN); 
The Oregon Juvenile Crime Prevention (JCP) Risk Assessment; 
Girls Link risk assessment instrument; 
Positive Achievement Change Tool (PACT); 
Youth Assessment and Screening Instrument (YASI); 
Youth Level of Service/Case Management Inventory (YLS/CMI); 
The Arizona Department of Juvenile Corrections Dynamic Risk Instrument (DRI); 
The Arizona Juvenile Risk and Needs Assessment.; 
The Juvenile Sanctions Center (JSC) risk assessment;
description: Response Rates: 
					Not applicable
description: Datasets: 
					
						DS1: Dataset
description: record abstracts
description: web-based survey
description: Youth involved in the juvenile justice system in Arizona, Arkansas, Florida, Georgia, Nebraska, Oregon, Virginia and Solano County, California.
				   		Smallest Geographic Unit: County
description: Participants included juvenile justice agencies that had implemented risk/needs assessments
during the preceding 10 to 12 years. They represent the range of agencies that use risk/needs assessments:
county probation, state probation, and state juvenile justice systems responsible for incarcerated
youth and those in aftercare.