The author(s) shown below used Federal funds provided by the U.S.

Department of Justice and prepared the following final report:

Document Title: Alameda County Placement Risk Assessment

Validation, Final Report

Author(s): Sharon Jones; Chris Baird

Document No.: 189240

Date Received: July 23, 2001

Award Number: 1998-JB-VX-0109

This report has not been published by the U.S. Department of Justice.

To provide better customer service, NCJRS has made this Federallyfunded grant final report available electronically in addition to

traditional paper copies.

Opinions or points of view expressed are those

of the author(s) and do not necessarily reflect

the official position or policies of the U.S.

Department of Justice.
Grant Award: # 1998-JB-VX-0 109 

Project Period: 1 ON98 - 8/3 1/00 

Headquarters Office 

(510) 208-0500 Fax (510) 208-051 1 

Midwest Office 426 South Yellowstone Dr., Suite 250 Madison, WI 53719 

(608) 831-8882 Fax (608) 831-6446 

1970 Broadway, Suite 500 Oakland, CA 94612 

This document is a research report submitted to the U.S. Department of Justice. This report

has not been published by the Department. Opinions or points of view expressed are those

of the author(s) and do not necessarily reflect the official position or policies of the

U.S. Department of Justice.
ALAMEDA COUNTY PLACEMENT RISK 

ASSESSMENT VALIDATION 

Executive Summary 

The Alameda County Probation Department was awarded a grant from the National 

Institute of Justice in 1998 to develop a risk assessment for probation placement cases. The 

project was a follow-up to a process that began in 1996 when the Department contracted with the 

National Council on Crime and Delinquency (NCCD) to construct a five-year plan to improve the 

effectiveness and efficiency of their Juvenile Services Division. NCCD was instrumental in 

developing the planning approach for graduated sanctions for the Office of Juvenile Justice and 

Delinquency Prevention’s Comprehensive Strategv for Serious, Violent, and Chronic Juvenile 

Ofleenders. In August of 1996, the Department adopted this strategy as its official governing 

policy for all system design and policy decisions. Part of this plan was to develop a risk 

assessment instrument for placement of adjudicated juveniles. 

The goal of the project was to implement a system-wide classification and placement 

system that would address the public concern for safety and effectiveness in dealing with juvenile 

crime. It would use a structured process that would assess the risk of future recidivism in 

combination with the severity of the current offense. This risk assessment project would develop 

a scientific and rational basis for making classification and placement decisions. It would ensure 

that extra-legal factors were not used in classification and decision making. Further, it would 

structure the process such that juveniles would be held accountable for delinquent behavior. 

The plan for the Department’s decision malung was shaped by four criteria: validity, 

reliability, equity, and utility: 

Validity: Does the system measure what it purports to measure? Does it 

accomplish its goals? 

Reliability: Do similar cases receive similar recommendations for placement 

services? 

Equity: Is the system fair to various groups? 

Utility: Is the system useful to practitioners? Is it simple to implement? 

To begin the development process, NCCD worked with a committee of Probation Unit 

Supervisors and assisted in the adaptation of a risk assessment instrument from an existing 

instrument that had been used and validated on probationers in California. This instrument would 

only address the relative risk of recidivism of the youth and would not take into account the 

1 

This document is a research report submitted to the U.S. Department of Justice. This report 

has not been published by the Department. Opinions or points of view expressed are those 

of the author(s) and do not necessarily reflect the official position or policies of the 

U.S. Department of Justice.
severity of the current offense. That factor would be included in a structured decision-making 

system that would be developed after the completion of the risk assessment instrument. 

For the validation study, Alameda County adopted the sample risk level cut-off scores. 

Previous studies had found that these scores were accurate in distinguishing between groups of 

offenders that had significantly different rates of re-offending. Youths classified as medium-risk 

were twice as likely to re-offend than youths classified as low-risk. Similarly, high-risk youth 

were twice as likely to re-offend than medium-risk youth. 

Nine probation officers (deputies and supervisors) were trained in the use of the draft 

instrument. Because of their current involvement in the system, all coders were familiar with the 

case files. The first phase of data collection involved a random sample of 500 cases that received 

field supervision as a disposition in 1996. Field supervision refers to a sanction in which the 

youth was maintained and supervised in the community with the aid of weekend programming, 

community service, restitution, electronic monitoring, or day reporting. The second data 

collection phase involved a random sample of 500 cases that received a placement order in 1996. 

Placement refers to any sanction in which the youth is placed out of home (e.g., a group home, 

camp, or residential treatment). From these populations it was possible to extrapolate to the total 

population of youths on field supervision (n=1,334) and the total population of youth in 

placement (n=774). Further, it was possible to determine the profile of the total populabon of 

youth under probation supervision (n=2,108). A number of data elements were collected on each 

child, and outcomes for each child were tracked for a year after disposition. 

After the data were cleaned, the total sample yielded 954 probation cases that were 

evaluated by the risk assessment instrument and followed through one year for intake actions, 

petitions filed, and petitions sustained. Three cases with zero scores were initially excluded as 

they were thought to be data collection errors. However, these cases were later confirmed and 

added back to the database for the total of 954. With these data, NCCD refined the risk 

assessment instrument, collapsing certain categories in which there was little difference in 

recidivism. 

When the instrument was first examined, it did not adequately distinguish the risk of 

recidivism for youth classified as low-, medium-, and high-risk. About half of the high-risk youth 

had a new sustained petition after one year, but almost a quarter of the low-risk youth did as well. 

Three steps were taken to improve the instrument: 

Risk assessment questions were simplified, making the instrument easier to use and 

increasing the difference in recidivism for the groups; 

2 

This document is a research report submitted to the U.S. Department of Justice. This report 

has not been published by the Department. Opinions or points of view expressed are those 

of the author(s) and do not necessarily reflect the official position or policies of the 

U.S. Department of Justice.
e A clearer delineation of risk for girls and boys was achieved through adjustment of values to 

one question; and 

0 The overall scale was adjusted to increase the instrument's predictive capacity. 

The sample of 954 cases was predominantly male, with only 28.3 percent (n=270) being 

female. Most of the sample consisted of African-American youth, which reflects the juvenile 

justice population in Alameda County. Youth between the ages of 14 and 15 were'the largest age 

group in the sample (45.2 percent, 11431) with youth 16 or older (36.5 percent, n=348) and 13 or 

younger (1 8.3 percent, n=175) following. 

There was about an even split in the disposition outcomes for the sample, with youth 

going to field supervision (50 percent, n=480) slightly higher than the number going to placement 

(49.6 percent, n=474). This split is relatively reflective of the general probation population in 

Alameda County. A 1995 study showed that 56.7 percent of youth went to field supervision, 

while 43.3 percent went to placement (including camp, n=1,610). 

There were some interesting differences between males and females on the final risk 

assessment score. Females tended to be older, had parents with better skill levels, and had fewer 

school problems. Males had more extensive criminal backgrounds and more institutional 

commitments. More females tended to have negative peers, but there were more gang members 

among the males. 

Whites were older and had lower rates of criminal history than other groups, but higher 

rates of drug and alcohol use. African-Americans were younger than the overall average, had 

more institutional commitments, and lower rates of drug and alcohol use. The rate of AfricanAmerican youth with parents lacking skills was higher than for other groups. They also had the 

highest rate of severe truancy. Hispanics had higher rates of constructive parenting, but the 

lowest rate of school attendance. They also had considerably higher rates of gang membership 

than other groups. 

Results of an analysis of the risk assessment questions also show that the problems of 

these youth were interrelated. For youth with little or no parental support, only 4 percent were 

attending school. The majority of these youth (85 percent) had severe truancy problems or had 

been expelled. More than half of these youth were involved with drugs, compared with only 30 

percent of those with constructive parenting. Drug use was also related to school performance. 

Only about 30 percent of those with good school records reported drug use, but almost 60 percent 

of those with serious problems in school had used drugs or had a problem with drugs. (Alcohol 

use was low among the entire population.) The majority of youth in the sample had delinquent 

peers, regardless of the skill level of their parents. 

3 

This document is a research report submitted to the U.S. Department of Justice. This report 

has not been published by the Department. Opinions or points of view expressed are those 

of the author(s) and do not necessarily reflect the official position or policies of the 

U.S. Department of Justice.
Total risk scores ranged from 0 to fifteen, with the majority of the cases falling between 

seven and nine. When these scores were grouped in risk categories, most of the cases in the 

sample fell into the “medium” risk category. A quarter fell into the high-risk category, and only 

about 11 percent fell into the low-risk category. 

Again, the goal of a validation study of a risk assessment instruments is to determine if 

the instrument is predictive of aggregate rates of recidivism. Youth in the sample who scored in 

the higher risk categories should have higher rates of recidivism than youth in the lower risk 

categories. It should be noted that while youth who score in the higher categories in the risk 

assessment might have higher rates of recidivism, the rates might not be as high as expected. 

Even though no risk assessment was in use at the time that placement decisions were made, it is 

reasonable that the good judgment of probation officers would cause higher risk youth to be sent 

to secure settings or assigned to intensive levels of supervision. As a result, those youth would 

have less opportunity to recidivate than the lower risk youth, who would be more likely to be 

placed on less intensive probation supervision. 

The risk assessment was evaluated for three measures of recidivism: intake actions, 

petitions filed, and petitions sustained through one year after the placement decision was made. 

Only about 60 percent of the total sample had another arrest after one year. These data clearly 

showed the difference in rates of recidivism among the low, medium, and high-risk levels. While 

only a quarter of the low-risk youth had an arrest after one year, more than three-quarters of highrisk youth did. The rate of recidivism for the medium-risk category was about the average for the 

total sample. 

The next measure of recidivism that was examined was the rate of youth with a petition 

filed after one year. Again, the data showed that youth in the lower levels of risk were much less 

likely than youth in the higher levels of risk to have a petition filed after one year. Three quarters 

of the high-risk youth had a new filed petition, while only about 20 percent of the low-risk youth 

did. The rate of recidivism of the medium-risk category was again about the same as the total 

sample. 

The next measure - petitions sustained -- was probably the best measure of recidivism of 

the four measures, because the youth actually had a finding in court that the incident of 

recidivism occurred. The number of youth with at least one sustained petition was tracked 

through one year after original disposition. High-risk youth were more than five times as likely 

as low-risk youth to have a sustained petition one year after disposition. The rate of recidivism 

for the medium-risk offenders was just slightly lower than the overall average. 

4 

This document is a research report submitted to the U.S. Department of Justice. This report 

has not been published by the Department. Opinions or points of view expressed are those 

of the author(s) and do not necessarily reflect the official position or policies of the 

U.S. Department of Justice.
The risk assessment instrument was effective in predicting which youth were more likely 

to be arrested, have a petition filed, and have a petition sustained. Thus, the instrument was valid 

in predicting risk of recidivating. An analysis of each of the risk assessment questions found that 

each one was valid in predicting risk of recidivism. 

The study also looked at whether similar cases received similar treatment through the risk 

assessment. Males and females fell into the medium category at equal rates. Males were more 

likely to score in the high-risk level than females. An analysis of each of the risk assessment 

questions showed that males scored higher than females on the factors of age (females were 

older), criminal history (males had more extensive criminal backgrounds), institutional 

commitments (males had more commitments), and peer relations (more males were in a gang). 

On the other hand, females scored higher on the measure of lack of parental skills. 

The study also looked at the fairness of the outcomes by risk level. In other words, do 

different groups of youth in the same risk levels re-offend at the same rates? This analysis would 

demonstrate if the risk assessment was more effective or less effective for various groups. Rates 

of petitions sustained after a year were higher for males than females, but high-risk youth were 

far more likely than low-risk youth to have a sustained petition: females eight times as likely and 

males almost five times as likely. 

More Ahcan-American and Hispanic youth fell into the high-risk category than did 

White youth. A third of Hispanic youth and 27 percent of Black youth were classified as highrisk, as compared with 19 percent of White youth. Although there were few Asians in the study, 

a greater percentage fell into the low-risk group than other races. An analysis of each of the risk 

assessment questions revealed the factors that contributed to the variance among races. AfricanAmericans were generally younger, had more extensive prior criminal records, more institutional 

commitments, parents who were lacking skills, and records of suspensions and expulsions from 

school. On the other hand, African-Americans were rated as having fewer problems with drugs 

and alcohol than the other races. Hispanics scored higher on the criminal history and school 

problem measures. The numbers in this analysis were very low, but the data show that for all 

racial groups high-risk youth were far more likely than low-risk youth to have a sustained 

petition. The recidivism rates for African Americans and Whites were higher than those for 

Hispanics. 

The sample was also tracked by disposition. Probation officers were making placements 

that were similar to those that would be indicated by the risk assessment. Only a small percent of 

cases fell into the low-risk category and were sent to placement; likewise, a small percent of cases 

in the high-risk category were placed on field supervision. Data showed that the rate of 

5 

This document is a research report submitted to the U.S. Department of Justice. This report 

has not been published by the Department. Opinions or points of view expressed are those 

of the author(s) and do not necessarily reflect the official position or policies of the 

U.S. Department of Justice.
recidivism for youth on field supervision increased dramatically among the low, medium and 

high levels of supervision. High-risk youth were six times more likely to have a sustained petition 

within one year than low-risk youth, and more than twice as likely as medium-risk youth. The 

pattern was the same for placement cases. 

NCCD presented these findings on April 18,2000 to a meeting of juvenile court judges, 

probation supervisors, and other County staff. The consensus of the meeting was that this 

instrument would be of help in making placement decisions. Probation officers will be 

completing the risk assessment form and submitting the score to the judge as part of their 

recommendations for placement. At a later date, the risk assessment score may be included in a 

placement matrix with severity of offense to indicate appropriate placements. 

This risk assessment study of Alameda County probationers not only yielded a validated 

instrument that could assist staff in making placement decisions, it directed the County toward 

areas of need. As was addressed in the first section of this document, the County was involved in 

implementing a continually-changing juvenile justice action plan. The results of the data 

collection effort for this grant were so revealing that new projects were proposed. 

First, the County came to realize how many youth on probation had severe truancy 

problems, and how strongly truancy was linked to other problems in youths’ lives. They used the 

study to propose to use Juvenile Accountability Incentive Block Grant funds for a truancy 

prevention project. The lead to that proposal states: 

Truancy is notably one of the highest predictors of chronic juvenile crime. According to a 

1996 sample study [the risk assessment data collection effort] 35 percent of those on 

probation in Alameda County are exhibiting severe truancy or behavior problems. 

Factors that appear to be correlated with poor attendance in school are poor parenting 

skills, poor peer support and influence, and drug and alcohol use. For example, among 

those exhibiting severe truancy or behavior problems in school, or have graduated or 

received a GED certificate, for whom 54 percent have parents with “generally 

constructiveH parenting skills. Only 3 percent of probationers exhibiting chronic truancy 

behavior have “good” and “supportive” peer relationships. By contrast, 17 percent of 

those who are regularly attending school, or have graduated or received a GED certificate 

have “good” and “supportive” peer relations. 

The proposal, which was funded, is a model truancy prevention strategy in two schools to 

improve attendance among youth on probation. The program includes child accountability, quick 

response to truancy, intensive family work, and a multi-disciplinary approach to case 

management. 

The project also led the County to examine the risk assessment that is used for intake to 

detention. The focus of the study was to determine the needs and risks of these youths such that 

6 

This document is a research report submitted to the U.S. Department of Justice. This report 

has not been published by the Department. Opinions or points of view expressed are those 

of the author(s) and do not necessarily reflect the official position or policies of the 

U.S. Department of Justice.
they can receive better probation services and be prevented fi-om offending. The results of this 

study were presented to the County on August 25,2000. 

Finally, the placement risk assessment project caused probation leaders to think about 

how resources are being spent in the department. Are probation efforts focused on those youth 

with the highest likelihood of re-offending, or are funds spread evenly across wards? This 10- 

month study will look at workload standards, the number of staff necessary for adequate 

supervision, and the time that needs to be allocated to each youth. This study will help the 

Department develop standards for probation work and deploy resources more effectively. The 

workload project will adopt the placement risk assessment categories of risk in setting these time 

standards. For example, how many probation hours should a low-risk offender require as 

opposed to a high-risk offender? 

This study showed that the placement risk assessment developed by Alameda County was 

valid, reliable, and fair for that juvenile justice population. Therefore, the instrument can be 

useful for staff who are making informed placement decisions. No risk instrument should take 

the place of the good judgement of probation officers and judges. However, this risk assessment 

offers a way to consider eight relevant factors in making that decision. These factors have been 

supported by juvenile justice research as being indicators of the risk of recidivism. 

June 1,2001 

PROPERTY OF 

National Criminal Justice Reference Service (NCJRS) 

Box 6000 ,,#. 

Rockville, MD 20849-6000 

7 

This document is a research report submitted to the U.S. Department of Justice. This report 

has not been published by the Department. Opinions or points of view expressed are those 

of the author(s) and do not necessarily reflect the official position or policies of the 

U.S. Department of Justice.