Analysis of Associations between 

Contemporaneous Job Corps 

Performance Measures and 

Impact Estimates from the 

National Job Corps Study

Executive Summary

January 31, 2011

Jane Fortson

Peter Z. Schochet

Contract Number:

DOLQ091A20941/DOLU101A21559

Mathematica Reference Number:

06808.300

Submitted to:

U.S. Department of Labor

200 Constitution Ave., NW

Washington, DC 20210

Project Officer: Bogdan Tereshchenko

Submitted by:

Mathematica Policy Research

P.O. Box 2393

Princeton, NJ 08543-2393

Telephone: (609) 799-3535

Facsimile: (609) 799-0005

Project Director: Peter Schochet

Analysis of Associations 

between Contemporaneous Job 

Corps Performance Measures 

and Impact Estimates from the 

National Job Corps Study

Executive Summary

January 31, 2011

Jane Fortson

Peter Z. Schochet

v

TABLES

1. Average Baseline Characteristics, by Overall Center

Performance Tercile................................................................................ 4

2. Correlations between Unadjusted Center Performance and

Adjusted Center Performance, All Components in All Years .................... 6

3. Correlations between Center-Level Impacts and

Multiyear Average Performance Ratings (Unadjusted,

NJCS-Adjusted, and ETA-652-Adjusted)................................................... 7

FIGURES

1. Design of Analysis of Job Corps Performance Measures and

Impact Estimates .................................................................................... 3

Analysis of Job Corps Performance Measures and Impact Estimates Report Mathematica Policy Research

1

EXECUTIVE SUMMARY

Job Corps is the nation’s largest vocationally focused education and training program for 

disadvantaged young people. It serves young men and women between the ages of 16 and 24 at 124 

center campuses nationwide, primarily in residential settings. The program’s goal is to prepare 

young people for successful careers. Each year, Job Corps serves more than 60,000 students at a 

cost of about $1.5 billion, which is more than 60 percent of all funds spent by the U.S. Department 

of Labor on youth training and employment services. To examine the effectiveness of Job Corps, 

the Department’s Employment and Training Administration sponsored the National Job Corps 

Study (NJCS) in 1993.

The NJCS used survey and administrative earnings records data to estimate the Job Corps 

program’s average impacts on students’ employment and related outcomes. From late 1994 to early 

1996, nearly 81,000 young people nationwide were randomly assigned to either a treatment group, 

who were allowed to enroll in Job Corps, or a control group, who were not allowed to enroll for a 

period of three years. NJCS findings are based on comparisons of the outcomes of about 9,500 

treatment group members in the research sample and 6,000 control group members. The main 

impact analysis found that Job Corps improved education and training outcomes (such as the receipt 

of General Educational Development and vocational certificates and time spent in school), 

significantly reduced criminal activity, and improved earnings and employment outcomes in the two 

years after program exit, although the longer-term analysis did not demonstrate that impacts were 

sustained beyond the two-year period.

1

The NJCS also examined the extent to which impacts (average treatment-control differences) 

on key outcomes were associated with the aggregate overall center performance measure used by 

Job Corps. The NJCS found that impacts on key outcomes were not associated with the overall 

aggregate measure of center performance.

2

This study extends the previous work analyzing the relationship between Job Corps 

performance measures and center-level impact estimates from the NJCS. We examine whether 

adjusting performance measures for student characteristics results in positive statistical associations 

between performance measures and impacts. Using linear regression, we constructed new measures 

of center performance that adjust for differences in individual and local area characteristics of center 

participants to measure the component of center performance that is not explained by these 

characteristics. We ran separate regressions on data from ETA-652 intake forms and from the 

NJCS baseline survey, adjusted not only the aggregate measure but also component performance 

measures in the Job Corps categories of program achievement, placement, and quality/compliance, 

and considered three years’ worth of performance measure data. Our goal was to test for any 

associations between center-level impacts and unadjusted or adjusted performance measures.

 Students in higher-performing centers had better 

outcomes; however, the same pattern was observed for the control group members who would have 

been assigned to those centers. 

 1 Schochet, Peter Z., John A. Burghardt, and Sheena M. McConnell. “Does Job Corps Work? Impact Findings 

from the National Job Corps Study.” American Economic Review, vol. 68, no. 5, December 2008, pp. 1864-1886.

2 Schochet, Peter Z., and John A. Burghardt. “Do Job Corps Performance Measures Track Program Impacts?” 

Journal of Policy Analysis and Management, vol. 27, no. 3, summer 2008, pp. 556-576.
Analysis of Job Corps Performance Measures and Impact Estimates Report Mathematica Policy Research

2

We address the following research questions:

• Are center performance rankings changed by regression adjustment?

• To what extent are regression-adjusted performance measures better able to distinguish 

between centers with larger impacts and those with smaller impacts?

• Are there specific performance measures (either unadjusted or adjusted) that are more 

associated with impacts than others or the summary (overall rating) measure that was 

used for the NJCS?

Figure 1 illustrates the steps in our analysis. First, we gathered performance measure data for all 

three years of the NJCS. After linking NJCS participants to Job Corps centers using intake 

counselors’ predictions of center assignment (available for both the treatment and control groups), 

we aggregated student characteristics to the center level. Then, we used regression models to adjust 

center performance measures for average student characteristics. We calculated NJCS impacts at the 

center level for seven different outcome measures, including educational services, educational 

attainment, arrests, and earnings. Finally, we compared adjusted and unadjusted performance

measures (including different components and different program years) to center-level impact 

estimates.

Our overall finding is that although regression adjustment changes the performance rankings of 

centers, the adjusted performance ratings remain uncorrelated with center-level impacts. In particular, we find 

that:

• The mix of students in high-performing centers is modestly different from students in 

low-performing centers (Table 1). 

• Regression-adjusting for characteristics accounts for some of the variance in the 

performance measures and changes center rankings, albeit not dramatically. Regressionadjusted and unadjusted performance measures are positively correlated (Table 2).

• The correlations between impacts and performance measures are generally not 

significantly different from zero (Table 3). It is noteworthy that regression-adjusted 

performance measures are no better than unadjusted performance measures in this 

regard. We find similar results whether we use the ETA-652 data or the more detailed 

NJCS baseline survey data. 

• Our findings hold for overall measures of performance as well as components of center 

performance and different program years; that is, the relationship between impacts and 

different performance measure components is also generally weak and shows no 

consistent pattern. 

• Similarly, among the subgroups we analyzed, there are no particular groups of centers for 

which relationships between performance measures and impacts are significant. 

• In contrast, when we create center-level “performance measure” constructs using 

treatment group outcomes from the NJCS follow-up survey data, we find them to be 

positively correlated with the NJCS impact estimates. Exploring these findings may be 

an avenue for future research.

In conclusion, although regression adjustment had some effect on the performance rankings, it 

did not make performance measures any more predictive of impacts. While the baseline covariates 
Analysis of Job Corps Performance Measures and Impact Estimates Report Mathematica Policy Research

3 

explain some portion of the variance in the performance measures, important unobserved 

differences among centers, possibly related to their propensity to produce impacts, appear to remain. 

Figure 1. Design of Analysis of Job Corps Performance Measures and Impact Estimates

Unadjusted Performance Measures 

PYs 1994-1996, Multiyear Average

- Overall

- Reading Gains

- Math Gains

- GED Rate

- Vocational Completion Rate

- Placement Rate

- Average Wage

- Quality Placement Rate

- Full-Time

- ARPA Rating

Step 1. Gather Performance Measure Data Step 4. Calculate Center-Level Impacts

Step 3. Regression-Adjust Performance Measures 

 for Participant Characteristics

Calculate Treatment-Control 

Differences in NJCS Follow-up 

Outcomes:

- Any Education and Training 

- Hours of Education and Training

- GED Receipt 

- Vocational Certificate Receipt

- Ever Arrested

- 1997 Annual Earnings

- 1998 Annual Earnings

Adjust for the 27 percent of 

treatments who did not enroll in a 

Job Corps center

Step 5. Compare Adjusted Performance Measures (PMs) to Center-Level Impacts to Test Whether 

Adjusting PMs for Participant Characteristics Makes Them Predictive of Impacts

Adjusted Performance Measures Center-Level Impacts

Step 2. (a) Match NJCS Participants to Centers 

Using Predicted Assignments

(b) Calculate Average Participant 

Characteristics for Each Center Using:

- NJCS Baseline Data

- Program Intake Data

- Local Area Characteristics
Analysis of Job Corps Performance Measures and Impact Estimates Report Mathematica Policy Research

4

Table 1. Average Baseline Characteristics, by Overall Center Performance Tercile

Unadjusted Overall Center 

Performance Tercile

Low Medium High p-value

Selected NJCS Baseline Characteristics 

Demographic Characteristics

Non-Hispanic White 0.245 0.344 0.391 0.023**

Non-Hispanic Black 0.558 0.479 0.249 0.000***

Hispanic 0.139 0.106 0.227 0.017**

Other race 0.058 0.071 0.133 0.040**

Female 0.388 0.361 0.399 0.663

Native language English 0.897 0.916 0.799 0.003***

Native language Spanish 0.061 0.048 0.124 0.026**

Native language other 0.042 0.035 0.077 0.066*

Age 15-17 0.443 0.447 0.427 0.708

Age 18-20 0.395 0.401 0.415 0.498

Age >20 0.162 0.153 0.158 0.817

Education and Skills

High school degree 0.151 0.152 0.191 0.034**

GED 0.042 0.047 0.057 0.147

Vocational degree 0.025 0.024 0.029 0.465

Highest grade completed 0-8 0.167 0.161 0.125 0.008***

Highest grade completed 9-11 0.659 0.657 0.649 0.803

Highest grade completed >11 0.174 0.181 0.226 0.009***

Employment History

Currently working 0.202 0.205 0.228 0.203

Earnings in past year <$1,000 0.507 0.483 0.497 0.590

Earnings in past year $1,000-$4,999 0.285 0.296 0.292 0.771

Earnings in past year $5,000-9,999 0.142 0.146 0.139 0.821

Earnings in past year >$10,000 0.066 0.075 0.071 0.596

Physical or emotional problem that limited work 0.054 0.050 0.058 0.517

Family Status

Has child 0.193 0.168 0.148 0.142

Socioeconomic Status

Did not receive food stamps over past year 0.574 0.602 0.630 0.088*

Received food stamps some of past year 0.058 0.074 0.065 0.092*

Received food stamps all of past year 0.368 0.325 0.306 0.035**

Criminal History

Ever arrested 0.267 0.298 0.301 0.231

Drug Use

Used no drugs over past year 0.711 0.663 0.622 0.001***

Used hard drugs occasionally over past year 0.040 0.058 0.101 0.000***

Used hard drugs frequently over past year 0.005 0.014 0.026 0.000***

Selected ETA- 652 Baseline Characteristics 

Demographic Characteristics

Male 0.612 0.639 0.601 0.663

Non-Hispanic White 0.266 0.368 0.429 0.020**

Non-Hispanic Black 0.587 0.511 0.269 0.000***

Hispanic 0.109 0.079 0.197 0.020**

American Indian 0.028 0.032 0.075 0.249

Asian 0.010 0.010 0.031 0.005***

Age 14-17 0.446 0.447 0.429 0.712

Age 18-20 0.393 0.401 0.413 0.493

Age >20 0.161 0.153 0.158 0.857

Education and Skills

Highest grade completed 0-8 0.178 0.175 0.124 0.002***

Highest grade completed 9-11 0.638 0.634 0.643 0.873
Analysis of Job Corps Performance Measures and Impact Estimates Report Mathematica Policy Research

5

Unadjusted Overall Center 

Performance Tercile

Low Medium High p-value

Highest grade completed >11 0.184 0.191 0.233 0.020**

Employment History

Estimated annual income $0-$400 0.010 0.014 0.022 0.042**

Estimated annual income $401-$6,528 0.236 0.265 0.283 0.210

Estimated annual income >$6,529 0.256 0.266 0.245 0.677

Estimated annual income missing 0.497 0.455 0.451 0.377

Socioeconomic Status

Receiving public assistance 0.458 0.403 0.389 0.034**

Health and Health Care

Covered by health insurance or Medicaid 0.328 0.363 0.432 0.013**

Local Area Characteristics 

Demographic Characteristics

Percentage white 0.708 0.740 0.790 0.014**

Percentage black 0.221 0.195 0.093 0.000***

Average household size 2.723 2.683 2.789 0.037**

Percentage urban 0.750 0.699 0.772 0.127

Percentage of families with a female head 0.199 0.192 0.171 0.020**

Percentage foreign-born 0.621 0.563 0.835 0.226

Total births 10512 7805 18741 0.037**

Percentage of births to teens <18 years 0.057 0.051 0.049 0.032**

Crime

Deaths by homicide and legal intervention (rate) 0.000 0.000 0.000 0.012**

Percentage of population in juvenile institutions 0.000 0.000 0.001 0.023**

Economic Characteristics

Percentage of families in poverty 0.140 0.121 0.121 0.129

Median household income 31726 33230 34064 0.049**

Percent households with income:

<$5,000 0.089 0.076 0.066 0.000***

$5,000-$9,999 0.107 0.104 0.101 0.339

$10,000-$14,999 0.099 0.095 0.097 0.590

$15,000-$24,999 0.186 0.184 0.187 0.810

$25,000-$49,999 0.320 0.330 0.338 0.003***

$50,000-$99,999 0.166 0.174 0.176 0.593

>$100,000 0.032 0.036 0.034 0.707

Unemployment rate, 16+ 0.061 0.060 0.067 0.117

Number of Centers 33 33 34

Sources: Performance measure data, NJCS baseline survey, ETA-652 intake form, 2008 ARF.

Notes: All centers are weighted equally; when constructing center-level averages, baseline 

characteristics are weighted using the baseline weight. Terciles are based on the three-year 

average overall rating. The reported p-value refers to an F-test which tests whether the three 

groups are jointly significant.

* Statistically significant at the 10 percent level.

** Statistically significant at the 5 percent level.

*** Statistically significant at the 1 percent level.
Analysis of Job Corps Performance Measures and Impact Estimates Report Mathematica Policy Research

6

Table 2. Correlations between Unadjusted Center Performance and Adjusted Center Performance, All 

Components in All Years

Correlation Between Unadjusted and Adjusted Performance Measures

NJCS-Adjusted ETA-652-Adjusted

Performance Measure PY94 PY95 PY96

Multiyear 

Average PY94 PY95 PY96

Multiyear 

Average

Overall 0.54 0.69 0.65 0.58 0.48 0.58 0.59 0.53

Reading Gains 0.54 0.74 -- 0.74 0.61 0.67 -- 0.64

Math Gains 0.66 0.82 -- 0.75 0.55 0.64 -- 0.56

GED Rate 0.53 0.64 0.64 0.49 0.39 0.54 0.57 0.45

Vocational Completion 

Rate 0.63 0.71 0.71 0.77 0.63 0.65 0.67 0.65

Placement Rate 0.55 0.56 0.51 0.55 0.48 0.51 0.52 0.49

Average Wage 0.59 0.40 0.48 0.46 0.45 0.41 0.48 0.40

Quality Placement 0.74 0.66 0.65 0.63 0.55 0.53 0.54 0.49

Full-Time -- 0.55 0.65 0.56 -- 0.50 0.54 0.52

ARPA Rating 0.72 0.62 -- 0.66 0.60 0.62 -- 0.58

Sample size = 100 centers.

Sources: Performance measure data, NJCS baseline survey, ETA-652 intake form, 2008 ARF.

Notes: All correlations are statistically significant at the 1 percent level. NJCS-adjusted and ETA-652-

adjusted ratings are based on adjustments that also include local area characteristics (from 

the 2008 ARF) but not center characteristics. NJCS adjustment controls for participant 

characteristics from the NJCS baseline survey using a forward-selection stepwise regression 

with inclusion and exclusion p-value thresholds of 0.20. ETA-652 adjustment controls for 

participant characteristics from the ETA-652 intake form, with all variables included in the 

model. Both adjustments use center-level averages of participant characteristics for the NJCS 

baseline sample. All centers are weighted equally; when constructing center-level averages, 

baseline characteristics are weighted using the baseline weight.
7

Analysis of Job Corps Performance Measures and Impact Estimates Report Mathematica Policy Research

Table 3. Correlations between Center- Level Impacts and Multiyear Average Performance Ratings (Unadjusted, NJCS- Adjusted, and ETA- 652-

Adjusted)

Overall Rating GED Rating

Vocational Completion 

Rating Average Wage Rating Placement Rating

Outcome for 

Impact Estimate Unadj

NJCSAdjETAAdj UnadjNJCSAdj ETA-Adj UnadjNJCSAdjETAAdj UnadjNJCSAdjETAAdj UnadjNJCSAdjETAAdj

Any Educational 

Services -0.02 -0.06 -0.01 -0.08 0.08 -0.02 0.08 0.06 0.07 0.09 0.13 0.15 0.05 -0.75 0.04

Hours of 

Educational 

Services 0.17* -0.03 0.08 0.02 -0.03 -0.03 0.19* 0.13 0.06 -0.01 -0.04 -0.04 0.19* 0.16 0.12

GED Receipt 0.15 -0.08 -0.10 0.12 -0.10 -0.11 0.13 0.02 -0.12 0.05 0.10 -0.08 0.13 0.06 -0.10

Vocational 

Certificate 

Receipt 0.13 -0.04 -0.01 0.00 -0.06 -0.14 0.14 0.04 0.03 -0.11 0.05 -0.07 0.23** 0.12 0.08

Ever Arrested -0.02 -0.06 -0.09 0.02 0.07 -0.03 -0.06 -0.10 -0.12 -0.04 -0.08 -0.01 -0.04 -0.04 0.03

1997 Annual 

Earnings -0.14 -0.19* -0.22** -0.22** -0.32*** -0.25** -0.08 -0.05 -0.13 0.07 0.03 -0.18* 0.03 0.04 -0.01

1998 Annual 

Earnings -0.09 -0.11 -0.11 -0.28*** -0.32*** -0.23** -0.02 -0.01 -0.01 0.05 0.02 -0.18* 0.08 0.11 0.03

Sample size = 100 centers.

Sources: Performance measure data, NJCS baseline survey, ETA-652 intake form, 2008 ARF, NJCS follow-up surveys.

Notes: Table shows the correlation based on a multiyear average of the center’s performance rating and the center-level impact estimate. NJCSadjusted and ETA-652-adjusted ratings are based on adjustments that also include local area characteristics (from the 2008 ARF) but not center 

characteristics. NJCS adjustment controls for participant characteristics from the NJCS baseline survey using a forward-selection stepwise 

regression with inclusion and exclusion p-value thresholds of 0.20. ETA-652 adjustment controls for participant characteristics from the ETA652 intake form, with all variables included in the model. Both adjustments use center-level averages of participant characteristics for the NJCS 

baseline sample. All centers are weighted equally; when constructing center-level averages, baseline characteristics are weighted using the 

baseline weight. Impacts are calculated using the follow-up weight and are adjusted for differences in participation across research groups.

* Statistically significant at the 10 percent level.

** Statistically significant at the 5 percent level.

*** Statistically significant at the 1 percent level.


www.mathematica-mpr.com

Improving public well-being by conducting high-quality, objective research and surveys

Princeton, NJ ■ Ann Arbor, MI ■ Cambridge, MA ■ Chicago, IL ■ Oakland, CA ■ Washington, DC

Mathematica® is a registered trademark of Mathematica Policy Research