Final Report

January 2008

Submitted by:

McNeil ETR

6320 Quadrangle Drive

Suite 180

Chapel Hill, NC 27517

919.493.1661

Submitted to:

Ms. Mary Vines

United States Department of Labor

Employment and Training Administration

200 Constitution Avenue, NW

Room N-5641

Washington, DC 20210
Process Evaluation 

of the 

Demand-Side Youth Offender Demonstration Project 

(Phase II) 

Final Report 

January 2008 

Submitted to 

Ms. Mary Vines 

U.S. Department of Labor 

Employment and Training Administration 

200 Constitution Avenue, NW, Room N-5641 

Washington, D.C. 20210

Submitted by 

McNeil Education, Training and Research 

6320 Quadrangle Drive, Suite 180 

Chapel Hill, NC 27517 

(919) 493-1661 
 Executive Summary 

McNeil Education, Training, and Research i

EXECUTIVE SUMMARY 

The Welfare-to-Work Partnership (The Partnership)1

, later known as Business Interface, Inc. 

(BI), was awarded funding by the Employment and Training Administration of the U.S. 

Department of Labor (DOL/ETA) in January 2003 to conduct a Demand-Side Youth Offender 

Demonstration Project (DSYODP). Under this grant, The Partnership served as an intermediary 

in connecting employers in four sites with young ex-offenders and youth at-risk of court 

involvement between the ages of 18 and 25. The DSYODP initiative built on the experience of 

The Partnership in serving welfare recipients under a DOL grant for the Welfare-to-Work 

program. 

Background 

The DSYODP initiative also built on the experience of DOL/ETA through three rounds of the 

Youth Offender Demonstration Project (YODP), which began in 1999. The goals of the YODP 

were to assist youth at-risk of court or gang involvement, youth offenders, and gang members 

ages 14-24 to find long-term employment at wage levels that would prevent future dependency 

and would break the cycle of crime and juvenile delinquency. 

McNeil Education, Training and Research (McNeil ETR) worked with DOL/ETA to conduct 

evaluations of both YODP and DSYODP Phases I and II. Phase II of DSYODP began in July 

2005 when it received a Task Order contract from DOL/ETA to conduct a process evaluation of 

the demonstration project in four cities: Chicago, Los Angeles, New York, and Washington, 

D.C. The purpose of the evaluation was to determine: 

(1) the efficacy of the business intermediary model implemented by the 

grantee; and 

(2) the impact of the services delivered on the employment, earnings and 

retention of youth ex-offenders and youth at-risk of court or gang 

involvement. 

Further, DOL/ETA expects that the evaluation will also “determine lessons from the 

implementation and operation of the DSYODP Phase II which can be shared with other 

communities wishing to replicate the business intermediary approach to serve ex-offenders and 

at-risk youth.” 

The basic strategy of the DSYODP model is to establish intermediary organizations in the 

demonstration cities, known as Business Resource Centers (Centers), whose primary purpose is 

to connect employers with youth. The “demand-side” component of the model comes from the 

demand by employers for persons who are qualified to fill positions they offer; thus, the 

employers are the primary customers of the Centers. 

 

1

 At the time of the award, the recipient organization was known as The Welfare-to-Work Partnership, though at the 

time of this report, the organization has subsequently become known as Business Interface, Inc. 
 Evaluation of DSYODP–II Final Report 

ii McNeil Education, Training, and Research 

For the youth participants, BI’s objective was to identify positions that pay “retention wages” 

that take into account the family size of the participant and the cost of living in the specific city 

of employment. By providing retention and career advancement services to the participants after 

hiring, BI hoped to encourage retention in employment for a period of at least six months. 

The substantial body of research on youth employment, youth involvement in the justice system, 

and the roles of employers and intermediaries reviewed for this research provided evaluators 

with a strong basis for understanding the goals of the DSYODP. Any evaluation of such 

programs must address their complexity by looking closely at the stakeholders involved and the 

various components of the model. 

Approach 

The evaluation of DSYODP includes two parts: the process evaluation of the Centers and an 

outcomes study of the youth’s success in obtaining employment, retention, and earning wages 

that provide financial stability. 

 

The process evaluation focused on the activities carried out by the Centers. The outcomes study 

focused primarily on the results of the matching process of meeting businesses’ needs to fill 

vacancies through placing youth offenders and youth at-risk of court involvement in those 

positions. 

There were two primary research questions of interest to DOL/ETA: 

• How does the business intermediary approach identify and recruit employers 

and broker job placements for youth ex-offenders and youth at-risk of court or 

gang involvement? How well does BI identify and recruit employers who are 

willing to hire this target population? Does the use of the intermediary satisfy 

employers’ needs? What criteria does BI utilize to hire BRC staff? What is the 

turnover rate of such staff? What are the performance requirements of such 

staff? Do they receive placement bonuses? If so, what are the results? 

• How does BI connect youth ex-offenders and at-risk youth to the services of 

the demonstration project and to One-Stop Career Center services and other 

services available in the community? How well does the intermediary connect 

youth ex-offenders and at-risk youth to services? Does the use of BI’s services 

reduce unemployment spells and otherwise improve employment outcomes 

for youth ex-offenders and at-risk youth? Does BI help youth avoid criminal 

activity and retain employment? 

Evaluators developed a series of study questions organized into six main categories that reflected 

the interests of DOL/ETA and BI, addressing the main goals of the evaluation to assess efficacy 

of the business intermediary model and the impact of the services delivered on the employment, 

earnings, and retention of youth. 
 Executive Summary 

McNeil Education, Training, and Research iii

The six categories of study questions were: 

• Development of a Network of Employers, 

• Employer Outcomes, 

• Project Organization and Staffing [at the Centers], 

• Service Delivery to Youth, 

• Participant Outcomes, and 

• Referring Organizations. 

Evaluators analyzed the questions in these categories to determine the data they would need to 

address the questions, the location of the data and the form they were in. Data were quantitative, 

located in the management information system (MIS) of BI and administrative data about 

employment and wages maintained by states in their unemployment insurance (UI) wage 

records. Other data were qualitative, collected through interviews with stakeholders in the 

demonstration cities: Employers, BI staff, youth participants, and youth development 

organizations from which BI recruited youth for placement. 

MIS data were collected directly from BI, which prepared compact discs containing their entire 

record of data from the demonstration sites. UI wage records were received from the states that 

were willing to share their records. Evaluators collected qualitative data during one preliminary 

and three extended visits to each site. Quantitative data were analyzed to provide descriptive 

statistics about the employers, the intermediary organizations, referring organizations and youth. 

Multivariate regression analyses were applied to examine retention and wage levels. Qualitative 

data were used to provide rich text documenting observations and interview findings by veteran 

field staff. 

Project Organization 

BI’s central office determined uniform goals for the staff across the demonstration sites as 

reported in Table ES.1. The Business Resource Consultant (BRC) was the primary contact with 

the employers, often with the help of the Center Managers. They were assisted by Human 

Resource Consultants (HRCs) whose main responsibility was to locate eligible and employable 

youth at youth development organizations, called referring organizations (ROs). The HRC 

screened each applicant for eligibility and for suitability for position openings located by the 

BRCs. The Retention Specialist (RS) followed up with clients after they were placed in 

employment to make sure that the match was effective and to reduce barriers that might affect 

the new employee’s job retention. 
 Evaluation of DSYODP–II Final Report 

iv McNeil Education, Training, and Research 

Table ES.1. Goals for Each Position in a Center and Its Priorities 

Position Goals Priorities 

Manager Manage staff to meet goals Achieve placement and retention goals 

BRC 1 5 placements/mo.; 90% retained Develop business relationships 

BRC2 5 placements/mo.; 90% retained Develop business relationships 

BRC3 5 placements/mo.; 90% retained Develop business relationships 

HRC 30 interviews; 15 referrals/mo. Good audit; add new agencies; refer 15 “hire-able” youth/mo. 

RS 90% retention rate Make sure youth do not leave jobs 

QCA 90% audit rate Review files; receptionist; office help 

A major hinderance for the demonstration was difficulty in finding and retaining BI staff. With 

this limitation, the intermediary model was fully operational in most sites for just over a year, 

and one site continued to experience staff turnover throughout the demonstration. Staff agreed 

that turnover was a limitation to the model that depended on building a complex network of 

trust-based relationships between BI and employers, ROs, and youth. 

Business Involvement 

BI recruited 602 businesses during the demonstration. These clients were drawn from small, 

medium and large businesses in a wide range of industries. Before working with BI, these 

businesses used few pro-active strategies for recruiting workers and they had openings for 

workers at all skill levels. Many of these business clients reported relatively high turnover rates 

for low skill workers but somewhat more stable patterns for skilled workers. The reasons 

workers gave for leaving the employ of these business clients were chiefly related to the jobs 

themselves: lack of interest, better advancement elsewhere, low pay, or lack of advancement. 

Businesses that BI recruited were looking for help with screening and recruitment. Most had 

offender policies that would limit some offenders from being placed, especially if their offenses 

involved theft, violence or felonies. Most business clients did not, however, request Federal 

bonding to offset the risk they would have from hiring an offender. Tax credits for hiring hard-to 

place workers seemed more appealing to businesses. 

From the site visit interviews, evaluators learned that BRCs used inductive processes to identify 

a small number of businesses that were always hiring entry-level staff, were open to hiring youth 

offenders or youth at-risk of court or gang involvement, and willing to pay the local retention 

wage. Once identified, BRCs knew they had to sell their services to the employers, and they 

worked hard to maintain contact to keep these reliable sources of job placements. 
 Executive Summary 

McNeil Education, Training, and Research v

Employers reported being pleased by the services BI offered them. The employers noted the 

professionalism of the BRCs and the care they took to understand the nature of the employers’ 

industry and its labor needs. They particularly appreciated savings in time and money by having 

pre-screened candidates from an organization that paid close attention to the specific job 

qualifications of their openings. While several employers commented that they were happy to 

give at-risk youth a chance, most said that the BI involvement strengthened their bottom line. 

Both the youthful job seekers and the employers reported that BI prepared candidates well for 

job interviews. 

BI staff members targeted employers that had openings all the time, so it is not surprising that the 

employers were disappointed that BI could not provide the numbers of workers they needed. 

Sometimes BRCs encouraged a youth to take a position for lower pay, an inconvenient shift, or 

fewer hours after negotiating with employers to give a salary increase or improve the hours 

within six months. 

Employers appreciated BRCs’ efforts on their behalf, but they were not willing or able to pay for 

the screening and placement services that would make BI financially independent of DOL/ETA 

funding. Staff members at the Centers were exploring other ways to finance their operations but 

the offices were closed before they were able to complete such arrangements. 

Referring Organizations 

BI recruited 243 referring organizations (ROs), 87 during Phase II. BI made it clear to the ROs 

that its goal was to place participants in career-oriented jobs and not part-time or seasonal work. 

The ROs served many populations, not just youth, including welfare-to-work clients, displaced 

workers, and persons with disabilities. 

BI worked with a variety of ROs, each offering a range of services to youth. The range of 

services included resume preparation, case management, work readiness, and soft skills training. 

ROs typically had a job placement service, so the BI engagement was not the only path youth 

had to employment. Few provided the follow-up services after placement, however, that BI 

offered. 

ROs provided computer training and opportunities for clients to complete high school by taking 

and passing General Equivalency Diploma (GED) tests. Some offered English as a Second 

Language (ESL) training as well. Some of the strategies that developed from their partnership 

with BI involved prescreening workshops, job matching with employers, and one-on-one 

preparation. 

Some ROs provided specific vocational training, such as health care, clerical skills and culinary 

experience. Most youth referred to BI arrived without such training or the training that the youth 

did receive did not match the needs of the businesses that were hiring at the time. The vast array 

of services offered by ROs could potentially have led to well-prepared BI candidates. 

Unfortunately, this was not always the case because many candidates were not job ready. BI staff 

in all the Centers mentioned their desire to receive better-qualified referrals from ROs. 
 Evaluation of DSYODP–II Final Report 

vi McNeil Education, Training, and Research 

Some clients required particular support services in obtaining work, such as assistance with 

transportation, child care and uniforms. Most of the ROs recruited by the Centers referred youth 

out to other organizations for childcare, housing, and health services rather than providing these 

services directly. ROs in all cities noted the difficulty in placing youth offenders, but BI was able 

to place some candidates that were offenders when the ROs were unable to do so. 

Overall, most of the ROs evaluators interviewed were pleased with BI’s involvement and 

enjoyed working with them. ROs appreciated the BI practice of giving feedback after youth 

screenings and job interviews. Good placements generally reflected well on the RO, and may 

have counted towards their own measures and goals. 

Participant Characteristics 

Across all four sites, 1,279 participants were enrolled. Out of these, 950 were determined eligible 

for employment and of those eligible, 413 were placed in employment at retention wages.2 New 

York had the largest number of total and eligible participants with 436 total and 398 participants 

eligible for placement. Los Angeles was the next largest with 325 total participants and 273 

eligible. Chicago had a total of 270 participants of which 152 were eligible. Washington, D.C. 

had the fewest number of participants overall, with 248 total and 127 eligible participants. 

Most participants were at-risk youth, approximately 22 years of age, single and without children. 

Of the 395 participants at-risk, 388 were considered eligible due to poverty. The two largest 

ethnic groups served were African-Americans and Hispanics. Hispanics outnumbered AfricanAmericans only in Chicago. There were slightly more females than male participants overall 

(51.9% female vs. 48.1% male), with greater variation among the sites. 

Eighty-two of the 413 youth placed in employment were employed at the time of referral to the 

project. The largest proportions of youth employed at referral were in Chicago and Washington, 

D.C. Most of the placed participants were high school graduates. 

Youth were largely recruited into DSYODP directly through the ROs. Youth also learned about 

BI from flyers posted on bulletin boards at the ROs, contact with other placed youth, or through 

friends in the DSYODP program. Youth reflected positive sentiments about the quality of 

preparation they received for employment. In general, the youth interviewed were pleased with 

having participated in a prescreening process, whether this was sponsored by an RO or BI. 

Employment Outcomes 

Of the 413 placements made by BI during Phase II of DSYODP, 18 were offenders. Overall, BI 

made fewer placements of either offenders or youth at-risk of court involvement during Phase II 

than it had during Phase I. Possibly, the decline in placements reflects the observation that the 

Centers were fully operational for fewer months in Phase II than in Phase I. These numbers do 

not include those who were placed at lower than the retention wage or for fewer than full-time 

 

2

 There are 440 placement records but 21participants were placed in employment more than once.
 Executive Summary 

McNeil Education, Training, and Research vii

hours. There were a total of 511 youth achieving employment through the four Centers whether 

or not it was at the retention wage. 

The industries that hired the most BI clients were information (e.g., telemarketing), hospitality, 

and “Other Services” (e.g., mailroom sorters or copier operators). In Chicago, the main industry 

for placements was healthcare; in New York, the most popular was information and in Los 

Angeles it was other services. There was no observable trend in the Washington, D.C. 

placements. Most placements across sites were in mid-size businesses. Most businesses hired one 

or a few BI clients, and individual ROs referred a small number of youth who were placed. 

Whether youth were at-risk or ex-offenders, all placements tended to work full-time as required 

by the project. Youth did, however, differ with respect to wages. At-risk youth tended to earn 

slightly higher wages on average than youth offenders. Overall, the placed youth earned an 

average of $9.52 an hour, which is notably less than the average wages documented in Phase I of 

the project ($9.65 for youth offenders and $10.27 for at-risk youth). Wages reported are based on 

the wage offered at the time of hiring, and some youth were prepared to work for lower wages 

just to get a job. Wages of those youth employed, but not counted as placements, were lower 

than youth who were counted as placed. 

On average, males earned less than females and married participants earned higher wages than 

singles, although there were few married youth placed in employment. Hispanics earned higher 

average wages than African-American and other ethnic groups. Participants with higher levels of 

education earned higher average wages than participants with less formal education. 

Evaluators found that termination data were not as reliable as placement data. Three of the four 

Centers were unable to hire a full-time RS even though some staff at each Center attempted to 

follow the placed youth. With that caveat, BI reported that about 20% of the placed youth 

terminated before the six-month period youth agreed to at hiring. The average retention time 

before termination for these youth was 7.3 weeks. The types of termination for those reported by 

the Centers were roughly equal between voluntary and involuntary terminations. The top five 

reasons for termination of the 82 placements reported were lateness or attendance problems 

(18.3%), poor performance (18.3%), disapproval of job or pay (14.6%), other job opportunities 

(9.8%), and personal, home-related or school conflicts with the job (9.8%). 

UI Wage Comparison Study 

Examining employment patterns with unemployment insurance wage (UI) records provides 

another vantage point for understanding the labor force outcomes for youth placed in 

employment by the intermediary compared to other youth employed without it. Because there 

was not enough time to observe the employment patterns for youth placed in Phase II, youth 

enrolled in BI during Phase I were used. 

Of the four sites solicited for UI wage records, only Florida (Phase I included Miami, and it was 

replaced in Phase II by Los Angeles.) and Maryland provided DOL/ETA with the approval to 

use data for the purposes of evaluating DSYODP. Both Maryland and Florida provided records 
 Evaluation of DSYODP–II Final Report 

viii McNeil Education, Training, and Research 

matched to youth enrolled in Phase I. For these two states, evaluators requested the wage records 

for all youth enrolled as eligible in Phase I, whether or not BI had placed them. 

Evaluators proposed a quasi-experimental study of outcomes using a comparison group of 

participants from Phase I of DSYODP. The comparison group consisted of those participants 

referred to the Miami or Washington, D.C. Centers and determined eligible for DSYODP but 

were not placed in employment through the efforts of the Centers. The distribution of 

characteristics for the comparison and placement groups was comparable by age, gender, 

ethnicity, educational attainment, marital status, offender classification, and number of children. 

Few of the youth were employed before enrolling in BI. 

Only Maryland had enough youth in both the placement and comparison groups to test for a 

difference of wages between those placed by BI and those that found work by other means. The 

median weekly wages for these two groups of Maryland youth are $153.07 for placed youth and 

$87.67 for the comparison group, a significant difference at the .05- level using the Satterthwaite 

T-test of unequal variances. 

Using the Maryland placement and comparison groups alone, evaluators tested a further 

variation to find that when individual characteristics were considered, a 1 percent increase in the 

number of employers a youth worked for was related to a 48 percent increase in average weekly 

income. Youth evidently left one employer for another as a faster way to increase wages than to 

remain with one employer and count on regularly occurring wage increases. 

Wages had a significant impact on persistence in employment. Every $100 increase in average 

weekly wages, led to an increase of 5.9 times for the proportion of enrollment quarters in which 

the youth had UI wage records, holding other factors constant. In short, youth with higher wages 

stayed employed more than youth with lower wages. This suggests that, by itself, placing youth 

in employment is not enough to strengthen tenure. Placing youth needs to be accompanied with 

higher wages to ensure that youth persist in employment. 

Youth offenders faired 0.158 times worse than youth at-risk for court involvement in the 

proportion of enrollment quarters in which youth had UI wage records. This finding reinforces 

the observation from earlier studies of the transition to work by youth offenders, that this 

population remains more vulnerable after placement than other youth. 

Conclusion 

The BI intermediary model provided services that employers appreciated. In this demonstration, 

youth from disadvantaged backgrounds, many of whom had never worked before, received 

employment at an average wage almost twice the national minimum wage. The application of the 

model was constrained by the short time the Centers were fully functioning, but the model 

warrants future applications in a more stable environment. 

Some of the limitations observed in the demonstration include the following: 
 Executive Summary 

McNeil Education, Training, and Research ix

• Training programs were not well-matched to high demand industries in the sites. 

Skills that required longer-term training, like construction, were rare. 

• Few offenders received placements; in New York, few were sent by referral 

organizations. Whether it is the demand-side model or just the limitations of this 

particular demonstration is not clear. 

• Even with retention-level wages, many workers did not get benefits, and that is a 

limitation for workers with families. 

• Youth development issues, punctuality, solving transportation and childcare 

challenges, completing high school, and acquiring soft skills continue to affect the 

labor force experience of these young workers—even with better paying jobs. 

• Staffing levels were not complete at any site, and lack of staff or turnover was serious 

at all sites. 

• DSYODP did not demonstrate viability after the DOL/ETA grant funding; there were 

sustainability options that were not pursued, but the question remains whether it is a 

sustainable model as a free-standing intermediary or as a component of a workreadiness effort. 

Several benefits worth noting were observed during the demonstration: 

• Business clients were pleased with the professional services and demeanor of the 

BRCs; they appreciated that the BRCs worked hard to understand their hiring needs; 

they appreciated that BI did not send candidates that it knew did not match the job 

position descriptions. In sum, BRCs did not waste the business client’s time and did 

provide a service they needed and that saved them time and money. 

• Businesses appreciated the pre-screening that BI performed before sending 

candidates; businesses and youth participants who were interviewed reported that the 

candidates were well-prepared for their interviews. Businesses reported that 

candidates were pre-screened for their particular openings, like giving a computer 

exercise before sending candidates out for a position with a telemarketing firm. 

Businesses were pleased that BI made sure that workers could pass their drug 

screening before sending candidates out. 

• Small business clients were particularly happy for the free service because they had 

no money for their hiring processes. 

• Business clients were pleased that the BRCs helped them meet their hiring needs, 

even with candidates that did not meet the BI placement criteria. 

• ROs that had referrals placed were pleased at the kind of jobs BI found for their 

referrals and the level of the wages they received. 
 Evaluation of DSYODP–II Final Report 

x McNeil Education, Training, and Research 
Table of Contents 

McNeil Education, Training, and Research i

TABLE OF CONTENTS 

Chapter Page 

EXECUTIVE SUMMARY ............................................................................................................. i

LIST OF TABLES......................................................................................................................... iii 

LIST OF FIGURES .........................................................................................................................v 

 

I. INTRODUCTION AND BACKGROUND ............................................................................ 1 

A. Introduction........................................................................................................................ 1 

B. Background ........................................................................................................................ 3 

II. EVALUATION APPROACH................................................................................................. 5 

A. Overview of Process Evaluation of Phase I....................................................................... 5 

B. Evaluation Design of Phase II............................................................................................ 6 

 1. Program Logic Model..................................................................................................... 7 

 2. Primary and Study Questions ....................................................................................... 10 

 3. Methodologies to Measure Outcomes........................................................................... 11 

 4. Data Requirements........................................................................................................ 13

C. Data Collection and Analysis........................................................................................... 14 

 1. Data Collection............................................................................................................. 14 

 2. Analysis......................................................................................................................... 19 

III. DESCRIPTION OF DSYODP PHASE II BUSINESS RESOURCE CENTERS................. 25 

A. Project Organization ........................................................................................................ 25 

B. Staffing............................................................................................................................. 26 

C. Management Information System (MIS) ......................................................................... 27 

IV. BUSINESS INVOLVEMENT ............................................................................................... 31 

A. Characteristics of Business Clients.................................................................................. 31 

B. Relationships between Business Clients and Business Interface..................................... 38 

 1. Recruiting Business Clients .......................................................................................... 40 

 2. Services to Businesses................................................................................................... 41

Summary............................................................................................................................... 45 

V. REFERRING ORGANIZATIONS........................................................................................ 47 

A. The Number of Referring Organizations Recruited by BI............................................... 47 

B. Populations Served........................................................................................................... 48 

C. Services Provided............................................................................................................. 50 

 1. Job-Related Services..................................................................................................... 50

 2. Skills Training Services................................................................................................. 51

 3. Additional Services ....................................................................................................... 51 

 4. Targeted Services.......................................................................................................... 52 

 5. Job Placement Services................................................................................................. 53 

 6. Support Services............................................................................................................ 54 

D. Relationships between ROs and BI.................................................................................. 55 

VI. PARTICIPANT CHARACTERISTICS AND SERVICES .................................................. 59 

A. Characteristics.................................................................................................................. 59 

B. Interviews with Youth Participants.................................................................................. 62 
 Evaluation of DSYODP–II Final Report 

ii McNeil Education, Training, and Research 

VII. EMPLOYMENT OUTCOMES ............................................................................................ 65 

A. Characteristics of Placements .......................................................................................... 65 

B. Industries, Businesses, Referring Organizations and Placements.................................... 67 

C. Hourly Wages and Hours per Week................................................................................. 70 

D. Placements at Retention versus Non-retention Wage Levels .......................................... 71 

E. Retention and Termination among Placements................................................................ 75 

VIII.FURTHER ANALYSIS - UI WAGE COMPARISON STUDY...........................................77 

 A. Hypotheses........................................................................................................................77 

 B. Data, Methods & Measures ..............................................................................................78 

 Data ................................................................................................................................78 

 Methods..............................................................................................................................81 

 Measures ............................................................................................................................82 

 C. Results .............................................................................................................................. 91 

 D. Discussion and Summary..................................................................................................93 

IX. CONCLUSION.......................................................................................................................95 

REFERENCES ............................................................................................................................102 

APPENDICES .............................................................................................................................106 

 Appendix A – Review of Prior Research ....................................................................... A-1 

 Appendix B – Study Questions........................................................................................B-1 

 Appendix C – Site Visit Protocols...................................................................................C-1 

 Appendix D – Correlation Coefficients .......................................................................... D-1 
 List of Tables 

McNeil Education, Training, and Research iii

LIST OF TABLES 

Table Page 

II.1. Program Logic Model for Demand-SideYouth Offender Demonstration Project ................. 9 

II.2. Preliminary Site Visit Schedule and Evaluator Assignments ............................................... 16 

II.3. Follow-up Site Visits............................................................................................................. 17 

II.4. Quarters of UI Wage Data Requested................................................................................... 19 

II.5. Core Themes from the Study Questions, Data Source, and Methods of Analysis........................ 20 

III.1. Goals for Each Position in a Center and Its Priorities ......................................................... 25 

IV.1. Total Number of Businesses Recruited by Business Interface and Total Businesses 

Recruited in Phase II............................................................................................................. 31 

IV.2. Total Recruited Businesses by Employee Size.................................................................... 32 

IV.3. Total Recruited Business by Annual Revenues................................................................... 32 

IV.4. Total Recruited Businesses by Industrial Classification in Rank Order.............................. 33 

IV.5. Recruitment Methods Used by Recruited Businesses Before BI Services.......................... 34 

IV.6. Types of Positions Businesses Were Seeking When Recruited by BI ............................... 34 

IV.7. Typical Unskilled Worker Retention Rates of Recruited Businesses Before Using BI 

Services................................................................................................................................. 35 

IV.8. Typical Skilled Worker Retention Rates of Recruited Businesses Before Using BI Services

............................................................................................................................................... 35 

IV.9. Reasons Recruited Business Give for Turnover Before Using BI Services ....................... 36 

IV.10. Recruited Businesses’ Offender Policies........................................................................... 36 

IV.11. Additional Services Requested by Recruited Businesses........................................37 

IV.12. Benefits Requested by Recruited Businesses....................................................38 

IV.13. Employers, Industries and Work Provided to Youth......................................................... 39 

V.1. Number of Referring Organizations Recruited by Site......................................................... 47 

V.2. Populations Served by Referring Organizations................................................................... 48 

V.3. Referring Organizations and Contact Position by Site ......................................................... 48 

V.4. Job-Related Services Provided by Referring Organizations................................................. 50 

V.5. Skills Training Services Provided by Referring Organizations............................................ 51 

V.6. Additional Services Provided by Referring Organizations................................................... 52 

V.7. Targeted Training Provided by Referring Organizations ..................................................... 52 

V.8. Job Placement Services Provided by Referring Organizations............................................. 53 

V.9. Supportive Services Provided by Referring Organizations .................................................. 54 

VI.1 Total Participants, Total Eligible Participants and Total Participants Placed ...................... 59 

VI.2. Businesses and Positions of Youth Interviewed (if Employed) .......................................... 62 

VII.1. Number of Placements by Quarter and Year by Site ......................................................... 65 

VII.2. Placements and Placement Rates by Status........................................................................ 66 

VII.3. Industry by Number of Employed, Placed and Terminated Youth.................................... 67 

VII.4. Most Popular Industries among Placements by Site .......................................................... 68 
 Evaluation of DSYODP–II Final Report 

iv McNeil Education, Training, and Research 

LIST OF TABLES (Continued) 

Table Page 

VII.5. Number of Placements by Businesses................................................................................ 68 

VII.6. Services Provided by ROs who Referred Placements........................................................ 69 

VII.7. Distribution of Placements among ROs ............................................................................. 69 

VII.8. Average Hours per Week and Average Hourly Wages by Status of Placed Participants.. 70 

VII.9. Average Wages for Placed Participants by Gender, Marital Status, and Ethnicity............ 71 

VII.10. Retention Wage Placement Standards.............................................................................. 72 

VII.11. Number of Employed Youth (Placements and Non-Placements) by Retention Wage 

Category, by Number of Children ................................................................................................ 73

VII.12. Number of Placements in Each Retention Wage Category by Number of Children ....... 74 

VII.13. Number of Participants Employed but Not at Retention Wage Levels............................ 75

VII.14. Number of Placements, Terminated Placements, and Weeks Employed Before 

Termination..........................................................................................76 

VIII.1. Quarters of UI Wage Data Requested and Number of Enrolled and Placed Youth from 

Phase I by Quarter and UI Wage State...........................................................89 

VIII.2. Characteristics of Youth Enrolled at Washington, DC and Miami Centers: Placements 

and Non-Placed Youth with Maryland and Florida UI Wage Records ......................94 

VIII.3. Quarters of UI Wage Data by State and Placement Status ....................................98 

VIII.4. Comparison of UI Wage Quarters Relative to Employment Start Date for Placed Youth 

.........................................................................................................98 

VIII.5. Comparison of Placement Wages for Two Consecutive Quarters Prior to Placement and 

After Placement (Retention Period) ..............................................................99 

VIII.6. Average UI Wages for Maryland from Enrollment Quarter Forward between Placements

 and Eligible Non-Placed Youth in DSYODP - Washington, DC Center ..................100 

VIII.7. OLS Regression Coefficients and T-Values for Measures of Average Wages and Job 

Tenure ..............................................................................................101 
List of Figures 

McNeil Education, Training, and Research v

LIST OF FIGURES 

Figure Page 

VI.1 Total Participants, Total Eligible Participants and Total Participants Placed ...................... 59 

VIII.1 Number of Youth by Number of Quarters Employed Based on UI Wage Records.........87 

Chapter I–Introduction and Background 

McNeil Education, Training, and Research 1

Chapter I 

 

INTRODUCTION AND BACKGROUND 

A. Introduction 

The Welfare-to-Work Partnership (The Partnership)1

, later known as Business Interface, Inc. 

(BI), was awarded funding by the Employment and Training Administration (ETA) of the U.S. 

Department of Labor/ Employment and Training Administration (DOL/ETA) in January 2003 

for a Demand-Side Youth Offender Demonstration Project (DSYODP). Under this grant, The 

Partnership served as an intermediary in connecting employers in four sites with young exoffenders and youth at-risk of court involvement between the ages of 18 and 25. The DSYODP 

initiative built on the experience of The Partnership in serving welfare recipients under a 

DOL/ETA grant for the Welfare-to-Work program. 

The DSYODP initiative built on the experience of DOL/ETA through three rounds of a similar 

project, the Youth Offender Demonstration Project (YODP), which began in 1999. The goals of 

the YODP were to assist youth at-risk of court or gang involvement, youth offenders, and gang 

members ages 14-24 to find long-term employment at wage levels that would prevent future 

dependency and would break the cycle of crime and juvenile delinquency. Its objectives were to 

identify effective community strategies that support youth in becoming work-ready and capable 

of attaining and keeping employment that will provide a future of economic stability and support 

civic engagement. Grantees were encouraged to build upon the services and partnerships already 

supporting target youth and fill in the gaps in service delivery and coordination. 

McNeil Education, Training, and Research (McNeil ETR) has worked with DOL/ETA to 

conduct evaluations of both YODP and DSYODP Phases I and II. Phase II of DSYODP began in 

July 2005 when it received a Task Order contract from DOL/ETA to conduct a process 

evaluation of the project The first phase evaluation of the demonstration had not been completed, 

and The Partnership contracted with McNeil ETR in September 2005 to complete the evaluation 

and prepare a Final Report that analyzed the project’s outcomes and implications for future 

efforts. McNeil ETR completed Phase I and then proceeded with the work of the Phase II 

evaluation. 

This document constitutes the Final Report for Phase II. The next section provides a brief 

background of the DSYODP intermediary model and evaluation. Chapter II discusses the scope 

and nature of the evaluation approach as reflected in this Final Report. Chapter III covers a basic 

description of the structure of DSYODP, particularly in the four sites. Chapters IV-VII provide 

extensive information on business involvement in DSYODP, referring organizations that worked 

with youth, characteristics of the youth, and their employment outcomes. Chapter VIII presents 

further analysis of the project outcomes using a combination of data collected from the DSYODP 

sites and Unemployment Insurance (UI) wage data for youth enrolled in the project during Phase 

 

1

 At the time of the award, the recipient organization was known as The Welfare-to-Work Partnership, though at the 

time of this report, the organization has subsequently become known as Business Interface, Inc. 
 Evaluation of DSYODP–II Final Report 

2 McNeil Education, Training, and Research 

I. Chapter IX concludes the report with a discussion of the implications of the analyses 

conducted for the evaluation during both phases of the project. 

There are several key terms repeated throughout this document. These terms are defined in the 

box below for your reference. 

KEY TERMS

Business – any entity recruited by the Centers at the four sites for which a Business Needs 

Analysis Form was completed and a record created in the database. Generally these are for-profit 

businesses, but include non-profit and public organizations as well. Some portion of businesses 

recruited to the program subsequently hire DSYODP participants, but not all businesses in the 

database may have filled positions by the end of the grant period. 

Intermediary – an entity that links businesses with individuals seeking employment. The scope 

of the activities entailed in that “link” varies greatly across intermediaries, and can include: job 

search, work experience, education, training, subsidized employment, placement, retention 

support, and advancement support. Intermediaries can be non-profit organizations, for-profit, 

educational institutions, and public or quasi-public organizations. 

Referring Organization – an entity performing one or more functions under the DSYODP, 

including: (a) making referrals of qualified individuals for placement at business clients of the 

Centers; (b) determining the eligibility of participants for the DSYODP; and (c) providing a 

range of services that facilitate participants to be successful, from pre-employment preparation to 

specific skills training to additional services, as well as targeted training in specific fields, job 

placement services, and supportive services that help participants be successful after placement. 

Referring organizations can include Once-Stop Career Centers, MOU partners such as 

Community-Based Organizations (CBOs), Faith-Based Organizations (FBOs), other service 

providers, Criminal Justice Agencies, as well as employers referring individuals to DSYODP. 

Participant – an individual for whom a Referral Form has been completed and a record entered 

in the DSYODP database. Participants must be between the ages of 18-25 at the time of 

enrollment and be either an ex-offender or at-risk of court involvement. Participants’ 

circumstances are reviewed to determine if they meet the eligibility requirements for DSYODP, 

and if so, an Eligibility Form is completed and an eligibility record created, and an Intake 

Interview Form would be completed at the next stage in the process. If an eligible participant 

gains employment through DSYODP, data is entered on an Employment Information Form and 

an employment record created; at this time, determination is made as to whether the employment 

meets retention wage level standards, in which case a “placement” is recorded as well. 

Placement – an eligible participant in the DSYODP project who achieves employment at a 

“retention wage” while enrolled in the project. Retention wages are a minimum threshold wage 

calculated based on the cost of living for each of the cities in which the project sites were located 

and the number of dependents. Youth must be employed full-time to be considered placements. 

Retention wages are discussed in more detail in Chapter VII. 
Chapter I–Introduction and Background 

McNeil Education, Training, and Research 3

B. Background 

This evaluation of Phase II of DSYODP builds on the knowledge gained through the first phase 

of the demonstration. As stated in the Request for Quotations (RFQ), the purpose of the 

evaluation was to determine: 

(1) the efficacy of the business intermediary model implemented by the 

grantee; and 

(2) the impact of the services delivered on the employment, earnings and 

retention of youth ex-offenders and youth at-risk of court or gang 

involvement. 

Further, DOL/ETA expected that the evaluation would also “determine lessons from the 

implementation and operation of the DSYODP-II project which can be shared with other 

communities wishing to replicate the business intermediary approach to serve ex-offenders and 

at-risk youth.” 

For both phases, BI helped businesses to “identify and capitalize on hiring, retention, and career 

advancement strategies outside the corporate mainstream.” A demand-side approach recognizes 

the importance of initiatives that fit the hiring needs of employers, while also assisting 

individuals in gaining employment and greater self-sufficiency. 

The basic strategy of the DSYODP model is to establish intermediary organizations, known as 

Business Resource Centers (Centers), whose primary purpose is to connect employers with 

youth. The “demand-side” component of the model comes from the demand by employers for 

persons who are qualified to fill positions they offer; thus, the employers are the primary 

customers of the Centers. 

In addition, the Centers act as intermediaries with the organizations that can refer youth who 

have been screened and prepared for making applications to the employers with positions. The 

Centers also offer retention services for individuals who are hired, including assistance in 

obtaining appropriate work clothing, child care referrals, and transportation. This package of 

services, at no cost to the businesses, is expected to address employers’ reticence about hiring 

young offenders or other youth “at-risk” of court involvement. Factors determining “at-risk” 

status are: 1) poverty, 2) drug and alcohol abuse, 3) gang involvement, and/or 4) a lack of 

supportive and/or stable family environment (Technical Assistance Manual, Volume 2, Business 

Interface, Inc.). 

In terms of the participants, BI’s objective was to identify positions that pay “retention wages” 

that took into account the family size of the participant and the cost of living in the specific city 

of employment. By providing retention and career advancement services to the participants after 

hiring, BI hoped to encourage retention in employment for a period of at least six months. 

During Phase I of DSYODP, the Centers were located in Chicago, Miami, New York, and 

Washington, D.C. The Phase I contract concluded in August 2005. BI was awarded another 

round of funding in July 2005 to continue the project. During the transition into the second phase 
 Evaluation of DSYODP–II Final Report 

4 McNeil Education, Training, and Research 

of the project, BI underwent tremendous changes in terms of staffing and corporate structure. 

Initial funds from Phase I were depleted in spring 2005 and new funds had not been dispersed by 

DOL/ETA before incumbent staff began leaving the organization. BI executive staff members 

were restructuring the organization and hiring new personnel. BI also made changes to the 

Technical Assistance Manual (TAM) and implemented a Quality and Compliance division to 

monitor data quality and accuracy in reporting. These changes are covered in greater detail in 

Chapter III of this report. The next chapter presents the evaluation approach, including the 

evaluation design and methods of data collection and analysis. 
Chapter II–Evaluation Approach 

McNeil Education, Training, and Research 5

Chapter II 

EVALUATION APPROACH 

In this chapter the overall approach to evaluation of the Demand-Side Youth Offender 

Demonstration Project (DSYODP), the evaluation design approved by DOL/ETA and the 

methods of data collection and analysis used to obtain and organize the information needed by 

DOL/ETA are presented. 

Prior to presenting a comprehensive description of the components of the Phase II evaluation 

approach and design, a brief overview of the process evaluation that was conducted for Phase I 

of DSYODP is provided to contextualize the evaluation approach developed for Phase II. 

A. Overview of Process Evaluation of Phase I 

An overview of the process evaluation of Phase I of DSYODP provides a useful foundation for 

understanding the scope of the evaluation for Phase II, which has been expanded in critical ways. 

Though the Phase I evaluation was actually conducted in two stages by The Urban Institute and 

McNeil ETR, the Interim and Final Reports for Phase I provide a comprehensive view of 

DSYODP as implemented and up through the end of the award, May 2005. 

The evaluation conducted by The Urban Institute focused on the initial implementation of 

DSYODP at the four sites. Consequently, the process evaluation looked at the context and local 

settings of the four sites, staffing at the Centers, and the MIS designed for DSYODP. Site visits 

were conducted to the four sites, enabling qualitative data collection through interviews with 

staff at the Centers, businesses, and referring organizations. It was during the site visits that the 

issue of staff turnover surfaced, which continued as a topic of interest. 

The use of the MIS and feedback from the staff on the MIS are well-documented in the Interim 

Report. The Urban Institute Interim Report also provides beneficial insights into the initial 

contacts made with potential business clients by the BRCs. The Interim Report offers 

information about the services available through the Centers to the businesses and to the 

participants, which of course are key components of the intermediary model. In addition, the 

Interim Report presents information about a number of referring organizations, obtained through 

interviews during the site visits; this provides a sense of the diversity among the referring 

organizations in terms of the types of services they provide and the youth with whom they work. 

Finally, The Urban Institute included data in the Interim Report on three key elements of the 

intermediary model – businesses, participants, and referring organizations. Most data were 

reported through April of 2004; because the Centers continued operations into 2005, the data 

necessarily reflected only part of the award period. 

The second stage of the evaluation of Phase I essentially picked up at the point where operations 

had ceased and thus a full database was available for analysis. The Welfare-to-work Partnership 
 Evaluation of DSYODP–II Final Report 

6 McNeil Education, Training, and Research 

(The Partnership) contracted with McNeil ETR to conduct data collection and analysis to 

produce a Final Report on the first phase of DSYODP. 

The starting point for this second stage of the process evaluation was the development of study 

questions for Phase I. McNeil ETR reviewed the study questions developed by The Partnership 

and The Urban Institute, and produced an evaluation design that addressed the study questions 

based on the data available after the completion of Phase I. The principal focus of this evaluation 

was the analysis of the database, with its extensive information on businesses, referring 

organizations, and participants. A secondary focus was collecting qualitative information from 

businesses regarding their experience with DSYODP; this information was collected via 

telephone calls, since site visits were not as feasible or cost-effective within the timeframe. Some 

additional information was collected from BI regarding staffing and the continuation of service. 

The Final Report for Phase I contains comprehensive descriptive data on the principal 

components of the model. The types of data in the Phase I Final Report established the minimum 

scope of evaluative information expected to be produced for Phase II. This includes information 

on business involvement, referring organizations, participants, and employment outcomes, which 

is covered in more detail in subsequent chapters. 

Because BI continued to collect extensive data on businesses, referring organizations, and 

participants in Phase II, the range of information available for analysis was considerable. A 

challenge for Phase II was to keep the study’s focus on the information that would be relevant 

for addressing the study questions, while also keeping in mind that other data may provide useful 

insights into the intermediary model, as implemented across the four sites. 

The remainder of this chapter provides an extensive description of the components of the Phase 

II evaluation conducted during the period of the contract from July 2005 through June 2007. The 

main components of the evaluation design have been divided into two sections: 

• Section B covers the evaluation design as outlined in the RFQ, and 

• Section C covers the evaluation tasks carried out after the design was approved, 

specifically data collection and analysis. 

B. Evaluation Design of Phase II 

The evaluation of the DSYODP includes two parts: the process evaluation of the Centers and an 

outcomes study of the youth’s success in obtaining employment, keeping employment once 

hired, and earning wages that provide a stable financial status. 

The process evaluation focused on the activities carried out by the Centers as BI continued the 

work begun in Phase I. Site visits enabled the evaluation team to document the operations of the 

Centers, the nature of their intermediary relationships (e.g., working with businesses, referring 

and other local organizations, and youth seeking employment), and the services provided by the 

Centers’ staff. For the three existing sites, the process evaluation documents the history of their 
Chapter II–Evaluation Approach 

McNeil Education, Training, and Research 7

operations and the changes the sites have made between Phase I and Phase II. For Los Angeles, 

the process evaluation documents its start-up activities and progress toward full implementation. 

The outcomes study focused primarily on the results of the matching process of meeting 

businesses’ needs to fill vacancies through placing youth offenders and youth at-risk of court 

involvement in those positions. Thus, there are outcomes for both businesses and for youth that 

can be measured through the collection and analysis of data on placements, wages, and retention. 

The evaluation design addresses the four main components identified in the Request for 

Quotations (RFQ): 

1. A program logic model, 

2. Evaluation (i.e., study) questions, 

3. Methodologies to measure outcomes of the demonstration project, and 

4. Data requirements necessary for the evaluation. 

The study questions are the heart of the evaluation design. They follow from the goals of the 

evaluation and establish the specification of the data collection methods through identification of 

the sources of the data. In turn, data collection must be sufficient and appropriate to address each 

of the study questions through data analysis. Appendix B contains a full list of the study 

questions. In addition, information on the data requirements for Phase II are provided, focusing 

on the data elements needed in the MIS. 

1. Program Logic Model 

Logic models have steadily gained broader acceptance and use within the field of program 

evaluation in recent years and McNeil ETR has found logic models to be valuable tools in the 

design and implementation of evaluations. As stated in the RFQ, the logic model will “provide 

the framework for evaluating the project as well as communicating about the project to The 

Partnership and the Federal Project Officer (FPO).” 

As also recognized in the RFQ, a logic model is not necessarily a static product, but rather an 

evolving picture that attempts to represent stakeholders’ understanding of how a particular 

program works and to what end. Logic models surface diverse perspectives of program outcomes 

and the activities that could be designed to achieve the desired results, and hopefully reach 

common agreement on the roadmap for describing how the program will go from design to 

planning to implementation and to successful operation. As the W.K. Kellogg Foundation (2001) 

notes: 

Evaluation – especially program logic model approaches – is a learning and 

management tool that can be used throughout a program’s life – no matter 

what your stake in the program. Using evaluation and the logic model results 

in effective programming and offers greater learning opportunities, better 

documentation of outcomes, and shared knowledge about what works and why. 

The logic model is a beneficial evaluation tool that facilitates effective 

program planning, implementation, and evaluation. 
 Evaluation of DSYODP–II Final Report 

8 McNeil Education, Training, and Research 

In Table II.1 the logic model for DSYODP is presented, representing the principal types of 

inputs, activities, outputs, and outcomes that compose the demonstration in Phase II. 

Several assumptions have been made in developing this logic model: 

a. Short- and intermediate-term outcomes have been identified for participants and 

businesses, but not other entities involved in DSYODP-II. 

b. Activities focus on the work done by staff at the Centers with participants, businesses, 

referring and other organizations. Staff at the Centers is viewed both as “inputs” (i.e., 

resources) in this logic model and also as intervening variables that can affect the 

outcomes for businesses and participants (e.g., through recruitment of businesses, 

services provided to businesses, preparation of youth for obtaining employment, and 

supporting retention). 

c. Specific activities are not tied to specific outcomes; many activities need to be 

undertaken in concert in order for outcomes for participants and businesses to be 

achieved. 

d. In general, the Centers are responsible for inputs, activities, and outputs, but 

outcomes are largely beyond their control because the outcomes, to a considerable 

extent, occur after the completion of the DSYODP grant. For example, the RS works 

with participants after they are placed, but given the number of placements per 

Center, that person will not have much time per participant to help ensure that 

“participants placed in jobs have skills, attitudes, and behaviors for success in 

employment (retention, advancement)” beyond the initial placement and follow-up 

period of six months. 

e. There are numerous factors that affect the accomplishment of outcomes that are 

outside the sphere of influence of the Centers and are not reflected in the logic model, 

such as community characteristics, economic and employment conditions, and 

support for and investment in youth, as well as characteristics of the participants 

themselves, such as prior work experience, education and literacy levels, peer 

relationships, adult support and guidance, living arrangements, and criminal records. 
Chapter II–Evaluation Approach 

McNeil Education, Training, and Research 9

Table II.1. Program Logic Model for Demand-Side 

Youth Offender Demonstration Project 

Inputs Activities Outputs Short-term 

Outcomes 

Intermediateterm Outcomes

• Funding 

• Existing 

organization 

• Potential 

collaborating 

partners 

• Organizational 

or 

interpersonal 

networks 

• Staff and 

volunteers 

• Facilities, 

equipment, 

and supplies 

Working with Businesses

• Assessment of 

demand for workers 

by local businesses 

• Identification and 

recruitment of 

businesses 

• Identification of 

specific positions 

available at businesses

• Referral of 

participants to 

businesses 

Working with Referring 

Organizations 

• Identification of 

referral sources for 

youth participants

• Identification of 

community resources 

for other support 

services 

• Center develops 

working relationships 

with area One-Stop 

Career Centers

Working with

Participants

• Development of tools 

to assess work 

readiness of youth 

referred to the Center 

• Design and 

implementation of 

pre-employment 

services for 

participants 

• Referral of 

participants to 

businesses with 

positions available 

• Design and 

implementation of 

retention services for 

participants placed 

with businesses 

• Number of 

positions 

filled by 

businesses 

• Number of 

youth 

completing 

work 

preparedness 

training or 

receiving 

support 

services to 

prepare for 

employment 

• Number and 

percent of 

youth 

achieving 

employment 

• Businesses hire 

referrals from 

the target 

population 

during the 

period of the 

demonstration. 

• Businesses are 

satisfied with 

services 

provided by the 

Center. 

• Participants

persist in 

employment for 

the duration of 

the six-month 

follow-up 

period. 

• Participant

youth are 

placed at 

retention wage 

levels. 

• Businesses

continue to hire 

youth in the 

target 

population. 

(Businesses 

who hired 

referrals during 

Phase I 

continue to hire 

youth; new 

businesses 

recruited during 

Phase II hire 

youth 

throughout the 

period of the 

demonstration.) 

• Participants 

attain wage 

increases over 

time. 

• Participants

attain job 

advancements. 

• Participants

persist in 

employment 

beyond the 

initial sixmonth followup period. 

• Participants

attain 

educational and 

vocational 

outcomes while 

employed. 

• Participants

have no 

involvement 

with the justice 

system. 

There are long-term outcomes that may be of considerable interest to BI and DOL/ETA that are 

not incorporated into the logic model. Because the focus of this logic model is on individual 
 Evaluation of DSYODP–II Final Report 

10 McNeil Education, Training, and Research 

participants and businesses, the model does not address outcomes for the larger community. 

However, as the W.K. Kellogg Foundation and others have noted, both logic models and 

outcomes can operate on more than one level, such as system- and community-levels. In the case 

of the DSYODP, the achievement of individual outcomes for participants and businesses may 

contribute to larger outcomes for the communities in which they reside. For example, an 

intermediate-term outcome might be that a group of businesses help to form a network of other 

employers to hire this target population. This could foster opportunities for more young adults 

over the long-term as well as strengthen local economic development. 

2. Primary and Study Questions 

In this segment the primary research questions as identified by DOL/ETA in its Request for 

Quotation (RFQ) are presented along with the hypotheses and study questions to provide a 

comprehensive outline for the focus and scope of the DSYODP Phase II evaluation. 

Primary Research Questions 

As stated in the RFQ, the primary research questions of this evaluation are: 

• How does the business intermediary approach identify and recruit employers 

and broker job placements for youth ex-offenders and youth at-risk of court or 

gang involvement? How well does BI identify and recruit employers who are 

willing to hire this target population? Does the use of the intermediary satisfy 

employers’ needs? What criteria does BI utilize to hire BRC staff? What is the 

turnover rate of such staff? What are the performance requirements of such 

staff? Do they receive placement bonuses? If so, what are the results? 

• How does BI connect youth ex-offenders and at-risk youth to the services of 

the demonstration project and to One-Stop Career Center services and other 

services available in the community? How well does the intermediary connect 

youth ex-offenders and at-risk youth to services? Does the use of BI’s services 

reduce unemployment spells and otherwise improve employment outcomes 

for youth ex-offenders and at-risk youth? Does BI help youth avoid criminal 

activity and retain employment? 

These two sets of research questions structure the evaluation around the two principal target 

populations – businesses and youth participants – and reinforce the interconnectedness of the 

target populations, where BI acts as an intermediary between the two. In addition, the research 

questions introduce the other two key components of the process: staff at the Centers and 

community organizations that can refer youth and provide valuable services. The study questions 

in the next segment address each of the main components. 
Chapter II–Evaluation Approach 

McNeil Education, Training, and Research 11

Study questions 

Study questions follow directly from the goals of the evaluation stated in the RFQ, which are to 

“determine: (1) the efficacy of the business intermediary model implemented by the grantee, BI; 

and (2) the impact of the services delivered on the employment, earnings, and retention of youth 

ex-offenders and youth at-risk of court or gang involvement.” For purposes of this evaluation, 

the efficacy of the model is addressed through the study questions concerning the interactions of 

the Centers with the key stakeholders (e.g., businesses, referring and other local organizations, 

and youth). The impact of services delivered is reflected in the study questions related to the 

outcomes achieved by placement of youth at employers paying wages expected to support 

retention in employment. 

A series of study questions organized into six main categories reflected the interests of 

DOL/ETA and BI at this stage in the demonstration, addressing the main goals of the evaluation 

to assess efficacy of the business intermediary model and the impact of the services delivered on 

the employment, earnings, and retention of youth. 

The six categories of study questions are: 

• Development of a Network of Employers 

• Employer Outcomes 

• Project Organization and Staffing [Business Resource Centers] 

• Service Delivery to Youth 

• Participant Outcomes 

• Referring Organizations 

Within each category are a series of more specific study questions, as presented in Appendix A. 

These study questions reflect discussions with DOL/ETA in the evaluation design phase that 

determined the feasibility of collecting the data needed to address some study questions 

originally proposed by our evaluation team as well as by DOL/ETA during the initial kick-off 

meeting. 

3. Methodologies to Measure Outcomes 

One of the principal components of the evaluation concerns the measurement of outcomes of the 

demonstration. As noted in the RFQ, “This could be done with a true experiment design, using 

random selection and random assignment, or could use a quasi-experimental design with a welldesigned comparison group, or other methodologies.” The evaluation team gave considerable 

thought to the possible approaches for measuring outcomes and certainly recognized that other 

methodologies can be valuable in determining the outcomes of a particular program, or 

“treatment,” when compared to outcomes under a condition of non-treatment. For example, 

random assignment is being used in the Impact Study of Round Three of the Youth Offender 

Demonstration. 

In considering methodologies for DSYODP, there were a number of factors to take into account 

in determining which methods may be feasible. Assuming that participants from Phase I of 
 Evaluation of DSYODP–II Final Report 

12 McNeil Education, Training, and Research 

DSYODP are part of the evaluation of outcomes, it would not be feasible to retrospectively 

create a control group or other comparison group. Alternatively, for new participants in Phase II, 

using random assignment or creating control groups might be feasible in theory but very 

challenging in practice. In effect, this would require identifying a pool of youth in each city that 

are comparable to those youth participating in DSYODP-II, but for whatever reason have not 

been or will not be referred to a Center for assistance in obtaining employment. Then comparable 

data on these youth would need to be collected over time to generate information on outcomes as 

well as any services they may have received that were comparable to services received by 

participants in DSYODP-II. Simply tracking these youth over the period of the evaluation would 

be a challenging and costly endeavor in itself. 

Four additional alternatives were considered that could have been used to place the outcomes for 

DSYODP-II in a relevant context by comparisons to other similar groups. These alternatives 

reflect comparison groups of increasing size and scope. Three alternatives were composed of 

datasets from the Youth Offender Demonstration Project: (1) approximately 150-200 youth in 

the two Employment Bonus programs from Phase III who had received workforce preparation 

and retention support services; (2) approximately 200 participants in Round Three for which data 

was collected on characteristics, comprehensive information on services planned and received, 

and employment and other outcomes; and (3) substantial data has been collected on three cohorts 

of participants from the three rounds of the Youth Offender Demonstration Project, through the 

Outcomes Study of Phase Three of the evaluation. A fourth alternative was state employment 

data for the four cities in DSYODP-II, which could have been analyzed by characteristics such as 

age, gender, and ethnicity. 

For each of the four alternatives, there would be target populations expected to share, to varying 

degrees, the characteristics of those participants in DSYODP who achieve placement at retention 

wages. However, there are also various factors that lessen the comparability of the populations, 

such as the differing approaches of the “treatment” programs, the contexts in which the programs 

operated, the characteristics of the populations, etc. Consequently, an outcomes study using a 

comparison group of participants from Phase I of DSYODP was proposed. The comparison 

group consisted of those participants referred to one of the four Centers who were determined 

eligible for DSYODP but were not placed in employment at retention wages through the efforts 

of the Centers. 

Our analysis of Phase I data found that participants in the two groups were similar in terms of 

age, gender, and marital status but differed with respect to ethnicity, educational attainment, 

offender status and number of children. UI wage data on employment, persistence, and earnings 

history were used as the measures of long-term outcomes for participants who were placed and 

for those who were not. For the independent variables, information was collected on the 

participants’ demographic backgrounds, education, criminal history, and other factors that may 

affect their progress in obtaining and keeping employment, and increasing wages. 

With any comparison group, questions can be raised about the degree to which the group is truly 

comparable to the group receiving the “treatment” offered through the program being evaluated. 

In many respects, the proposed comparison group seems to offer a particularly appealing means 

to addressing those questions. The comparison group in effect should have the same type of 
Chapter II–Evaluation Approach 

McNeil Education, Training, and Research 13

participants as those who actually received the treatment, if it is assumed the principal treatment 

was the attainment of employment through DSYODP. 

4. Data Requirements 

In this section three principal components that address data requirements for the evaluation as 

specified in the RFQ are discussed: Quantitative data elements in the MIS, qualitative data 

collected primarily during site visits, and administrative data to be obtained from state agencies. 

Quantitative Data Elements (MIS) 

The data elements developed for the evaluation of Phase II of DSYODP were based primarily on 

two factors: The existing database created by The Partnership for Phase I and the study questions 

that were eventually approved for Phase II. However, during the evaluation design phase, 

DOL/ETA and the evaluation team determined that the potential value of adding data elements to 

the MIS was outweighed by the additional cost and time delays that would have been incurred by 

asking BI to make such modifications. 

Qualitative Data 

Qualitative data complements the quantitative data available through the MIS maintained by BI 

to help address a number of study questions for the Phase II evaluation. Data collection 

instruments were developed to obtain data primarily through the site visits to the four Centers 

(see Appendix C). 

Administrative Data 

As specified in the RFQ, ETA worked with the evaluation team to determine the “data 

specifications for administrative data to be solicited from BI or appropriate state agency 

administrative files... [which] will likely include UI wage records...” While other administrative 

data sets may have data on DSYODP participants in Phase I, the UI wage records provide the 

most complete and the most cost-effective data set that could be obtained and used in analysis for 

evaluation purposes. 

Evaluators placed the principal focus on obtaining UI wage records from State Employment 

Security Agencies (SESAs) for those participants who obtained employment through DSYODP, 

and for a comparison group of eligible DSYODP participants not placed through BI. A primary 

advantage of UI wage records is that wages earned by participants can be obtained that 

potentially cover periods of employment before, during, and after their period of participation in 

DSYODP. Given that BI staff members monitored employment outcomes for a period of six 

months after an individual’s employment start date, UI wage data could cover periods beyond 

the initial six-month follow-up period for Phase I participants. 

There are some limitations for the use of UI wage data. For example, SESAs do not typically 

maintain data on the number of weeks worked within a quarter or the number of hours worked 

per week; thus, an aggregate amount for earnings in a quarter masks other information such as 
 Evaluation of DSYODP–II Final Report 

14 McNeil Education, Training, and Research 

whether the individual worked full-time and for the full quarter. In addition, some participants 

may have accepted initial employment outside the state in which the Center operated (e.g., 

Virginia for participants from the Washington, D.C. Center) or they may have moved to another 

state during or after the period of the demonstration (whether in the group of participants placed 

through DSYODP or in the comparison group), and their earnings may not be fully counted. 

The process of obtaining UI wage data, including the timeline for processing and analyzing the 

data, is discussed in considerable detail in the next section. 

C. Data Collection and Analysis 

This section is divided into two parts: Data collection and analysis. Both sections distinguish 

between the evaluation process for administrative and field-based data. 

1. Data Collection 

Data collection encompassed three principal elements: 

a. Management Information System (MIS) 

b. Data Gathered During Site Visits 

c. Administrative Data 

The specification of data came directly from the study questions. A comprehensive table later in 

this section (Table II.5) relates the data sources and methods of analysis to each study question 

topic. In addition, the approach for the use of the MIS maintained by BI for the acquisition of 

data for the quantitative analysis is presented along with a description of the approach to the site 

visits and assignments of evaluators for the visits. 

To answer the study questions for both the process and outcomes evaluation, evaluators drew 

upon an array of data sources at the four project sites, both quantitative and qualitative, that 

address the questions. These data collection methods included: 

• Semi-structured interviews with program planners, staff at the Centers, 

representatives of referring organizations, businesses, youth and other stakeholders 

during visits to project sites; 

• Acquisition of data from project MIS and administrative records systems (e.g., State 

Employment Security Agencies’ UI wage records); 

• Review of program documentation, including individual project progress reports from 

Phases I and II, and other relevant research reports and findings; 

• Observations of project planning meetings and program operations; and 
Chapter II–Evaluation Approach 

McNeil Education, Training, and Research 15

• System observations (e.g., identifying the interconnected systems that support project 

development and implementation such as local Once-Stop Career Centers, 

community and faith-based organizations, and other local resources that broker prescreening and quality job training for area businesses). 

To ensure validity and reliability of the data being collected, evaluators used standard 

triangulation techniques and multiple sources of information, as practical. In preparation for data 

analysis, evaluators, for example, compared data from different sources and attempted to identify 

and reconcile discrepancies. For example, to address the question “What has been the experience 

of the employers with the persistence of BI participants?” data were used from the MIS to 

provide quantitative data on retention, complemented by interviews with businesses that hired 

DSYODP participants. 

Quantitative measures that represent the outputs and outcomes from DSYODP-II and from the 

UI wage records data were specified. Typically, measures indicate “number and percent” of 

participants, businesses, and referring organizations who have achieved a particular result (e.g., 

businesses recruited, participants placed). Data from the MIS was obtained at two points in the 

evaluation to conduct analyses and prepare the Interim and Final Reports. 

MIS 

Quantitative data on Phase II operations came mostly from the MIS. The MIS was developed 

prior to Phase II of DSYODP as a comprehensive relational database accessible to staff at each 

of the Centers. An IT Consultant at BI headquarters managed the system and various levels of 

accessibility were granted to BI executives and staff. BI staff members used the MIS, along with 

a system of paper forms, to track the activities and correspondence with business clients, 

referring organizations and participants. 

The MIS was not modified for Phase II although the paper forms used by BI staff to record data 

and the accompanying Technical Assistance Manual were modified to reflect an increased 

emphasis on data quality assurance. McNeil researchers received MIS data used for this report in 

the form of an SQL database. 

Data Gathered During Site Visits 

McNeil ETR conducted a preliminary site visit and two follow up site visits to each BI location. 

The three visits assured that each site was visited at least every six months of the evaluation 

contract as required by the RFQ. 

Since the staffs of three locations were reconstituted for Phase II, it took several months before a 

substantial number of placements were made. Los Angeles, on the other hand, was initiating the 

demand-side intermediary model for the first time. The preliminary visits provided evaluators 

with the opportunity to gather information on the transition from Phase I into Phase II and on the 

launching of Los Angeles as a new site. Two more visits, with opportunities to study business 

clients’ and youth clients’ experience, followed when there had been time to expand the demandside model in each location. 
 Evaluation of DSYODP–II Final Report 

16 McNeil Education, Training, and Research 

Preliminary site visits 

The preliminary site visits occurred between April and June 2006, based on the availability of 

local staff and the time it took for each site to become operational. A knowledgeable McNeil 

ETR evaluator, with field experience from the Youth Offender Demonstration Project 

Evaluation, made each visit. Each visit took approximately one day. The goals of the preliminary 

visit were to assist evaluators in developing an in-depth understanding of the status and context 

of each site and to help them design appropriate data collection protocols for the visits to follow. 

Prior to visiting each Center, McNeil ETR worked with the BI Grant and Evaluation Manager to 

schedule the visit and allow for adequate staff preparation at each of the Centers. Letters were 

emailed to each Center Manager to explain the agenda and purpose of the visit, identify staff for 

interviews, and list documents and information to be collected by the researcher during the visit. 

During the preliminary visits, McNeil ETR evaluators gathered qualitative data through 

interviews with all BI staff, observations of BI staff operations, and documents. The documents 

included quarterly reports sent by BI to DOL/ETA, job descriptions for BI employees, MIS data 

entry screens, business and referring organization contact lists, and reports on local employment 

conditions at each site. 

Each preliminary site visit concentrated on the core activities of BI staff at the center, their 

interactions with business clients, referring organizations, service providers and youth. 

Evaluators probed BI staff during the interviews for information on their goals, strategies, 

challenges and successes with implementation/ resumption of the project. Evaluators observed 

the office layout and work environment at each center, interactions between staff and clients, and 

data entry and staff use of the MIS. After each visit, each researcher produced a preliminary site 

visit report using the preliminary site visit field guide. Appendix C contains the field guide for 

the preliminary visits. 

The preliminary visit schedule and evaluator assignments are reported in Table II.2. 

Table II.2. Preliminary Site Visit Schedule and Evaluator 

Assignments 

Location Evaluator Date of Visit 

Chicago C. Robinson May 22, 2006 

Los Angeles S. Jenks May 5, 2006 

New York L. MacGillivray April 19, 2006 

Washington, D.C. R. Stone-El June 15, 2006 

Follow-up Site Visits 

The two follow-up visits to the BI project sites occurred in late fall 2006 and spring 2007. 

Timing of the second visit depended on the implementation status of the projects as well as the 

availability of local staff; the third visits needed to be completed by March 2007 to produce site 

visit reports in preparation for the Final Report on the demonstration. A key objective of these 
Chapter II–Evaluation Approach 

McNeil Education, Training, and Research 17

visits was to interview business clients, referring organizations’ staff and youth participants 

within the local BI network. 

Table II.3 indicates the dates of the second and third visits. 

Table II.3. Follow-up Site Visits 

DSYODP Location Evaluator(s) Second Visit Third Visit 

Chicago Corre Robinson Nov. 13-15, 2006 March 13-15, 2007 

Los Angeles Stephen Jenks Nov. 13-15, 2006 March 12-14, 2007 

New York Lois MacGillivray Nov. 7-9, 2006 March 13, 2007 

Washington, D.C. Robert Stone-El Nov. 7-9, 2006 March 27-29, 2007 

The follow-up visits focused on aspects of the study questions that could not be understood 

through quantitative data sources, such as the MIS and UI wage records: 

1. Identifying the strategies that BI staff used to recruit business clients, referring 

organizations and youth; 

2. Identifying the strategies BI used to broker the relationship between youth and 

business clients, including the degree of workforce preparation BI provided and the 

time and extent of other services it delivered to youth; 

3. Gaining the perspective of employers on their willingness to hire youth offenders, pay 

a retention wage, or alter hiring practices to hire project youth; 

4. Gaining the perspective of employers about their experience in hiring youth clients 

and their satisfaction with the young workers’ performance; 

5. Examining the staffing of the local Center: performance expectations, turnover rate, 

reasons given for staff turnover, and reasons for staff persistence; 

6. Assessing the degree of outreach by the local intermediary organization to local OneStop Career Centers and to other service agencies with the mission to support youth 

offenders and youth at-risk of court involvement; and, 

7. Reviewing MIS data on youth characteristics, services received, employment 

placements and wages earned, and clarifying ambiguities in the database. 

Appendix C includes field guides for eight staff members interviewed during the second visit: 

Business Resource Manager, BRC, HRC, and RS at the Centers; Division Manager at BI; 

representatives of businesses who hired DSYODP participants; representatives of organizations 

who referred participants to DSYODP and provided services to participants; and the participants 

placed in retention wage positions. 

The third site visit included interviews with businesses, participants, and referring organization 

contacts. Evaluators also conducted a focus group interview with staffs at three of the four sites 
 Evaluation of DSYODP–II Final Report 

18 McNeil Education, Training, and Research 

to capture each group’s overall perceptions about the project’s implementation, infrastructure 

and operations, data collection tools, project challenges, successes, and lessons learned. The site 

visit protocols for the third site visits can also be found in Appendix C. 

In addition to the visits to the four Centers, the evaluation team conducted two site visits with 

corporate officials of BI. Evaluators interviewed the Executive Staff and IT Consultant at BI 

headquarters in Baltimore, MD on December 20, 2006, and followed up with a second interview 

with the CEO at the Washington, D.C. Center on March 27, 2007. The protocol for the first visit 

of these visits is included in Appendix C. 

Administrative Data 

Earlier in this chapter the basic evaluation approach proposed with respect to the specification of 

administrative data on the outcomes of participants in the DSYODP was discussed. The 

collection of these data provides information on the extent to which participation in DSYODP 

had an effect on earnings and other employment outcomes for participants. In this segment, a 

summary of the steps in the process undertaken for obtaining UI wage data is presented along 

with the approach for obtaining, processing, and analyzing the data. 

Obtaining UI wage data requires a concentrated effort over many months to secure agreements 

with State Employment Security Agencies (SESAs). The steps in the process followed to 

determine the willingness of SESAs to participate and provide data composed the following: 

1. McNeil ETR contacted DOL/ETA regional offices for assistance in working with 

SESAs. 

2. DOL/ETA and McNeil ETR specified the UI wage data to be requested. 

3. DOL/ETA submitted letters to SESAs requesting UI Wage Record Data. 

4. McNeil ETR, with DOL/ETA assistance, negotiated with each SESA for production 

of data. 

5. SESAs processed request and produced data. 

6. McNeil ETR received data and undertook review and cleaning of data. 

7. McNeil ETR prepared data for analysis. 

Each of the seven steps in the process for obtaining UI wage data was projected to take at least 

one month, and potentially up to three months. Essentially the evaluation team had to “back into” 

the quarter of UI wage data that represented the last quarter for which data could be requested on 

DSYODP participants and have time for the remaining steps in the process. Working back from 

the contract completion date of June 30, 2007, the number of months needed for each step was 

estimated, resulting in the decision that the last quarter of UI data collected would be for the first 

quarter of 2006. 

Because of the significant time lag for the collection and processing of UI wage data by the 

SESAs, McNeil ETR and DOL/ETA agreed that the appropriate population for collection and 

analysis of UI wage data would be participants in Phase I. As shown in Table II.4, UI wage data 

could be collected for an extended period, in effect covering all participants in Phase I regardless 

of when they were placed. 
Chapter II–Evaluation Approach 

McNeil Education, Training, and Research 19

Table II.4. Quarters of UI Wage Data Requested 

Qtr 4th 1st 2nd 3rd 4th 1st 2nd 3rd 4th 1st 2nd 3rd 4th 1st

Year ‘03 ‘04 ‘05 ‘06 

Phase I Pre Pre Prg Prg Prg Prg Prg Prg Prg Prg Prg Post Post Post

Notes: Abbreviations are defined as follows: Pre: the period prior to the beginning of that Phase; Prg: the period of 

program operation of that Phase; Post: the period after the Phase (grant) has been completed 

The number of quarters of data for any particular participant would of course depend on the 

actual quarter in which that individual began employment through DSYODP. Given that 

participants in Phase I had different periods and lengths of employment (i.e., numbers of quarters 

of wage data), the analysis needed to take into account the different bases of data. For Phase I, 

this could range from four to 12 quarters of wage data. In addition, participants’ quarters of 

employment will vary even if they started employment in the same quarter. Some participants 

who began employment in the first quarter of 2004 may continue employment through the period 

of data extraction (e.g., first quarter of 2006) whereas others may have lost employment at some 

point during that period. 

Average quarterly wages for each participant were calculated for each period before, during and 

after DSYODP while taking into account those quarters where youth were not employed. For 

Phase I, the earliest quarters that would serve as the “pre” quarters would be the fourth quarter of 

2002 and the first quarter of 2003. However, for analysis purposes, the “pre” quarters would be 

determined for each individual participant based on his or her employment start date. “Post” 

quarters, obviously, encompass the period after Phase I of the project was completed. Data from 

these periods offer information on the extent to which youth persisted in employment and if 

differences in wage levels persisted for youth placed in employment during DSYODP relative to 

non-placed youth. 

Based on McNeil ETR’s experience with the collection of wage data for the Phase Three 

evaluation of the Youth Offender Demonstration, the evaluators expected that one or more states 

would not provide UI wage data. Initially DOL/ETA submitted letters to each SESA, 

representing the states in which DSYODP participants in Phase I were most likely to obtain 

employment (District of Columbia and Maryland for the D.C. site; Florida for Miami, Illinois for 

Chicago, and New York for New York City). New York indicated via a written response that 

they would not participate in the provision of UI wage data. While early negotiations with 

Illinois seemed to be positive, an agreement could not be achieved to the satisfaction of all 

parties. The final group of SESAs to provide UI wage data were Florida and Maryland. 

2. Analysis 

As discussed earlier, data analysis included both qualitative and quantitative methods that were 

organized around the study questions in the RFQ. In this part is a description of how each of 

these analytic techniques was applied with respect to each of the study questions. Table II.5 

summarizes the core topics from each of the study questions, the data source or collection 

technique, and method of analysis used to assess the data collected. 
 Evaluation of DSYODP–II Final Report 

20 McNeil Education, Training, and Research 

Table II.5. Core Themes from the Study Questions, Data Source, and Methods of Analysis 

Study Question Topic Data Source Method of Analysis

A. Development of a Network of Employers 

Approaches used to identify and recruit employers interviews/ documents description 

Relative effectiveness of approaches interviews/ observations description 

Employer willingness to pay retention wage interviews/ MIS description/ descriptive 

statistics 

Challenges encountered and strategies used by Center staff interviews/ documents description 

Level of business involvement in model interviews description 

Expansion of employer pool over time interviews/ MIS description/ descriptive 

statistics 

Types of employers MIS descriptive statistics 

Types of industries represented MIS descriptive statistics 

Types of industries with vacancies vs. those that hire youth interviews description 

No. placements by employer and industry MIS descriptive statistics 

No. of businesses w/ repeat placements MIS descriptive statistics 

B. Employer Outcomes 

Are employer’s needs for workers being met interviews description 

Types of businesses w/ needs that are or are not being met interviews description 

Fill rate interviews/ MIS description/ descriptive 

statistics 

Adjustments made by Centers to meet vacancy needs interviews description 

Expectations about future vacancies and skills required interviews description 

Employer satisfaction w/ BRC and youth interviews description 

Relative value of services interviews description 

Reasons for working with BRC interviews description 

Recommendations interviews description 

Services requested by businesses MIS descriptive statistics 

Skills and requirements for positions interviews /MIS description 

Turnover rates of placements MIS descriptive statistics 

C. Project Organization and Staffing 

Appropriateness of staff design and level of staffing observations/ interviews description/ descriptive 

statistics 

Recruitment and selection of staff observations/ interviews description 

Performance requirements and expectations of staff observations/ interviews description 

Evaluation, incentives and performance interviews/ documents description 

Staff turnover by center and position over time interviews/ documents description/ descriptive 

statistics 
Chapter II–Evaluation Approach 

McNeil Education, Training, and Research 21

Table II.5. Core Themes from the Study Questions, Data Source, 

and Methods of Analysis (Continued)

Study Question Topic Data Source Method of Analysis 

Reasons for turnover interviews/ documents description 

MIS level of operation observatios/ interviews description/ descriptive 

statistics 

MIS enhancements interviews/ MIS/ 

documents 

description/ descriptive 

statistics 

MIS training observations/interviews/ 

documents 

description/ descriptive 

statistics 

Internal usage of MIS observations/interviews description/ descriptive 

statistics 

Development and implementation of sustainability observations/ interviews description/ descriptive 

statistics 

D. Service Delivery to Youth

How needs of youth are assessed interviews/MIS/ documents description/ descriptive 

statistics 

Extent of services received as identified by youth MIS descriptive statistics 

Resources and networks used to facilitate youth jobreadiness interviews/ documents description 

Needs beyond those provided by grant MIS/interviews description 

Strategies employed and their effectiveness on youth 

retention 

interviews/ MIS/ 

documents 

description/ descriptive 

statistics 

E. Participant Outcomes

Characteristics of youth (age, gender, classification, etc.) MIS descriptive statistics 

Characteristics and placement MIS descriptive statistics / 

multivariate regression 

Placement and retention wage levels MIS/ UI wage data descriptive statistics / 

multivariate regression 

Benefits received by participants MIS descriptive statistics 

Changes in wages MIS/ UI wage data descriptive statistics / 

multivariate regression 

Retention and termination periods MIS/ UI wage data descriptive statistics / 

multivariate regression 

Non-wage advancements MIS descriptive statistics 

Certifications and licensures MIS descriptive statistics 

Educational achievements MIS descriptive statistics 

Recidivism and court avoidance by classification MIS descriptive statistics 

Skills present/lacking at referral MIS descriptive statistics 

F. Referring Organizations

Types MIS descriptive statistics 

Populations served MIS descriptive statistics 

Services offered MIS descriptive statistics 

Number of referrals by type of organization MIS descriptive statistics 

The qualitative data were used to evaluate processes and systems within and across each of the 

four sites. They included in-depth descriptions of: 
 Evaluation of DSYODP–II Final Report 

22 McNeil Education, Training, and Research 

• Observations and semi-structured interviews made during site visits with center staff, 

business clients and referring organization representatives; and 

• Data abstractions from records and documentation collected from the Centers, 

referring organizations and businesses involved. 

In addition to text, these descriptions yielded tables, graphs and arrays of key features to support 

the discussion. 

Qualitative analyses for this report consisted of case studies for each site, triangulation of themes 

based on comparisons of subject matter from staff interviews, observations and reports, and 

interpretive analysis of themes embodied within and across each of the case studies. McNeil 

ETR evaluators read and discussed each site visit report with respect to the interim and final 

analyses and identified themes and topics of concern to DOL/ETA. Then the evaluation team 

went through a comparable process for the overall site reports at the end of the third visits. Using 

NVivo, the evaluators devised analysis frameworks, (conceptual thematic categories that serve as 

a structural outline under which the text can be sorted), and imported the site visit reports into 

NVivo for querying. 

Quantitative methods were used for the outcome analysis and entailed performing descriptive 

analysis and multivariate regression analysis of data collected from site visits, MIS data and UI 

wage records. Descriptive analyses yielded univariate statistics including frequencies, averages 

and other measures of distribution for key data elements. These are arrayed and presented in 

tables and graphs that enhance the qualitative descriptions. 

Upon receiving the MIS data, McNeil ETR researchers extracted the SQL data into Access tables 

and Excel spreadsheets for cleaning, sorting and formatting. The resulting files were saved as 

comma-separated (CSV) files and then read into SAS for merging and analysis. Quantitative 

analyses of the outcomes consisted of descriptive statistics (i.e. frequencies, averages, etc.) and 

cross-tabulations for data elements relevant to the study questions. These are presented in tables 

and graphs and accompanied by text descriptions. 

Including UI wage records helps to extend the analysis at the participant level, namely our 

capacity to predict employment, differences in youth wage levels, changes in wages, and 

differences in youth persistence over time using multivariate regression analysis. Since UI wage 

data are aggregated quarterly, descriptive statistics on average wages and the number of quarters 

youth are employed were used to frame the analysis. Since participants could begin and 

terminate employment at various points in any given quarter, their quarterly wage data were 

dissaggregated into weekly wages by using the dates participants began and terminated their 

employment. 

Multiple regression analysis of MIS and UI wage data compared wage levels and employment 

outcomes for youth placed by the Centers and youth who were not placed by the Centers. Factors 

such as age, gender, education, family size, and industry were controlled for and their impact 
Chapter II–Evaluation Approach 

McNeil Education, Training, and Research 23

estimated on the likelihood of employment (defined as placement by the Centers at retention 

wages), length of employment, average wages, and changes in wages. 

In the next chapter, the implementation and resumption of DSYODP since Phase I up through 

the termination of the Centers’ activities in March 2007 is discussed. 
 Evaluation of DSYODP–II Final Report 

24 McNeil Education, Training, and Research 
Chapter III–Description of DSYODP Phase II Business Resource Centers 

McNeil Education, Training, and Research 25

Chapter III 

DESCRIPTION OF DSYODP PHASE II BUSINESS RESOURCE CENTERS 

In this chapter, characteristics of the DSYODP Phase II sites are described, which include the 

headquarters and four Centers. The project organization and staffing at the four Centers is 

discussed along with the MIS utilized by the staff in the project. 

A. Project Organization 

Three of the four Centers from Phase I continued to the second phase of DSYODP: Chicago, 

Washington, D.C., and New York. Los Angeles was initiated as a site after BI received multiple 

invitations to locate in the Los Angeles County area. The headquarters also moved from 

Washington, D.C. to Baltimore, MD during Phase II of the demonstration while the Washington, 

D.C. Center remained. 

As was noted earlier, the transition from Phase I to Phase II had significant impact on staffing 

levels and staff turnover at the Centers. Staff positions were also redesigned in the transition 

from Phase I to II. (A more extensive description of staff roles and responsibilities, recruitment, 

and backgrounds appears in the Interim Report, along with additional information about their use 

of the MIS and feedback on the MIS through the time of the first site visits.) During Phase II, 

each Center was expected to consist of a Center Manager, up to three BRCs, one HRC, one RS, 

and a Quality and Compliance Auditor (QCA). 

The Center Managers replaced the Vice Presidents from Phase I but their roles essentially 

remained the same. Center Managers were responsible for supervising staff and training new 

staff, except the QCAs. Managers conducted outreach within the community and among 

workforce development professionals to make the mission of BI better known and understood, 

reported to Chief Executive Officer, and ensured that their staff met its goals and priorities. The 

goals and priorities are reported in Table III.1. 

Table III.1. Goals for Each Position in a Center and Its Priorities 

Position Goals Priorities 

Manager Manage staff to meet goals Achieve placement and retention goals 

BRC 1 5 placements/mo.; 90% retained Develop business relationships 

BRC2 5 placements/mo.; 90% retained Develop business relationships 

BRC3 5 placements/mo.; 90% retained Develop business relationships 

HRC 30 interviews; 15 referrals/mo. Good audit; add new agencies; refer 15 “hire-able” youth/mo. 

RS 90% retention rate Make sure youth do not leave jobs 

QCA 90% audit rate Review files; receptionist; office help 
 Evaluation of DSYODP–II Final Report 

26 McNeil Education, Training, and Research 

The Center Managers acted as BRCs as well, except in Washington, D.C. where the manager 

was also the corporate Executive Manager and training specialist for a period of time. Two of the 

Center Managers worked with BI in Phase I: Luis Angomas in Chicago and David DuPuy in 

New York. Thressa Connor McMahon, Grant and Evaluation Manager in Chicago, served as the 

Quality and Compliance Manager during Phase I and took on a managerial role during Phase II 

after the Center Manager resigned. 

BI added the HRC position in DSYODP Phase II. The HRC position was designed to work with 

ROs in identifying good candidates for enrollment at the Centers and with the BRCs to place 

them. BRC positions were refined so that they maintained exclusive contact with business clients 

without having to identify participants for employment. They also worked with an RS to broker 

employer-employee relationships as needed. The RS position remained the same throughout both 

phases of the project. 

During Phase I, it counted against the retention score if a youth left one position for another, 

better one. In Phase II, such moves were condoned. Termed a “hand up,” career advancement 

activity was not counted against a Center’s retention percentage score. 

In Phase II, the QCA replaced the Program Assistant in performing Center audits. Each QCA 

verified the data entries of the other staff and reported the data to the QC Supervisor in Chicago. 

The QC Supervisor served as a Quality and Compliance staff person during Phase I. 

B. Staffing 

Staffing was integral to the successful implementation of the model, yet recruiting and retaining 

staff proved to be difficult for all four Centers. No location was fully staffed at the time of each 

of the evaluator site visits. Los Angeles had hired an entire staff in October 2005 that needed to 

be replaced by March 2006, and all of the Washington, D.C. staff left their positions within two 

months of the first visit. 

Initially, all the hiring was to be arranged by the human resources staff at BI headquarters, and 

the need to hire for all four Centers in July-August 2005 delayed the hiring everywhere. In New 

York, one BRC was hired by October 2005 and three other staff members were hired in 

December. The QCA was hired in April, the day before the preliminary visit. Because the Los 

Angeles staff left by March 2006, the team in place (minus an HRC) was just completing its 

orientation training at the time of the evaluator’s preliminary visit in May 2006. In Chicago, 

there were three staff members in place in May 2006, with these staff covering multiple roles. At 

the time of the Washington, D.C. preliminary visit in June 2006, the staff members were all new. 

The manager was from the corporate staff covering for an unfilled position, and the rest of the 

staff had been hired in January and March 2006. 

Staffing issues at headquarters confounded staffing concerns at the Centers. The human 

resources person in place at the beginning of the grant left the firm before the hiring was 

completed. The finance officer left the firm in early 2006. The information technology (IT) staff 
Chapter III–Description of DSYODP Phase II Business Resource Centers 

McNeil Education, Training, and Research 27

person left the firm, and a new IT specialist left the firm shortly after arriving. The MIS was not 

updated with new staff names, and reports were delayed as a result of the IT transitions. 

All the managers reported that Center and headquarters turnover had to be brought under control 

because it interfered with their ability to meet BI goals and to honor relationships they developed 

with business partners. Across the four sites, there were potentially 28 job positions (seven at 

each site) to be filled at any point in time (not including executive and administrative staff). Of 

the 28 positions, 10 had a single person for the duration of the grant, 11 had two persons in the 

position, and three had three persons fill the position. 

Staff worked hard according to manager reports, and many considered the work to be a “labor of 

love” for young people who lack many advantages in life. Staff reported that the work was not a 

“9 to 5” job because the youth clients often needed help on weekends and evenings. RSs held 

workshops on budgeting and banking on Saturdays, and took youth to get their driver’s licenses. 

BRCs often received multiple calls from anxious new workers until they settled into their jobs. 

All the Center Managers had business backgrounds. Their hiring philosophies differed by Center. 

New York’s BRCs and HRC held master’s degrees but had no previous workforce development 

experience. While the Los Angeles staff was less formally educated, they had decades of 

workforce and human services delivery experience. The average ages in New York and Los 

Angeles were about 40 years while the Washington, D.C. staff’s average age was 25 years. All 

the Centers aimed to hire culturally diverse staff that matched the cultural backgrounds of their 

clients. Some staff members in New York, Chicago and Los Angeles were also bi-lingual in 

English and Spanish. 

C. Management Information System (MIS) 

To develop uniform processes across the organization, BI revised its Technical Assistance 

Manual (TAM) used during Phase I and maintained use of a centrally designed MIS to capture 

common information from all Centers. The MIS began as a Microsoft Access database that 

allowed the staff to enter new business information, new participant information and activities 

regarding businesses, referring organizations and participants. Each section of the MIS 

corresponded to one of the paper forms. Data were entered directly into the MIS from the paper 

forms. For example, information about a new business was both collected on the Business Needs 

Analysis (BNA) form and entered into the BNA table in the Access database. 

Drop-down menus were developed for some kinds of information and Centers could produce 

reports from any module completed on site. Center staff members were responsible for entering 

data in the appropriate screens from the approved forms. Depending on whether a specific staff 

member had authority to make corrections or updates, a Correction or Update Form was 

completed and approved by the Center Manager and Auditor. Centers reported that the MIS 

responsibilities of the missing staff were not maintained, and it was unlikely that anyone 

retroactively completed these data. 
 Evaluation of DSYODP–II Final Report 

28 McNeil Education, Training, and Research 

Some reports about Center activity were generated by the corporate office, rather than by the 

individual Centers. Centers were not credited with placements until the Program Auditor 

Approval Module (PAAM) report was verified through the central Quality and Compliance 

office. Managers expressed frustration over delays in receiving the PAAM report. 

Overall, staff reported that the Microsoft Access MIS was user-friendly, and the multiple forms 

were easy to manage. The problems that arose were less about the MIS than the burden of data 

entry, computer crashes, slow administrative responses, and lack of IT support. For example, in 

Washington, D.C. the MIS database was only on one computer, so all the staff had to use one 

staff person’s computer to enter data. Staff members at several Centers also used extended hours 

to enter data because they did not have time to enter it during regular working hours. 

In September 2006, BI officially converted its Microsoft Access database to SQL Server 

accessible on the Internet. The transition to a new database format was part of an overall 

upgrading of BI’s information systems. In December 2006, the Centers received new faxes, 

phones, copiers and MIS login IDs. Each component was linked to a central server at the BI 

headquarters via a Local Access Network (LAN) connection. 

The new MIS transition was challenging for staff members in each of the Centers. Staff reported 

having to learn how to use the new system abruptly without adequate training. Some data were 

lost during the change over and printouts of reports and forms were misaligned relative to what 

was visible on screen. Staff did appreciate the speed and similar appearance of the SQL screens 

to the paper forms, although the reporting features were limited under the new system. 

Some staff members were more dependent on the MIS than others to perform their job duties. 

The QCAs, QC Supervisor and Grant and Evaluation Manager were heavily dependent on the 

MIS to generate reports for headquarters and DOL/ETA. The database transition complicated 

both their responsibilities in that there were often discrepancies between the paperwork 

submitted by Centers to the QCA and information in the SQL database. To compensate, staff 

relied on paperwork for accuracy and for a brief period during the implementation, the Grant and 

Evaluation Manager used both the old Access database and the new SQL database to generate 

her quarterly reports to DOL/ETA. 

Problems in the new MIS were addressed gradually through an electronic Help Desk 

implemented by BI’s IT Consultant. Staff members appreciated the Help Desk but lacked 

confidence in the quality of data as a result of the implementation. Initially, the new MIS 

affected staff productivity and the capacity of executive personnel to provide DOL/ETA with 

timely and accurate reports. Near the end of the project, a flood at the central office in Baltimore 

damaged the server and cut short access to the database. Staff members were unable to access the 

MIS for several weeks but paper records were completed. 

Center Managers also used “white boards” of various kinds to track progress and placements. 

The Center Manager in Washington, D.C., in the role as Executive Manager, prepared a 

spreadsheet that enabled Centers to array the participants available for placement along with 

demographic information and skill sets. According to Center Managers, it solved the problem of 

knowing which candidates were available and would be good matches to openings uncovered by 
Chapter III–Description of DSYODP Phase II Business Resource Centers 

McNeil Education, Training, and Research 29

BRCs. Los Angeles used this spreadsheet form while Chicago and New York staff utilized a 

wall-mounted white board system for tracking recruitment activity and morning meeting 

discussions. The limitations of using the white boards were that they were not tied to the MIS 

database, which required staff members to duplicate the process of data entry by hand. 

In the next section, the businesses recruited by BI and the involvement of BI with business 

clients is described. 
 Evaluation of DSYODP–II Final Report 

30 McNeil Education, Training, and Research 
Chapter IV–Business Involvement 

McNeil Education, Training, and Research 31

Chapter IV 

BUSINESS INVOLVEMENT 

This chapter of the Final Report on DSYODP describes the businesses that BI recruited to place 

youth clients. The chapter is divided into two sections: A quantitative description of the 

businesses recruited by BI based on the data collected in the MIS and a qualitative description, 

based on data collected from evaluation visits to the four cities, of how BI and these businesses 

interacted to place youth job seekers. 

A. Characteristics of Business Clients 

The quantitative data BI collected about the employers with which it worked describes the 

industries, the size of their revenues, and the number of their employees. When they were 

contacted by the BRCs, businesses were asked about their current job openings, their typical 

retention rates and the reasons for job turnover. Potential employers were asked about their 

expectations for education and training, benefits and support services, and their willingness to 

hire offenders. 

The employers’ responses to these queries were recorded on the Business Needs Analysis (BNA) 

forms, entered into the database locally, and then forwarded to the central database BI 

maintained. Table IV.1 to Table IV.12 report these features of business clients recruited for the 

four cities in the demonstration. 

Table IV.1 reports the total number of businesses recruited during both phases of the 

demonstration (602) and the total number recruited in Phase II (203). Businesses recruited during 

Phase I remain relevant in Phase II because BRCs depended on the entire list of business clients 

they recruited to place youth in Phase II. 

Table IV.1. Total Number of Businesses Recruited by Business Interface 

and Total Businesses Recruited in Phase II 

City Total Businesses 

Recruited 

 Number Percent 

Total 

Phase I 

Total 

Phase II 

Percent of Total 

Recruited in Phase II 

Chicago 202 34% 163 39 19% 

Washington, D.C. 132 22% 90 42 32% 

Los Angeles 111 18% NA 111 100% 

New York 157 26% 146 11 7% 

Total 602 100% 399 203 34% 

The number of clients BI recruited ranges between 111 and 202 among the demonstration cities 

and the recruitment pattern varied. Data suggest that three of the four demonstration sites 

depended on the early years of business recruitment for Phase II. Los Angles, new to DSYODP 
 Evaluation of DSYODP–II Final Report 

32 McNeil Education, Training, and Research 

in Phase II, recruited 100 percent of its business clients in Phase II, while Chicago (19%) and 

New York (7%) recruited relatively few new businesses in Phase II. Washington, D.C. recruited 

32 percent of its businesses in Phase II. 

BRCs recruited business clients of varying sizes as measured by the number of employees and 

the size of revenues. Tables IV.2 and IV.3 report these size differences. 

Table IV.2. Total Recruited Businesses by Employee Size 

City Fewer than 50 

Employees 50 to 400 Employees More than 400 

Employees 

 Number Percent Number Percent Number Percent Total 

Chicago 51 25% 107 53% 44 22% 202 

Washington, D.C. 53 40% 38 29% 41 31% 132 

Los Angeles 22 20% 40 37% 47 43% 109 

New York 38 24% 57 36% 62 40% 157 

Total 164 27% 242 40% 194 32% 600* 

*Two businesses are missing data. Row percentages are presented for each site. 

Table IV.2 reports size differences by employees in three categories: Business clients with fewer 

than 50 employees, those with 50 to 400 employees, and those over 400 employees. All four 

demonstration sites recruited businesses in all three employee size categories. BRCs in Chicago 

favored mid-size businesses while Washington, D.C. recruited more small businesses. Both Los 

Angeles and New York recruited the largest proportion of large businesses, but they recruited 

almost as many clients with a medium-size workforce as well. 

Table IV.3. Total Recruited Business by Annual Revenues 

City Less than $500, 000 $500, 000 to $5million Over $5million Total 

 Number Percent Number Percent Number Percent 

Chicago 35 17% 94 47% 73 36% 202 

Washington, D.C. 18 14% 42 32% 72 54% 132 

Los Angeles 5 5% 45 41% 59 54% 109 

New York 37 23% 67 43% 53 34% 157 

Total 95 16% 248 41% 257 43% 600* 

*Two businesses are missing data. Row percentages are presented for each site. 

There are three categories of businesses by annual revenue: Less than $500,000, between 

$500,000 and $5 million, and over $5 million. Considering the business clients by the size of 

their total annual revenue indicates a somewhat different pattern from the comparison by number 

of employees. The business clients in Chicago are predominately from the middle revenue 

category as they were by the number of employees. While Washington, D.C. clients were chiefly 

from the smallest category by employee size, the majority of its clients were from the largest 

category when considered from a revenue stand point. Businesses recruited in Washington, D.C. 

were mostly stores or franchises associated with national chains. The clients in Los Angeles were 
Chapter IV–Business Involvement 

McNeil Education, Training, and Research 33

from the largest of the three size categories by either measure while the New York clients were 

from the middle category by revenue and the largest category by employee size. 

Another way of examining the business clients recruited by BI is to consider the industries it 

drew from. Table IV.4 provides the North American Industrial Classification System (NAICS) 

definitions for BI’s business clients. 

Table IV.4. Total Recruited Businesses by Industrial Classification in Rank Order 

NAICS Title Chicago Washington, 

D.C. Los Angeles New York Total 

Other Services (except Public Administration) 10 17 41 38 106 

Health Care and Social Assistance 47 10 20 12 89 

Retail Trade 23 24 10 17 74 

Accommodation and Food Services 16 23 5 18 62 

Professional, Scientific and Technical Services 31 6 1 13 51 

Finance and Insurance 6 5 6 19 36 

Construction 11 13 3 7 34 

Transportation and Warehousing 4 13 8 9 34 

Administrative and Support, Waste 

Management, and Remediation Services 15 9 0 6 30 

Manufacturing 15 1 7 1 24 

Arts, Entertainment and Recreation 1 4 1 6 12 

Wholesale Trade 9 0 1 0 10 

Public Administration 4 1 1 2 8 

Information 4 2 0 2 8 

Management of Companies and Enterprises 1 1 1 4 7 

Utilities 1 3 1 1 6 

Real Estate and Rental and Leasing 2 0 1 1 4 

Educational Services 2 0 1 1 4 

Total 202 132 108 157 599* 

*Three businesses are missing industry data. 

Table IV.4 ranks the industries of business clients for all four demonstration sites, but the 

predominant industries within a site might differ. Note that “Other” services not elsewhere 

classified, is the single largest category for the four cities together and for both New York and 

Los Angeles. For Chicago, healthcare is clearly the predominant industry while it also had the 

most clients from the field of professional, scientific and technical services. Washington, D.C. 

has a more varied industrial profile with more retail and hospitality clients than any other 

industrial group. Taken together, service industries of all kinds are more represented than 

manufacturing, utilities, information, or management fields. 

When BRCs contacted a potential business client, they inquired about its current methods of 

locating employees and the kinds of employees it was trying to hire. BRCs recorded this 

information on the Business Needs Assessment form. Tables IV.5 and IV.6 report these features 

of recruited business clients. 
 Evaluation of DSYODP–II Final Report 

34 McNeil Education, Training, and Research 

Table IV.5 indicates that business clients favored a cluster of job search procedures almost 

equally: Company website, employee referrals, direct applications, and newspaper 

advertisements. Only four percent of the businesses used a pro-active approach to matching 

candidates to positions through an employment agency, so the business intermediary approach 

was an innovation for almost all the firms. 

Table IV.5. Recruitment Methods Used by Recruited Businesses Before BI Services 

Recruitment Method Chicago Washington, 

D.C. 

Los 

Angeles New York Total Percent 

Company Website 51 25 16 42 134 22 % 

Employee Referrals 76 25 4 23 128 21% 

Direct Applications 16 21 56 29 122 20% 

Newspaper 26 40 16 36 118 20% 

Community-Based Organizations 10 4 6 13 33 6% 

Other Website 5 11 3 7 26 4% 

Employment Agencies 13 1 4 5 23 4% 

Post Flyers 5 5 5 2 17 3% 

Total* 202 132 110 157 601* 100% 

*One business is missing information in LA. 

BRCs recruited business clients who were likely to have positions they could fill. Table IV.6 

shows that most of the openings available in recruited companies were in the unskilled and 

skilled worker categories for which BI youth were more likely to qualify. Although many youth 

were lacking in specific workforce skills, some youth such as those completing Job Corps, had a 

trade certificate that would qualify them for a skilled position. 

Table IV.6. Types of Positions Businesses Were Seeking When Recruited by BI (N=602) 

Unskilled Positions Skilled Positions Management Positions Site 

Number Percent Number Percent Number Percent 

Total 

Chicago 108 36% 170 56% 20 7% 305 

Washington, D.C. 90 38% 112 48% 30 13% 234 

Los Angeles 90 37% 105 44% 46 19% 241 

New York 118 49% 95 40% 25 10% 240 

Total 406 40% 482 47% 121 12% 1020 

Row percentages are included for each site. 

A factor in selling the business intermediary approach to business clients was a commitment by 

the youth placements that they would remain on the job for six months. A question for BRCs was 

what the job retention rate was for these companies before placing youth clients. They also 

wanted to know the reasons departing employees reported for leaving. The rates of retention 

varied between skilled and unskilled workers. Tables IV.7 and IV.8 indicate these differences.1

 

1 The basis for calculating the retention rate is not clear, that is, there is no indication on the BNA form as to whether this rate is based on a 60-

day or a six-month basis. There is no guidance in the Technical Assistance Manual, but data gathered during our field interviews suggest that 

different industries have different standards for reporting retention. This would mean that the bases differed across business client reports.
Chapter IV–Business Involvement 

McNeil Education, Training, and Research 35

Table IV.7 reports retention rates for unskilled workers among recruited business clients. The 

majority of employers reported employee retention averages of 50% or higher for unskilled 

employees, except for Los Angeles where the retention rate for most employers was between 

40% and 50% among unskilled employees. 

Table IV.7. Typical Unskilled Worker Retention Rates of Recruited Businesses 

Before Using BI Services (N=600)* 

Chicago Washington, D.C. Los Angeles New York Total Retention Rate 

Number Percent Number Percent Number Percent Number Percent Number Percent 

Less than 40% 8 4% 16 12% 13 12% 30 19% 67 11% 

40% - 50% 32 16% 17 13% 41 37% 40 26% 130 22% 

50% - 75% 93 46% 52 40% 39 36% 46 29% 230 38% 

75% - 85% 47 23% 30 23% 11 10% 20 13% 108 18% 

Greater than 85% 22 11% 16 12% 6 5% 21 13% 65 11% 

Total* 202 34% 131 22% 110 18% 157 26% 600 100% 

*Two businesses are missing information. 

Table IV.8 indicates that businesses in some cities were able to retain higher numbers of skilled 

employees on average. Business clients in Chicago and Washington, D.C. reported most skilled 

employee retention rates at 75% or higher. In Los Angeles and New York the skilled employee 

retention rates are reported at 50% or higher. 

Table IV.8. Typical Skilled Worker Retention Rates of Recruited Businesses 

Before Using BI Services (N=600)* 

Chicago Washington, D.C. Los Angeles New York Total Retention Rate 

Number Percent Number Percent Number Percent Number Percent Number Percent 

Less than 40% 2 1% 9 7% 3 3% 17 11% 31 5% 

40% - 50% 16 8% 9 7% 14 13% 12 7% 51 8% 

50% - 75% 47 23% 41 31% 48 44% 48 31% 184 31% 

75% - 85% 88 44% 49 37% 33 30% 42 27% 212 35% 

Greater than 85% 49 24% 23 18% 12 11% 38 24% 122 20% 

Total* 202 34% 131 22% 110 18% 157 26% 600 100% 

*Two businesses are missing information. 

Table IV.9 reports the main reason business clients say their employees offered for leaving 

positions. The most commonly reported reason for leaving a job was overwhelmingly a lack of 

interest in the work. The next four reasons cluster around the jobs themselves: Advancement 

found elsewhere, low pay at the existing position, no opportunity for advancement, and lay-offs. 

Two reasons given relate to the workers’ circumstances: Transportation and child care issues. 

Finally, lack of benefits and temporary disabilities complete the list. 
 Evaluation of DSYODP–II Final Report 

36 McNeil Education, Training, and Research 

Table IV.9. Reasons Recruited Business Give for Turnover 

Before Using BI Services (N=602)* 

Reason Chicago Washington, 

D.C. 

Los 

Angeles New York Total 

Lack of Interest 88 65 21 67 241 

Advancement 9 16 27 33 85 

Low pay 24 9 23 15 71 

No advancement opportunity 25 9 14 21 69 

Lay-off 29 19 8 10 66 

Poor transportation 22 7 12 1 42 

Childcare issues 3 4 1 5 13 

No benefits 2 3 3 4 12 

Temporary Disability 0 0 0 1 1 

Total 202 132 109 157 600* 

*Two businesses in LA were missing information on turnover. 

The BRCs also investigated the services business clients expected BI to offer them, and the 

services were almost evenly divided between screening job candidates and recruiting them. 

Placing youth offenders in jobs with retention-level wages was a major consideration for the 

Demand-side Youth Offender Demonstration Project. Consequently, BRCs asked potential 

business clients if they had a policy about hiring offenders and the nature of the policy. Table 

IV.10 reports the business clients’ responses. 

Table IV.10. Recruited Businesses’ Offender Policies (N=602) 

 Have Policy Policy 

 Number Percent Theft Drug Use Felonies Violence Other 

Chicago 187 93% 101 54 104 121 1 

Washington, D.C. 76 58% 37 21 35 47 7 

Los Angeles 46 42% 35 17 27 38 3 

New York 56 36% 30 18 44 29 0 

Total 365 61% 203 110 210 235 11 

Percentages are calculated as a proportion of the total number of businesses recruited for each site.

Overall, sixty-one percent of the businesses recruited by BI had offender policies. Convictions 

involving violence were the strongest barrier across all sites. Virtually all the business clients 

recruited in Chicago reported having offender policies, and these affected candidates involved in 

violent crimes, felonies of any kind, and theft. Over half the businesses in Washington, D.C. 

reported offender policies affecting applicants with histories of violence, theft, or felonies. Fortytwo percent of the businesses recruited in Los Angeles maintained policies about hiring 

offenders, especially those with histories of violence or theft in their backgrounds. About onethird of the New York business clients had offender policies, and the main concern was with 

applicants with convictions for felonies. 
Chapter IV–Business Involvement 

McNeil Education, Training, and Research 37

When BRCs recruited clients, they inquired about the training services and employer benefits the 

clients would expect. Tables IV.11 and IV.12 report businesses’ responses. 

Table IV.11. Additional Services Requested by Recruited Businesses (N=602) 

Training/Education 

Services Chicago Washington, D.C. Los Angeles New York Total 

Training 

 Industry Specific 97 42 61 43 243 

 Basic Job Skills 90 53 44 19 206 

 Vocational 17 9 1 6 33 

 Financial Management 7 4 0 1 12 

Education 

 GED 69 33 62 54 218 

 ESL 31 4 26 3 64 

 Basic Literacy 29 20 1 13 63 

The training services requested by businesses were categorized as industry specific, basic job 

skills, vocational skills or financial management skills. Industry-specific includes such skills as 

familiarity with computers for the telemarketing jobs while vocational skills mean having 

certification or experience in a trade, such as heating, ventilation and air conditioning (HVAC). 

Having basic job skills means that the entry-level workers have solved work readiness issues, 

such as punctuality, dependability, transportation, and child care. Financial management skills 

include being able to open a bank account and making life choices that allow a worker to live 

within available means. 

The education services requested were GED, ESL, and basic literacy. GED refers to a general 

equivalency diploma that is earned by passing a series of tests that approximate high school 

content knowledge and skills. ESL refers to English as a Second Language instruction for nonnative speakers. Basic literacy is instruction that brings a student to a level of reading, 

calculating and writing needed for work activities and safety. 

Table IV.11 suggests that Chicago employers had the most expectations that BI would locate 

workers who had industry specific and basic job skills. Although less often demanded, the 

employers in other cities prioritized these two types of skills as well. 

Most employers who noted education services expected BI to locate workers with GEDs. 

Chicago and Los Angeles also expected that BI would find workers who would have ESL 

instruction if they were not native speakers. Chicago, Washington, D.C. and New York expected 

that BI would find workers with at least basic literacy. Some businesses also requested additional 

help from the BRCs and these benefits are reported in Table IV.12. 
 Evaluation of DSYODP–II Final Report 

38 McNeil Education, Training, and Research 

Table IV.12. Benefits Requested by Recruited Businesses (N=602) 

Requested Benefits Chicago Washington, 

D.C. 

Los 

Angeles New York Total 

Federal Bonding 9 6 0 8 23 

Tax Credits 80 18 4 15 117 

Health Insurance 7 9 13 2 31 

Transportation 31 5 4 4 44 

Uniforms 80 16 23 8 127 

Totals 207 54 44 37 342 

The benefits that employers requested included Federal bonding, tax credits, health insurance, 

transportation and uniforms. Federal bonding is a program that provides low-cost insurance to 

employers willing to take a chance on hiring offenders. In some cases, a sponsoring organization 

will purchase the Federal bonds and offer them to business clients as incentives to hire persons 

the sponsor is recommending. Tax credits provide a tax reduction for businesses willing to hire 

hard-to-place workers, such as offenders. Providing health insurance is a way for businesses that 

do not offer benefits to receive some assurance from a sponsoring organization that a new 

worker will be reliable because s/he is able to get medical attention as needed. 

Transportation and uniforms are services frequently offered by sponsoring organizations to assist 

low-income workers in attaining stable work. Businesses that asked for transportation assistance 

usually were asking that someone provide bus or subway tokens/passes to see the entry-level 

worker through to his/her first paycheck. Similarly, some positions required that a new worker 

provide his/her own uniform and/or tools. Sponsoring organizations may have been asked to find 

new or good used clothing, tools, work shoes, tool belt, etc. to help a new worker meet the 

uniform requirements. 

Table IV.12 shows that few of the employers requested Federal bonding, but more were 

interested in receiving a tax credit, particularly in Chicago where 80 businesses requested this 

benefit. Few business clients requested health insurance, the most requests coming from Los 

Angeles. Chicago employers were more likely to ask that new workers receive transportation 

assistance than business clients in any other city. Chicago clients were also more likely to ask the 

BRCs to help new workers with finding required uniforms than employers in other cities. 

The information that follows was drawn from a qualitative analysis of the final site visit reports 

prepared by the evaluators for each site. 

B. Relationships between Business Clients and Business Interface 

Data for this section came from the interview protocols prepared for site visits to the 

demonstration cities. The questions of interest to DOL/ETA about employers shaped the site visit 

interviews that are reported in this section, including: 
Chapter IV–Business Involvement 

McNeil Education, Training, and Research 39

• How does the business intermediary approach identify and recruit employers and 

broker job placements for youth ex-offenders and youth at-risk of court or gang 

involvement? 

• How well does BI identify and recruit employers who are willing to hire this target 

population? and,

• Does the use of the intermediary satisfy employers’ needs?

The evaluation visit protocols listed 17 questions that helped the site visitor to understand the 

business, how it connected with BI, what jobs it offered entry-level workers, and the level of 

satisfaction with BI referrals. Evaluators asked these questions about business clients of the BI 

staff. During the second and third site visits, evaluators also contacted employers that were 

clients of the Centers to ask these questions of them directly. The majority of these interviews 

were conducted in person at the place of business. If that proved difficult to arrange, the 

interviewer conducted some interviews by telephone. The evaluators documented the responses 

they received in field notes that became accounts of their site visits. McNeil ETR then prepared 

an NVivo qualitative analysis of these documents by coding them for the topics of interest. 

Table IV.13 identifies the employers contacted by McNeil ETR evaluators, their industries and 

the occupations held by BI placements. The narrative that follows includes reports from both 

staff and business representatives. 

Table IV.13. Employers, Industries and Work Provided to Youth 

Employer Industry Occupation 

Chicago 

 “W” Hotel Chicago Center Hospitality Customer Service 

 LifeSource Blood Banking Call Center Operator 

 Intercontinental Hotel Hospitality Customer Service 

 Allstate Insurance Insurance No Placements 

 Woodbridge Nursing Pavilion Nursing Home Certified Nurse Assistant 

 Little Sisters of the Poor Nursing Home Certified Nurse Assistant 

Los Angeles 

 TeleInterpreters Professional Services Administrative Support 

 Inter-Con Security Security Security Guard 

 Securitas Security Services Security Security Guard 

 Universal Protection Services Security Security Guard 

 Florence Filter Manufacturing Administrative Support 

 Neway Packaging Manufacturing Customer Service 

New York City 

 FreshDirect Home Delivery Grocer Packers, Runners 

 St. Moritz Security Services Security Security Guard 

 King TeleServices Call Center Call Center Operator 

 Pitney Bowes Management Services Administrative Services Administrative Support 

 FJC Security Services Security Security Guard 

 Metro One Security Security Guard 

Washington, D.C. 

 Simply Wireless Cell Phones Sales 

 Jiffy Lube Auto Maintenance Lube Tech 

 America’s Best Contacts & Eyeglasses Prescription Eyewear Receptionist 

 Avis Budget Group Auto Rental Customer Service 
 Evaluation of DSYODP–II Final Report 

40 McNeil Education, Training, and Research 

1. Recruiting Business Clients 

From the site visits, evaluators learned that the BRCs considered a number of factors in choosing 

employers to contact. They considered companies that looked for entry-level workers, were 

hiring all the time and were willing to pay the local retention wage. 

All four Centers took an inductive approach to locating such businesses. They searched the 

Crain’s publication, local want ads and the Internet. They looked for industry trends, such as the 

explosive growth of the security industry, and they attended career fairs. Los Angeles had the 

advantage of a Center director who had been in private sector employment for many years. She 

brought her network of colleagues and contacts that fueled the search for appropriate companies 

in the area. Chicago developed a particular focus on placing healthcare aides, particularly 

Spanish-speaking females while the New York Center did not find healthcare employers willing 

to pay the retention wage. 

All the BRCs knew that they had to sell employers on their services. In New York, the manager 

prepared a script, called “Talk Track,” that all the BRCs used in telephone conversations with 

prospective employers. Chicago, Los Angeles and Washington, D.C. BRCs believed that they 

needed to meet employers face to face to convince them that these youth were serious about 

working and would give them good value for their wages. The Washington, D.C. Center 

assumed that they had only a few minutes to convince a new employer that the service was 

effective and free. The manager of the Los Angeles Center made a particular point of contacting 

public political officials to convince them that the BI services were a benefit to the community; it 

is the only Center that had an on-going relationship with juvenile corrections offices, Juvenile 

Alternative Work Services and Los Angeles County Probation. 

All the Centers believed that it was important to develop dependable relationships with a few 

good sources of employment rather than maintain a scattershot of companies that they could not 

serve well. This strategy allowed the Centers to focus on certain employers that: 

• Had job openings all year long, 

• Offered jobs with characteristics and requirements that the Centers could match 

reliably, 

• Met the minimum retention wage requirements, 

• Understood the youth being referred and were willing to give them a chance, 

• Offered full-time employment that also provided a career path, 

• Processed hiring decisions quickly, 

• Were located along major public transportation routes, and 

• Treated young workers with respect. 

Staff at the New York Center commented several times that its stability was an asset in 

developing and maintaining such relationships. The staff in Washington, D.C., on the other hand, 

reported that staff turnover made these relationships hard to develop. One employer emphasized 

that the BI services were valued, but that the current BRC was the third person they worked with 

in one year. 
Chapter IV–Business Involvement 

McNeil Education, Training, and Research 41

The employers that met all or most of these characteristics included such industries as: 

• Security firms, 

• Optical sales, 

• Auto repair and maintenance, 

• Auto rentals, 

• Nursing homes and hospitals, 

• Legal support services, 

• Back office operations, 

• Call Centers, and 

• Hotels and dining facilities. 

The range of training programs provided by the referring organizations (ROs) limited the range 

of industries for which youth were prepared. Computer skills and customer services offer a range 

of applications in the workforce, but some rapidly expanding industries that hire entry-level 

workers, such as construction, building maintenance, and landscaping were not considered 

because the youth were not offered training in these fields. In Los Angeles the ROs offered little 

training, and the BRCs searched for jobs in any industry that would hire participants at the 

retention wage. 

Some employers were realistic that youth would not want to stay in entry-level positions for 

long. Pitney-Bowes in New York assumed that youth would be moving on in time, and arranged 

flexible hours if the employee wanted to stay in school and work too. LifeSources, an employer 

in Chicago, needed registered nurses and was willing to schedule hours to allow the certified 

nurse assistants to continue studying. Chicago staffs were exploring positions at employers, like 

Target, that would also offer such flexibility. The America’s Best Contacts and Eyeglasses and 

Jiffy Lube in Washington, D.C. described the career paths a youth could follow if he/she liked 

the work. At Jiffy Lube, for example, new employees were informed of a possible career path: 

• Lube Tech, 

• Third Keyholder, 

• Assistant Manager, and 

• Manager. 

Jiffy Lube was uniquely fond of the DSYODP project because of their long history in working 

with formerly convicted youth. 

2. Services to Businesses 

The four Centers provided multiple services to business clients. The employers served by the 

four Centers reported that BRCs offered them what they needed, including: 

• A professional approach, 

• Candidates that match the job requirements closely, 
 Evaluation of DSYODP–II Final Report 

42 McNeil Education, Training, and Research 

• Candidates pre-screened to address issues—personal, such as drug use or offender 

status, and professional, such as appropriate resume and proper attire, 

• A steady stream of candidates for multiple openings, 

• Willingness to negotiate hours, shifts and wages, 

• Follow-up with employers and placements to be sure that the match was working out, 

• Savings of time and money, and 

• Free services. 

Employers at all four sites reported on the professional demeanor of the BRCs. They appreciated 

that the BRCs had good listening skills and that they took the time to learn something about their 

industry and their specific circumstances. Several employers specifically mentioned that BI staff 

took a hands-on approach to its work. These employers reported that BI staff took the employers’ 

point-of-view in developing the relationship, and they compared favorably with other services 

that just wanted to “get a body placed.” A Chicago employer reported that she held BI to a 

higher standard than other organizations because of her experience with good placements. 

At each Center, interviewed employers reported that BI staffs worked hard to understand the 

specific work requirements for the positions that they needed filled. In New York, BRCs wrote 

detailed position descriptions for openings that were then sent to the ROs. Other Centers kept 

spreadsheets of requirements to match with youth applicants in their database. Employers 

reported that the care BI took in matching candidates to positions had reduced turnover and the 

amount of time they needed to spend preparing the candidate for the work. As one security 

employer in New York reported, BI candidates arrived “floor-ready.” A Chicago employer 

reported that turnover had been reduced 5-10% since he hired from BI. 

Centers worked hard to make sure that youth were ready for their job interviews. The Centers 

learned what factors to examine to screen candidates for positions that were available. New York 

City used its threshold questions to screen youth initially. The youth needed to: 

• Arrive on the day and time arranged, 

• Know the name of the person at the Center he/she came to meet, and 

• Know the employer whose position for which he/she was applying. 

The youth, literally, needed to meet these expectations to enter the Center and be interviewed. 

All four of the Centers reported interviewing candidates before sending them to the employer 

interview. They reviewed the youth’s resumes to be sure that they were sufficiently detailed to 

address the responsibilities of the jobs for which they were applying. 

Both Los Angeles and New York staffs coached youth to have a positive attitude about their job 

prospects saying, “This may not be the job you want but it will get you the experience for the job 

you want.” In Los Angeles, the example was given that security services experience was a 

requirement for Transportation Safety Authority positions that paid well and had benefits. 

BRCs learned that qualifications needed for employment varied. Some positions did not require a 

high school diploma if the candidate had other skills, such as strong verbal skills in the case of 

telephone call Centers. In Los Angeles, one security firm wanted younger applicants because the 
Chapter IV–Business Involvement 

McNeil Education, Training, and Research 43

employee would have to patrol an area on bicycle. In Los Angeles, the staffs found that if the 

candidate’s qualifications were strong, the employer would provide all the needed training. 

Offenders were considered on a case-by-case basis at one security firm, but were not considered 

at all by a car rental agency. A call center had no concerns about a youth offender if the crime 

involved did not entail violence because there was no direct customer contact. Other employers 

did not want to consider offenders because of legal issues and did not distinguish the kind of 

work or work setting an offender would occupy. The effort that BRCs made to understand the 

industries and the employers they served allowed them to place clients who might otherwise 

have not been considered at all. 

Employers reported that the youth arrived for their interviews with a good understanding of the 

positions, and the youth interviewed reported that BI had prepared them well for their job 

interviews. The Washington, D.C. Center reported that it initially heard complaints from the 

employers that the candidates were not passing drug tests. It changed its screening process to be 

sure that candidates could pass these tests lest the employers lose trust in BI’s referrals. One 

Washington, D.C. employer reported, however, that he would hire a BI referral before 

considering those referred from other sources. 

Employers at all sites appreciated the effort BI staffs made to understand why a recommended 

client was not hired. Employers saw this as another example of the Centers’ efforts to have 

employers’ interests in mind and to keep learning how to do their work better. 

The Centers reported that employers often needed multiple employees at one time. One Chicago 

employer reported needing 20 to 40 employees every other week; three Los Angeles employers 

reported having 25 to 60 openings at any one time; and a New York security services employer 

needed all the candidates that BI could find. 

The Centers struggled to meet these demands, but in the interest of serving employers, they 

referred candidates even if not all of them met the qualifications for being counted as a BI 

placement. Many ROs had job seekers above the 25-year-old age limit; and BI staff would 

recommend them along with those who met the age criterion. BRCs in New York and Los 

Angeles were also willing to negotiate for the retention wage. They might accept a placement at 

fewer than full-time hours if the employer gave reasonable assurance that the job would become 

full-time. They accepted night shift placement if doing so would gain the retention wage. In 

Washington, D.C., youth offenders were prepared to take positions below the retention wage to 

demonstrate to employers that they were work-ready and were finished with their legal problems. 

Employers appreciated the effort the BRCs made to get them the workers they needed and in a 

timely way. 

Only New York had a full-time RS, but all four Centers followed-up with clients after 

placement. Employers appreciated that BI remained involved to be sure a participant was 

successful and to assist if there was a problem. Some Human Resource offices did not track the 

retention of new employees. Once the person was hired, he/she could be assigned to multiple 

sites, and only the local manager would know if he/she were still working. The Human Resource 

contact remained appreciative that someone was following up with the placement. 
 Evaluation of DSYODP–II Final Report 

44 McNeil Education, Training, and Research 

A theme throughout the employer interviews was that working with BI saved them time. They 

cited the good matches and the pre-screening as the two most valuable services that BI offered 

them. Part of the time saving was the readiness of the person to work; part was the reduced 

turnover, which saved them the new recruitment effort. The Washington, D.C. Center reported 

that one happy customer, Simply Wireless, recommended BI to Cingular Wireless. 

Since a number of the employers were small operations, they appreciated that BI services were 

free. They typically did not contract with a firm to search for candidates, relying on newspapers 

and Internet searches. With BI, these employers found better candidates in less time. 

None of the employers was willing to pay enough for placements to substitute for the DOL/ETA 

funding. Two employers in New York were considering out-sourcing their Human Resources 

operation and wanted BI to consider bidding for it. Several New York BRCs believed that they 

could operate a work readiness program that would make them eligible for workforce training 

dollars. Between the outsourced private work and public training dollars, the BRCs believed that 

they could maintain a financially viable intermediary service. The operation closed before such 

options could be pursued. 

The Centers learned what barriers were difficult to overcome in serving businesses: 

• The lack of enough, well-prepared candidates, 

• The youth and immaturity of some clients, 

• Turnover due to wage and benefits issues, and 

• The lack of transportation options. 

Every Center reported difficulty in finding enough well-prepared candidates for the position 

openings that existed. Some of the ROs that were effective in preparing work-ready candidates 

graduated only a handful of youth at a time. Some youth remained unprepared, even if they had 

completed high school and had other work experience. One Chicago employer reported that the 

clients referred had poor communication skills, and this was an area of weakness that BI staff in 

other Centers also recognized. 

Even though BI used a retention wage as a way to induce youth to remain employed at least six 

months, the wage was too low for some clients to maintain independent living. Some employers, 

moreover, offered the retention wage but not benefits, a problem for workers with family 

obligations. 

The lack of a vehicle and a driver’s license limited the options for some clients. In Chicago, 

LifeSource was reluctant to hire BI youth because the employer needed workers in a suburban 

office and that required a car. In Los Angeles, the job opportunities were all over the LA Basin, 

and it was difficult to find positions that clients could access from public transportation. In 

Chicago, the job openings were all near the BI Center, and in New York, jobs were sought that 

could be reached from the subway system. 
Chapter IV–Business Involvement 

McNeil Education, Training, and Research 45

Summary 

In this chapter, the characteristics of business clients were examined when BI recruited them and 

the interactions of the BRCs and business clients once they were working together were 

described. The descriptions at the time of recruitment are drawn from BI’s central database of 

Business Needs Analysis forms. The interactions after recruitment are based on site visit 

interviews with BI staff, business clients, and youth. 

Over the period of performance, BRCs in each demonstration city recruited over 100 business 

clients. These clients were drawn from small, medium and large businesses in a wide range of 

industries. Before working with BI, these businesses used few pro-active strategies for recruiting 

workers and they had openings for workers at all skill levels. Many of these business clients 

reported relatively high turnover rates for low skill workers but somewhat more stable patterns 

for skilled workers. The reasons workers gave for leaving employment of these business clients 

were chiefly related to the jobs themselves: lack of interest, better advancement elsewhere, low 

pay, or lack of advancement. 

Businesses that BI recruited were looking for help with screening and recruitment. Most had 

offender policies that would limit some offenders from being placed, especially if their offenses 

involved theft, violence or felonies. Most business clients did not, however, request Federal 

bonding to offset the risk they would have from hiring an offender. Tax credits for hiring hard-to 

place workers seemed more appealing to businesses. 

Many employers requested BI to locate workers with industry-specific training, basic work skills 

and vocational training; few expected BI to specify financial management as a required skill. 

Some employers expected BI to locate workers that had earned their GEDs, received ESL 

instruction if not native speakers, or had at least basic literacy education. Few business clients 

expected BI to assure that workers would have health insurance or transportation assistance, but 

businesses in Chicago expected help with locating uniforms for its new (mostly healthcare) 

workers. 

The primary employer study questions posed by DOL/ETA for the DSYODP were: 

• How did the business intermediary approach identify and recruit employers and 

broker job placements for youth ex-offenders and youth at-risk of court or gang 

involvement? 

• How well did BI identify and recruit employers who are willing to hire this target 

population? and

• Did the use of the intermediary satisfy employers’ needs?

From the site visit interviews, evaluators learned that BRCs used inductive processes to identify 

a small number of businesses that were always hiring entry-level staff, were open to hiring youth 

offenders or youth at-risk of court or gang involvement, and willing to pay the local retention 

wage. Once identified, BRCs knew they had to sell their services to the employers, and they 

worked hard to maintain contact to keep these reliable sources of job placements. 
 Evaluation of DSYODP–II Final Report 

46 McNeil Education, Training, and Research 

Some employers were willing to consider hiring offenders while others were not. The number of 

placements was limited more by the paucity of work-ready youth than the employers’ attitudes or 

the number of job openings. 

Employers reported being pleased by the services BI offered them. The employers noted the 

professionalism of the BRCs and the care they took to understand the nature of the employers’ 

industry and its labor needs. Employers liked the experience of having BRCs contact them if a 

referral was not hired in order to learn better how to improve quality. They particularly 

appreciated savings in time and money by having pre-screened candidates from an organization 

that paid close attention to the specific job qualifications of their openings. While several 

employers commented that they were happy to give disadvantaged youth a chance, most said that 

the BI involvement strengthened their bottom line. 

Both the youthful job seekers and the employers reported that BI prepared candidates well for 

job interviews. BI staff reviewed resumes, held mock interviews, and gave candidates exercises 

to make sure that they were confident if asked to demonstrate work-related skills. Employers 

were pleased that BI staff remained engaged with youth after placement to assist them with their 

transition to work. 

Employers reported that few entry-level workers would remain in those positions for long. Some 

made it possible for their employees to continue with their education; some showed the newly 

hired a career path open to them. Other employers reported that their industry was used to high 

rates of turnover and did not expect the young workers to stay longer than the six-month period 

to which they had agreed. 

BI staffs targeted employers that had openings all the time, so it is not surprising that the 

employers were disappointed that BI could not provide the numbers of workers they needed. 

Sometimes BRCs encouraged a youth to take a position for lower pay, an inconvenient shift, or 

fewer hours after negotiating with employers to give a salary increase or improve the hours 

within six months. BRCs negotiated with the employers to pay the retention wage to younger 

workers eligible under the BI criteria for placement in exchange for BI finding enough older 

workers to meet the employers’ needs. 

Employers appreciated BRCs’ efforts on their behalf, but they were not willing or able to pay for 

the screening and placement services that would make BI financially independent of DOL/ETA 

funding. Staff members at the Centers were exploring other ways to finance their operations but 

the Centers were closed before they were able to complete such arrangements. 

Interacting with business clients was one half of the relationship developed in the business 

intermediary model. BI also established relationships with organizations that prepared youth for 

entry-level work. Describing these organizations is the topic of the next chapter. 
Chapter V–Referring Organizations 

McNeil Education, Training, and Research 47

Chapter V 

REFERRING ORGANIZATIONS 

This chapter of the Final Report on DSYODP describes the ROs that referred youth participants 

to BI. The chapter is divided into four sections based on the combination of data collected from 

the centralized BI database and qualitative descriptions from data collected during the site visits 

to the four cities and/or phone interviews with ROs. Sections include the following: 

A. The Number of ROs Recruited by BI, 

B. Populations Served by ROs, 

C. Services Provided to Youth by ROs, and 

D. Relationships Between ROs and BI. 

A. The Number of Referring Organizations Recruited by BI 

With respect to Business Clients, ROs represent the opposite end of the intermediary model by 

referring youth to the Centers for placement in employment. HRCs demonstrated various 

strategies in identifying ROs. This included utilizing referrals from other ROs, contacting 

vocational schools, job readiness organizations, job placement organizations, and other 

community-based and/or government employment service agencies that serviced youth meeting 

the demographic requirements of BI. 

Once ROs were identified, HRCs established contact with key personnel in the organization, like 

youth case managers or career development specialists, to determine if BI could work with the 

RO. Once confirmed, the HRC entered the RO into the database and worked with their contacts 

to establish a protocol for recruiting youth. Of course, not all youth referred by ROs were eligible 

for DSYODP or placed in employment. Table V.1 shows the total number of ROs recruited by 

BI and the total recruited during Phase II. 

 

Table V.1. Number of Referring Organizations Recruited by Site 

Cities Total ROs Total Phase II 

Percent 

Recruited in 

Phase II 

Chicago 67 15 22.4% 

Washington, D.C. 47 11 23.4% 

Los Angeles 48 48 100% 

New York 81 20 24.7% 

Total 243 87 35.8% 

Similar to businesses, the majority of each Center’s ROs were recruited during Phase I of 

DSYODP with the exception of Los Angeles, which began in Phase II. Of the total 243 referring 

organizations, New York had the largest share followed by Chicago, Los Angeles and 

Washington, D.C.. There were a total of 87 referring organizations recruited during Phase II. For 
 Evaluation of DSYODP–II Final Report 

48 McNeil Education, Training, and Research 

Phase II, Los Angeles had the largest share of recruitments followed by New York, Chicago and 

then Washington, D.C.. 

B. Populations Served 

Information on ROs was collected by the Centers on a Referring Organizations form. Table V.2 

shows characteristics of the populations served by the four BI Centers based on data collected 

from this form. 

Table V.2. Populations Served by Referring Organizations 

Of the 243 ROs recruited, most worked with youth offenders and/or youth at-risk of court 

involvement. Over half the organizations served WIA-eligible clients, and almost half provided 

services to persons with disabilities and Welfare-to-Work eligible clients. Ninety organizations 

provided services to clients living in Empowerment Zones, and roughly a quarter serviced 

seniors. 

In addition to data from the MIS, data collected from site visit interviews suggests that ROs 

recruited served a broad range of constituents; from recent high school graduates to those 

seeking career changes. Table V.3 shows a list of ROs interviewed by evaluators during visits to 

the sites and by phone. 

Table V.3. Referring Organizations and Contact Position by Site 

Referring Organizations Contact Position 

Chicago 

College of Office Technology Career Services Assistant 

Illinois School of Health Careers Director of Career Services 

Casa Central Job Counselor and Case Manager 

Uhlic Children’s Advantage Network (UCAN) Career Services Director 

Central States SER Lead Youth Development Specialist 

Coyne Institute Career Development Representative 

Los Angeles 

South Los Angeles WorkSource Center EO Officer and Business Services 

Los Angeles Job Corps Center Senior Career Transition Specialist 

Los Angeles County Probation Employment Resource Coordinator and Deputy 

Probation Officer 

WorkSource Center, Urban League Assistant Director 

Youth 

Offenders 

WIA 

Eligible 

WtW 

Eligible Seniors Persons with Disabilities 

Empowerment 

Zone 

Chicago 55 46 25 19 26 27 

Washington, D.C. 39 25 21 14 24 20 

Los Angeles 44 25 14 18 28 12 

New York 66 49 41 13 31 31 

Total 204 145 101 64 109 90 
Chapter V–Referring Organizations 

McNeil Education, Training, and Research 49

WorkSource Center, Urban League Business Specialist 

Community build Case Manager 

PV Jobs Case Manager 

American Security Group Training Coordinator 

New York City 

South Bronx Job Corps Center Career Development Specialist 

Vocational Foundation Workforce Manager 

The City of New York Business Link Account Manager 

Stanley Isaacs Community Center Youth Employment Job Developer 

STRIVE (Support and Training Result in Valuable 

Employees) 

Job Developer 

Goodwill/ Brooklyn Job Developer 

FEGS (Federation Employment and Guidance 

Service) 

Job Developer 

Washington, D.C. 

LAYC (Latin American Youth Center ) Job Developer 

VW Associates Washington, D.C. Chamber of Commerce 

Career Blazers Supervisor 

DOES (Department of Employment Services) Job Developer 

Some ROs served participants as young as 16, while others as old as 53. The College of Office 

Technology in Chicago estimated that 25% of its students were youth. It extracted a list of 

students in the BI demographic range to strategize on ways to funnel youth to the Center. 

Participants typically came from low-income households and were plagued with financial-based 

needs such as access to a working phone, transportation, etc. The population served also included 

youth that were wards of the state. Youth who screened positive for drugs were typically not 

allowed into programs. 

At VW Associates in Washington, D.C., the program focused on rehabilitating ex-offenders ages 

18 to 53. It provided skills training to prepare the youth participants for employment, but to get 

into the program youth were required to have either a diploma or GED. STRIVE in NY worked 

with clients that had difficulty keeping jobs. These included people with a record of tardiness or 

repeat offences. Their mission was to reduce the barriers to stable employment. 

Business Link in New York referred welfare clients transitioning to work. The office serves 

clients of all ages, but one staff member was the primary contact with BI and referred only youth 

18 to 25. The eligible youth in the pool were referred to the RO by their caseworkers as 

employable. When it received a posting, the staff mailed it to eligible clients on public 

assistance. FEGS in NY prepares persons on public assistance for employment. 

In Washington, D.C., DOES focused on servicing 20 “hot spot” neighborhoods identified by the 

mayor as having high crime and unemployment rates. Many of the residents of these areas had 

no employment income, no retirement, and no social security to look forward to. 

In New York, Vocation Foundation helped youth between 16 and 24 years by providing job 

training. It drew youth from all five boroughs of the city, focusing on youth from economically 

disadvantaged backgrounds. Vocation Foundation recruited low-income students primarily 

through the New York Public Housing Authority. High school guidance counselors also referred 

students who were unlikely to complete diploma studies by age 18. 
 Evaluation of DSYODP–II Final Report 

50 McNeil Education, Training, and Research 

C. Services Provided 

Because of the nature of the intermediary model implemented by BI, ROs play a crucial role in 

providing essential services that facilitate the matching of youth to business clients with 

vacancies. The Centers were not designed to directly provide a range of services, but rather they 

linked participants to organizations that could provide different types of support throughout the 

process, from application through placement and retention. The following tables describe the 

services available from ROs across the four sites. These data represent the capacity of ROs to 

provide services to participants; data on participants does not indicate the extent to which 

individuals actually received specific services from these organizations. 

1. Job-Related Services 

Many ROs were job training institutions of various types, including community colleges, nonaccredited career colleges, or state-sponsored job training programs. While some focused 

specifically on students’ completing their high school or GED and receiving certification in a 

trade, others required high school diplomas or GEDs before students were able to enter their 

programs. The crucial service as identified by both representatives of the ROs and Center staffs 

was assessing each youth’s readiness for work, and then providing individualized services to 

help improve the level of work-readiness. Table V.4 shows job-related services provided by 

ROs. 

Table V.4. Job-Related Services Provided by Referring Organizations 

Chicago Washington, 

D.C. 

Los 

Angeles 

New 

York Total Percent of 

Total ROs 

N 67 47 48 81 243 

Resume Writing 52 36 35 69 192 79.0% 

Case Management 47 38 38 68 191 78.6% 

Interviewing 49 36 32 66 183 75.3% 

Pre-Employment Service 45 28 24 62 159 65.4% 

Time Management 31 30 30 45 136 56.0% 

Skills Risk Assessment 28 27 19 50 124 51.0% 

Professional Clothing 28 30 23 37 118 48.6% 

Conflict Management 27 22 16 45 110 45.3% 

TABE (Test of Adult Basic 

Skills) Testing 35 17 14 28 94 38.7% 

Drug Screening 18 17 13 17 65 26.7% 

Re-Entry Service 14 13 13 23 63 25.9% 

Criminal History 9 6 10 12 37 15.2% 

TB Testing 5 9 7 7 28 11.5% 

As evident in the table, most ROs provided students with skills needed for resume writing, case 

management, interviewing, and other pre-employment services. Fewer ROs offered services to 

assist youth with soft skills development or services associated with at-risk behavior or criminal 

history. This is indicative of the difficulty faced by ROs and BI in placing ex-offenders, 
Chapter V–Referring Organizations 

McNeil Education, Training, and Research 51

especially convicted felons. A few ROs had the resources and ability to screen participants onsite. In Chicago, New York or Washington, D.C., for example, business partners and BI staff 

members came on-site to do pre-screening, recruitment and drug tests. The HRC later contacted 

the clients who passed the screening with the date for a pre-employment interview or test. 

2. Skills Training Services 

Table V.5 shows the skills training provided by ROs in each of the four cities. Over 60% of ROs 

offered computer training in one form or another. Over 40% of ROs reported offering GED 

preparation or vocational training and close to 40% offered employer-specific training. This 

training may have included accredited training in technical fields and employment preparation in 

the form of career plans managed on a case-by-case basis. For example, DOES in Washington, 

D.C. provided job training through on-the-job, subsidized employment activities as part of the 

mayor’s program to improve labor market success for residents in 20 neighborhoods in the city. 

Table V.5. Skills Training Services Provided by Referring Organizations 

Chicago Washington, 

D.C. 

Los 

Angeles 

New 

York Total 

Percent 

of Total 

ROs 

N 67 47 48 81 243 

Computer Training 35 31 27 56 149 61.3% 

GED 22 23 16 50 111 45.7% 

Vocational Training 24 27 23 36 110 45.3% 

Employer-Specific Training 24 26 12 33 95 39.1% 

Literacy Training 10 22 12 23 67 27.6% 

ESL 14 12 12 29 67 27.6% 

Subsidized Training 12 15 11 24 62 25.5% 

Finance Literacy 15 16 9 20 60 24.7% 

Fewer ROs offered services like literacy training, English as a Second Language (ESL) training, 

subsidized training or financial literacy. These services could also be characterized as supportive 

services (discussed in Table VI.9) that foster the development of life skills important for job 

attainment, civic participation, and functionality. 

3. Additional Services 

Table V.6 shows the additional training services offered across the four cities. Based on 

experience with three rounds of the Youth Offender Demonstration Project, McNeil ETR learned 

that these services can be particularly important for disadvantaged youth. In effect, they can be 

as important as those services listed previously as “job-related” because they increase jobreadiness and a youth’s chances of success once employed. 
 Evaluation of DSYODP–II Final Report 

52 McNeil Education, Training, and Research 

Table V.6. Additional Services Provided by Referring Organizations 

Chicago Washington, 

D.C. 

Los 

Angeles 

New 

York Total 

Percent 

of Total 

ROs 

N 67 47 48 81 243 

Self Image, Self Esteem 33 31 31 48 143 58.8% 

Communicating 33 30 25 49 137 56.4% 

Clothing Assistance 30 33 25 38 126 51.9% 

Teamwork 28 23 18 35 104 42.8% 

Anger Management 19 26 18 27 90 37.0% 

Education Tools 20 22 15 27 84 34.6% 

Legal Assistance 6 12 13 12 43 17.7% 

BI relied on ROs to provide some degree of soft skills training to participants they referred. 

Whenever necessary, BRCs gave workshops on attitude and proper workplace etiquette and 

attire. 

Some ROs were community service organizations offering a wide range of social services and 

programs to the Hispanic community. For example LAYC offered 10-week training sessions, or 

modules, in which participants were evaluated based on their attendance and motivation. Another 

RO, the South Bronx Job Corps Center in New York, had an anger management class if the 

youth needed it. Students were also given a clothing allotment every 60 days, which they used to 

shop for work clothes. 

4. Targeted Services 

Targeted Services are prescribed career tracks that RO participants embark upon. Their hope, 

although not always the case, was to attain employment in their given track following the 

completion of their program. Table V.7 shows the targeted services across the 4 cities. 

Table V.7. Targeted Training Provided by Referring Organizations 

Chicago Washington, 

D.C. 

Los 

Angeles 

New 

York Total 

Percent 

of Total 

ROs 

N 67 47 48 81 243 

Computer/Data Entry 22 29 13 33 97 39.9% 

Clerical/Administrative 20 24 15 32 91 37.4% 

HealthCare 21 16 11 20 68 28.0% 

Customer Service/Telemarketing 12 14 12 25 63 25.9% 

Janitorial/Maintenance 11 16 9 22 58 23.9% 

Technology 9 18 7 15 49 20.2% 

Food Service 6 14 5 21 46 18.9% 

Hospitality 13 14 4 12 43 17.7% 

Retail Sales 7 10 6 14 37 15.2% 

Bank Teller 5 9 4 15 33 13.6% 
Chapter V–Referring Organizations 

McNeil Education, Training, and Research 53

The top targeted services offered by ROs included: computer/data entry, clerical administrative, 

healthcare, customer service/telemarketing, and janitorial/maintenance. These account for over 

half of the specified targeted services. 

ROs across the 4 sites offered career tracks that captured the targeted services and in some 

respects went beyond the scope of the services. The College of Office Technology in Chicago 

trained students of all ages for careers in computer operations, computer network technology, 

administrative medical assistance and phlebotomy. VW Associates in Washington, D.C. offered 

cable installation training. While in New York, the South Bronx Job Corps Center offered 

culinary training, OSHA-Construction/Facility Maintenance, security, A+ Computer 

Certification, business technology, and accounting training. 

The Illinois School of Health Careers, a Chicago RO, offered fully accredited college training 

programs such as Medical Assisting, Dental Assisting, and Massage Therapy. Coyne American 

Institute (Chicago) trained electrical technicians, construction and maintenance and heating, air 

conditioning and refrigeration. It also trained youth for careers in medical administrative 

assistance, medical billing and coding specialists. In the course of their studies, students in health 

career studies spent time on site at a doctor’s or hospital office as part of the curriculum. 

As is shown in Chapter VII, there is a clear relationship between the types of targeted training 

offered by ROs recruited at each site and the types of industries in which youth were placed. 

5. Job Placement Services 

Although BI was instrumental in the job placement of participants, ROs also participated in the 

process. Table V.8 displays job placement services offered by ROs across the 4 cities. Almost 

three quarters of ROs across the four sites provided assistance to youth in preparing an 

employment development plan, and significant numbers of organizations offered the remaining 

services in each city. 

Table V.8. Job Placement Services Provided by Referring Organizations 

Chicago Washington, 

D.C. 

Los 

Angeles 

New 

York Total 

Percent 

of Total 

ROs 

N 67 47 48 81 243 

Employment Development Plan 49 35 31 64 179 73.7% 

Private Sector Employment 26 30 21 41 118 48.6% 

Unpaid work experience 27 20 11 26 84 34.6% 

Job Club/ Peer-to-peer Support 29 13 14 21 77 31.7% 

Job Shadowing 17 18 7 19 61 25.1% 

Based on the interviews, evaluators found that ROs that aided their participants in the job 

placement and retention process believed they saw higher numbers of success, i.e. placement and 

retention. The director of the Illinois School of Health Careers in Chicago was responsible for 

tracking students from training through their transition to work. He often asked the HRC and 
 Evaluation of DSYODP–II Final Report 

54 McNeil Education, Training, and Research 

BRC to participate in the training. He also made weekly contact with BI staff members about the 

students in the program. 

In NY, the South Bronx Job Corps Center and BI provided retention services for six months after 

placement. The RO staff called every month while the BI RS called weekly. The RO believed 

that the combined retention services indicated to the students that others cared about them. The 

Stanley Isaacs Community Center (NY) required a quarterly follow-up with placed youth for two 

years. Youth were expected to come to the Center with a pay stub, and one staff member was 

tasked with trying to locate the youth and encourage their compliance. 

Based on the evaluators’ experience with the Youth Offender Demonstration Project (YODP), 

the first four types of job placement services in Table V.8 can all be very important for younger 

job seekers. An unpaid work experience may actually be a better starting point than private 

sector employment because individuals can test out their job skills in a less intimidating 

environment. Grantees in YODP also found that job clubs or some related form of peer-to-peer 

support could provide significant encouragement to other youth as individuals share stories of 

both frustration and success in seeking employment. In some cases, youth seem to be more 

receptive to the lessons learned by peers who have been going through the same process, rather 

than hearing advice solely from case managers or other adult professionals. 

6. Support Services 

Support services refer to programs offered through ROs to assist participants in areas of life 

outside of work but relative to retention. These areas include: childcare, transportation, housing, 

health services, financial planning education, mentoring, youth services, and counseling. Some 

ROs addressed the needs of youth that could serve as barriers to employment and retention, 

namely, childcare and matching offenses with particular employers so they could prioritize their 

referrals. Table V.9 displays the support services provided by ROs at each site. 

Table V.9. Supportive Services Provided by Referring Organizations 

Chicago Washington, 

D.C. 

Los 

Angeles 

New 

York Total 

Percent 

of Total 

ROs 

N 67 47 48 81 243

Childcare Services 

 Financial Assistance 8 5 7 5 25 10.3% 

 Referrals 38 25 21 52 136 56.0% 

Transportation 

 Passes/Financial Assistance 40 17 27 33 117 48.1% 

 Referrals 19 18 14 32 83 34.2% 

Housing 

 Emergency Assistance 10 9 11 11 41 16.9% 

 Referrals 36 24 26 49 135 55.6% 

Health Services 

 Mental Health Counsel 16 7 10 15 48 19.8% 
Chapter V–Referring Organizations 

McNeil Education, Training, and Research 55

Table V.9. Supportive Services Provided by Referring Organizations 

(continued) 

Chicago Washington, 

D.C. 

Los 

Angeles 

New 

York Total 

Percent 

of Total 

ROs 

 Physical Health Services 9 4 8 6 27 11.1% 

 Referrals 34 20 22 42 118 48.6% 

Financial Planning Education 17 12 11 22 62 25.5% 

Mentoring 20 21 20 29 90 37.0% 

Youth Services 27 22 14 37 100 41.2% 

Additional Services Provided* 3 0 3 3 9 3.7% 

*Additional services include general career counseling, pre- and post-employment counseling and transitional shelter in Chicago; case 

management, food/showers and transitional housing in Los Angeles; and case handling, clerical/computer training and support groups in 

New York. 

Most of the ROs recruited by the Centers referred youth to other organizations for childcare, 

housing, and health services rather than providing these services. Many ROs did provide 

transportation assistance to youth in the form of transit subsidies or financial assistance. The 

Stanley Isaacs Community Center gave participants Metro transit fare for several days to attend 

job interviews, but they did not receive childcare. Central States SER provided bus passes and 

had supportive services dollars to assist disadvantaged youth with clothing. For services they did 

not provide, like mental health care, youth were referred to other local institutions. 

Youth services were also popular among the recruited ROs. In Washington, D.C., LAYC had the 

resources to offer a foster care system (all male ages 16-21), Transitional Living Program (male), 

AmeriCorps, evening reporting alternative incarceration, charter school, and homeless youth 

facilities. In Chicago, Central States SER offered two afterschool programs through 21st Century 

Community Learning Center and Supplemental Educational Services (SES) grants to local high 

schools. 

The Stanley Isaacs Community Center provided services ranging from childcare to elder care, 

after school programs, GED classes and SAT preparation classes. New York’s STRIVE had 

support services such as clothing, housing, counseling, and childcare. STRIVE sent staff to go 

with clients to court appearances. It referred clients for GED classes, but did not provide funding 

for education and training. 

D. Relationships between ROs and BI 

BI and the ROs interacted, ultimately, to serve the needs of employers by preparing youth in 

their quest for employment. Overall, most ROs interviewed were pleased with BI and enjoyed 

working with them. Good placements generally reflected well on the RO, and may have counted 

towards their own measures and goals. BI made it clear to the ROs that its goal was to place 

participants in career-oriented jobs and not part-time or seasonal work. ROs stated that their 

relationship with BI worked well and that BI found outstanding positions for the youth they 

referred. 
 Evaluation of DSYODP–II Final Report 

56 McNeil Education, Training, and Research 

ROs appreciated the BI practice of giving feedback after youth screenings and job interviews. In 

New York, the South Bronx Job Corps Center commented on the fact that BI did pre-screening 

and offered feedback about the youths’ experiences. Its staff reported that BI helped the students 

find jobs because it reinforced the same standards of dress and attendance as the Job Corps 

career development staff. 

BI worked with a variety of ROs, each offering a range of services to youth. The vast array of 

services offered by ROs could potentially have led to well-prepared BI candidates. This was not 

always the case. BI staff in all the Centers mentioned their desire to have received better 

qualified referrals from ROs. Unfortunately many candidates were not job ready. BI staff 

recalled the difficulty they had instilling enthusiasm and pride in the candidates. The lack of 

confidence, work readiness and professionalism in so many candidates indicates the inability of 

certain ROs to instill these characteristics. The former Executive Manager of BI stated that what 

these organizations teach as job readiness is not what BI does; BI teaches the work culture and 

the necessary soft skills. 

If candidates were not well prepared coming from the ROs, they were much more difficult to 

place in employment. Many candidates came to BI expecting a job. Although some of them had 

received minimal training in certain fields at training institutions, they often were not prepared 

with respect to having accurate resumes, appropriate dress, professional attitude, and other “soft 

skills” needed to survive on the job. Oftentimes, participants did not show up to BI on time nor 

did they report to the employer for interviews. 

BI staff admitted that they would like to have had the opportunity to refer candidates back to the 

ROs if they were significantly unprepared for work. Conversely, a few ROs believed that they 

should have been given feedback on their participants that BRCs could not place. 

Some ROs admitted that instilling professionalism was a challenge. BI staff also indicated that 

some ROs did not do a good job in painting realistic pictures to its participants about life and 

work. Youth complained about work and were not satisfied with wages. BI and ROs agreed that 

it was necessary to emphasize the importance of attitude to participants and to build genuine 

relationships with them. 

ROs in all cities noted the difficulty involved in placing youth offenders. ROs benefited from 

their relationship with BI because of its ability to place candidates that were offenders. Some of 

the strategies that developed from their partnership with BI involved prescreening workshops, 

job-matching with employers, and one-on-one preparation. Since September 11, 2001, BRCs in 

Washington, D.C. noted that participants with criminal records have had more difficulty passing 

security clearances because employers check security and credit. However, if a participant’s 

records were sealed that could work in his/her favor. 

Some ROs have made arrangements directly with employers that would not normally hire 

offenders. One way into job opportunity is through subcontracting services and labor. Yet this 

requires youths to own transportation. VW Associates in Washington, D.C. was able to work out 

agreements with subcontractors who were willing to hire offenders who were able to pass a drug 
Chapter V–Referring Organizations 

McNeil Education, Training, and Research 57

test. They accepted certain types of offenders; that is, no violent offenders were selected because 

some subcontractors go into homes. 

In addition to ex-offender status, for a number of youth participants, transportation was a major 

obstacle. This appeared most evident in Washington, D.C. Many youth had to turn down jobs 

because they weren’t able to consistently arrive on time or had to return home during late hours. 

BI was aware of these issues. Perhaps greater collaboration with the ROs could have rectified 

some of these issues. The HRC in Washington, D.C. also stated that the city was experiencing a 

mental health crisis. This was evident, he said, in the youth. One RO noted that the need for 

mental health and pre-employment services is paramount to job placement for the urban, 18 to 

25-year old age group. Participants often coped with issues of domestic violence and single 

motherhood. 

BI realized that most participants seeking job training from ROs did not know the difference 

between non-accredited programs and accredited institutions. Although both offered skill 

training services, non-accredited schools attempted to give students the impression that their 

education was all they needed to work in a given field. Although these institutions had 

counselors that worked with students on job etiquette and honing resumes (basic “career 

services), they were not able to get participants placed with the same consistency as BI. Thus 

participants’ needs went unmet and many did not acquire higher paying jobs for which they were 

qualified for. BRCs had to redirect these candidates into fields that harmonized their existing 

skills with the needs of employers. 

A number of ROs are not able to offer the support services their participants need. One RO 

admitted that it could not help students with childcare needs. The best it could do was to refer 

them someplace where they could get appropriate service. ROs that served youth with more 

serious problems found that if they cannot offer the services, they refer youth to appropriate 

programs to deal with addiction, homelessness, or transition from incarceration. Finding ROs 

that can provide drug screening was challenging in some of the BI cities. Both BI and ROs 

agreed that it was important to catch the youth before they failed the drug test and eventually 

returned to the criminal justice system. 

HRCs were the primary recruiters for ROs at each of the Centers. In some cases, Center 

Managers also recruited and interacted with ROs. In New York, STRIVE was recruited jointly 

by the Center Manager and a BRC. In Los Angeles for example, the Center Manager talked 

directly to the senior manager of any RO that regularly referred youth to them who were not 

work-ready. The RO believed it was her responsibility to tell ROs to improve the job-related 

skills of the youth before attempting to place them with employers. 

BI also came with its own set of issues with respect to maintaining working relations with ROs. 

For example, ROs expressed concern about working with more than one HRC as a result of staff 

turnover in the Washington, D.C. Center. ROs also expressed some frustration with the process 

of trying to gain employment for their participants. The staff of the Chicago RO, The College of 

Office Technology, recalled having sent to BI over 40 people, mostly business and clerical 

skilled students, but none was placed. 
 Evaluation of DSYODP–II Final Report 

58 McNeil Education, Training, and Research 

ROs that supported clients by providing childcare, giving extra preparation, working on 

computer skills, exposing them to administrative functions, and teaching telephone etiquette 

were considered the most effective by BI staffs. These ROs made it easier for BI staffs to do 

their job. For the most part, ROs were pleased to have the relationship they had with BI. 

Cultivating these relations was as much an art form for the HRC as it was a science. At the center 

of these relations were the youth participants, whose characteristics are discussed in more detail 

in the next chapter. 
Chapter VI–Participant Characteristics and Services 

McNeil Education, Training, and Research 59

Chapter VI 

PARTICIPANT CHARACTERISTICS AND SERVICES 

In this section, the characteristics of participants in DSYODP that were placed in employment at 

retention wages (i.e. placements) as well as the services these participants requested are 

discussed. The differences between placements and those youth employed below these levels are 

examined along with data collected from youth during interviews either in person during site 

visits or by phone. MIS data for these participants cover the period from July 2005 through 

December 31, 2006 and include the four DOL/ETA-authorized study sites of Phase II: Chicago, 

Washington, D.C., Los Angeles and New York. These data represent the project during the 

period in which each of the Centers was “functioning at adequate staffing levels”1

 and placing 

youth for Phase II. 

A. Characteristics 

Across all four sites, 1,279 participants were enrolled. Out of these, 950 were determined eligible 

for employment and of those eligible, 413 were placed in employment at retention wages.2 New 

York had the largest number of total and eligible participants with 436 total and 398 participants 

eligible for placement. Los Angeles was the next largest with 325 total participants and 273 

eligible. Washington, D.C. had the fewest number of participants overall, with 248 total, and 127 

eligible participants. 

248

325

436

270

398

273

127 152 188

56 53

116

0

100

200

300

400

500

CH DC LA NY

Total Participants (N=1279) Participants Eligible (N=950) Participants Placed*

Figure VI.1 Total Participants, Total Eligible Participants and Total Participants Placed* 

*Sources: MIS data from July 2005 through December 31, 2006. Counts for Total Participants are from the MIS Participant table/form. Counts 

for Total Eligible Participants are from the MIS Eligibility table/form. Counts of Total Participants Placed are from the MIS Employment 

Information table/form. 

Note: Counts do not include the participants in Baltimore, only the four DOL/ETA authorized sites. There are 440 placement records but there 

were 21 participants placed more than once. 

 

1

 There were 20 recorded placements in January 2007. Most were in New York and Los Angeles. 2

 There are 440 placement records but 21participants were placed in employment more than once.
 Evaluation of DSYODP–II Final Report 

60 McNeil Education, Training, and Research 

The proportion of eligible participants who were placed in employment varied across the four 

sites. Both Los Angeles and New York had the lowest proportion of participants placed relative 

to the total number who were eligible. Roughly 47% or 188 of the 398 eligible youth were placed 

in New York. Over 20%, or 56, of the 273 eligible youth in Los Angeles were placed. Chicago 

maintained the fewest number of eligible participants and the highest percentage of placements, 

with 116 persons out of 152 participants placed, or 76%. Washington, D.C. placed 53 of its 127 

eligible youth (42%) in employment at retention wages. 

There are explanations for these differences. In the case of Los Angeles, the site was brand new 

to this phase and, moreover, had a significant six-month lag in start-up time. This may explain its 

low placement rate since it may not have had the available jobs or relationships with businesses 

to place more participants. Chicago, on the other hand, is the oldest incumbent site with 

experience, well-established relations and strategies for placing youth. In general, Chicago is 

very selective with businesses, youth and referring organizations. It had a strong preference for 

working with organizations that focus on specific training that matches the local job market. 

Moreover, the evaluators learned from the site visits that the Chicago Center staff members were 

extremely stringent in their selection of participants. They only enrolled participants whom they 

were reasonably confident they could place. Chicago also “recycled” participants by sending 

them on multiple job interviews. This increased the individual’s chances of securing employment 

and reduced the need for a large number of enrollees in the program. 

Table VI.1 shows the demographic characteristics for placed participants across the four sites. 

Most participants were at-risk youth roughly 22 years of age, single and without children. Of the 

395 participants at-risk, 388 were considered eligible due to poverty. The two largest ethnic 

groups served were African-Americans and Hispanics. Hispanics outnumbered AfricanAmericans only in Chicago. 

There were slightly more females than male participants overall (51.9% female vs. 48.1% male), 

with greater variation among the sites. Los Angeles and New York both had larger proportions of 

males than females (in Los Angeles, 57.1% male and in New York, 61.2% male), while Chicago 

and Washington, D.C. had a preponderance of female participants placed in employment (in 

Chicago, 73.3% female and in Washington, D.C. 61.5% female). 

Eighty-two of the 413 youth placed in employment were employed at the time of referral to the 

project. The largest proportions of youth employed at referral were in Chicago and Washington, 

D.C.. Few youth reported receiving job-seeking and treatment services at the time of enrollment. 

Of those who did receive these services, transportation was the most needed and largely in Los 

Angeles and New York. 

 

Most of the placed participants were high school graduates. Most of the remaining participants 

had pursued or were in pursuit of high school equivalents or college education. Interestingly, 

most of the vocational graduates were in Chicago while New York participants composed the 

bulk of those who received or were preparing to obtain GEDs. 
Chapter VI–Participant Characteristics and Services 

McNeil Education, Training, and Research 61

Table VI.1. Demographic Characteristics of Participants Placed in Employment* 

Chicago D.C. Los Angeles New York Total 

Total Placements 116 53 56 188 413 

Age at Employment Start Date 

Average 22.1 21.4 21.2 21.7 21.7 

Standard Deviation (years) 2 1.9 2.1 1.9 2 

Gender 

Female 85 32 24 73 214 

 73.3% 61.5% 42.9% 38.8% 51.9% 

Male 31 20 32 115 198 

 26.7% 38.5% 57.1% 61.2% 48.1% 

Classification 

At- Risk of Court Involvement 114 50 49 182 395 

Youth Offender 2 3 7 6 18 

Ethnicity 

African-American 40 43 37 105 225 

Asian-American / Pacific Islander 3 0 1 1 5 

Caucasian 1 0 1 6 8 

Hispanic / Latino/a 67 9 14 72 162 

Multiracial 0 0 1 2 3 

Native American 1 0 2 0 3 

Other 4 0 0 2 6 

Education 

High School Graduate 25 35 31 76 167 

GED Completed 6 1 7 30 44 

In College 26 6 11 24 67 

Vocational Graduate 35 2 1 3 41 

GED Preparation 3 4 2 32 41 

Out-of-School/ No Degree 13 2 3 12 30 

College Graduate 7 2 0 7 16 

In High School 1 0 1 4 6 

Marital Status 

Married 13 2 1 6 22 

Single 103 50 55 182 390 

Number of Placed Participants w/ Children 

Number of Children: 

0 102 46 42 164 354 

1 12 5 13 19 49 

2 2 1 1 5 9 

Number of Placed Participants Employed at Time of Referral 

 42 19 8 13 82 

*Data represent MIS data from July 2005 through December 31, 2006. One participant in D.C. missing demographic information. 
 Evaluation of DSYODP–II Final Report 

62 McNeil Education, Training, and Research 

B. Interviews with Youth Participants 

The evaluators interviewed youth during the second and third site visits or by phone in the event 

that face-to-face interviews could not be scheduled during the visits. Responses to questions 

from the interviews with youth provide additional insights on their recruitment and placement 

experiences. Table VI.2 provides a glimpse of the types of businesses and positions held by 

youth placed by BI who were interviewed as part of the evaluation. In some cases, youth were 

interviewed during the evaluator’s visit with their employer. 

Table VI.2. Businesses and Positions of Youth Interviewed (if Employed) 

Business Participant Position 

Chicago 

 Little Sisters of the Poor CNA 

 Little Sisters of the Poor CNA 

 “W” Hotel Chicago Center Hostess 

 Woodbridge Nursing Pavilion CNA 

 Lakeview Nursing Home Resigned as a CNA 

 Samland Institute Office Assistant 

 Woodbridge Nursing Pavilion Laid off as a Receptionist 

Los Angeles

 RotoRooter Plumber 

 Business Interface Receptionist 

 Universal Protection Services Security Guard 

 Neway Packaging Customer Service 

 Universal Protection Services Security Guard 

 Securitas Security Guard 

New York City 

 King TeleServices Information Services Position 

 Fresh Direct Delivery 

 St. Moritz Security Services Resigned as Security Guard 

 King Teleservices Information Services Position 

Washington, D.C. 

 Filene’s Basement Salesperson 

 Access Litigation Executive Assistant 

 Access Litigation IT Department 

 Avis Budget Group Rental Sales Associate 

Youth were largely recruited into DSYODP directly through the ROs. Youth also learned about 

BI from flyers posted on bulletin boards at the ROs, inadvertent contact with other placed youth, 

or through friends in the DSYODP program. 

Youth reflected positive sentiments about the quality of preparation they received for 

employment. In general, the youth interviewed were pleased with having participated in a 

prescreening process, whether this was sponsored by an RO or BI. Youth in Los Angeles noted 

that Job Corps helped them shape their initial resume and cover letter, but BI gave them a clearer 

perspective on the interview process. Several youth in each of the sites noted that the BRC’s 

mock interview sessions helped to reduce anxiety and clarify expectations. One youth, 

previously employed at Lakeview Nursing Home in Chicago, noted that the job she got through 
Chapter VI–Participant Characteristics and Services 

McNeil Education, Training, and Research 63

BI was her first job. A young Spanish-speaking female, she described herself as very shy and 

nervous, so she considered the mock interviews and resume training with the HRC in Chicago to 

be critical to her achieving employment. In Washington, D.C. and New York, youths commented 

on the benefits of prescreening provided by BI such that they were prepared for their job 

interviews when they arrived, they knew what kind of work they were expected to perform. 

In other cases, BRCs went beyond their job description to provide transportation and clothing to 

youth. Youth also received services like childcare from ROs and other organizations like Action 

for Children. These were particularly noted among the youth in Chicago who were, in many 

cases, young mothers with few financial resources. 

The youth interviewed maintained strong sentiments about the jobs they obtained while enrolled 

with BI. Youth in New York commented that the initial shifts were often difficult, starting early 

in the morning and having to work weekends. Work at Fresh Direct was labor intensive, often 

requiring youth to walk several flights of stairs with packages but tips were lucrative. 

Youth’s choices regarding employment seemed to be reflective of their relationship with key 

contacts in their respective RO and/or BI staff. This had a direct impact on job retention. The 

trust between youth participants and BI staffs was an ongoing theme in the comments by the 

youth interviewed. In the absence of an RS, youth often maintained relations with BRCs and 

HRCs during the initial phases of employment. This relationship factored into youth’s decisions 

and level of commitment to employment. For example, in Los Angeles, a youth participant 

preferred a position in customer service but none were available with the employer at the time of 

application. The BRC asked the youth to consider an alternative position until one was made 

available in their area of interest. Because the BRC was supportive and identified a good 

employer match, the youth agreed to try the position. 

New York hired an RS who contacted youth once a week to check on their progress. Youth 

commented that this helped them maintain their goals through difficult initial periods of 

employment. In some cases, as with the youth in New York who resigned from St. Moritz, youth 

sometimes made hasty decisions to quit their job without consulting with the RS or someone at 

BI. This was a source of frustration for employers and staff at BI, especially in New York where 

the RS prided herself on her ability to mediate the factors affecting retention. Throughout Phase 

II, there were few RSs to monitor youth’s progress, overall, across each of the sites. 

Center staffs encouraged youth to think about and spell out their career goals. Youth often noted 

their goals of continuing schooling and were encouraged by BI staff members to continue in their 

efforts to obtain promotions or new positions. This gentle push by BI staffs for youth to advance 

their education was of particular importance for participants working in trades and skilled 

positions like those in health care in Chicago and those in law enforcement in Los Angeles. 

Obtaining degrees and certifications were often the method by which people obtained 

advancement in these types of positions. 

In the next chapter, outcomes with respect to employment are discussed in more detail. 
 Evaluation of DSYODP–II Final Report 

64 McNeil Education, Training, and Research 
Chapter VII–Employment Outcomes 

McNeil Education, Training, and Research 65

Chapter VII 

EMPLOYMENT OUTCOMES 

This chapter focuses on employment outcomes for participants placed at each of the four 

Centers. Specifically, this chapter explores data from the MIS on placement activity at each of 

the Centers, data on wages and hours, and how employment was spread across participant 

groups, businesses and ROs. 

A. Characteristics of Placements 

Similar to Phase I, the sites in Phase II of the DSYODP commenced operations at different 

points during 2005. Beginning in the 3rd quarter of 2005, placement activity increased across the 

four sites as they ramped up staffing and operations. Los Angeles went through a complete 

rehiring of staff in early 2006. 

Table VII.1 shows the number of placements (youth placed in employment at retention wages) 

by quarter and year for each of the sites. While most sites witnessed declines in placements in the 

3rd quarter of 2006, New York managed to achieve over half of all its placements in that quarter. 

This can be explained by its relationship with two large employers: Fresh Direct and King 

Teleservices. These employers expanded during the holiday season to accommodate the increase 

in business. Placement activity slowed toward the end of 2006 for Los Angeles and New York. 

Washington, D.C. had plans to continue its operations and remained fully staffed for the longest 

period of time. 

Table VII.1. Number of Placements by Quarter and Year by Site 

(based on Employment Start Date) 

Site Quarter 2005 2006 Total 

Chicago, IL 1 20 

2 28 

3 9 15 

4 19 25 

 Total 28 88 116 

 

Washington, D.C. 1 13 

2 10 

3 1 7 

4 5 17 

 Total 6 47 53 

 

Los Angeles, CA 1 2 

2 25 

3 4 13 

4 2 10 

 Total 6 50 56 
 Evaluation of DSYODP–II Final Report 

66 McNeil Education, Training, and Research 

Table VII.1. Number of Placements by Quarter and Year by Site 

(based on Employment Start Date) (continued) 

Site Quarter 2005 2006 Total 

New York, NY 1 25 

2 35 

3 6 95 

4 9 18 

 Total 15 173 188 

 

Total 55 358 413 

Table VII.2 shows the distribution of eligible and placed youth at-risk of court involvement and 

youth offenders across the four sites. Similar to Phase I, most of the participants in Phase II were 

youth at-risk of court involvement; however, total placements for both at-risk and youth offender 

participants were lower during Phase II. The total number of youth placed in employment in 

Phase I was 616, 105 of which were youth offenders. It is important to note that Phase I lasted 

approximately 24 months while Phase II data covered an 18 month period. 

Taking into account the difference in length of periods examined for each phase of DSYODP, 

note that placement rates declined in Phase II by roughly 20% below those in Phase I. Also, each 

of the four sites in Phase II recruited very low numbers of offenders relative to at-risk youth who 

were eligible for employment. 

Los Angeles had an exceptional number of youth offenders in their pool of eligible youth but 

only 20% were placed in employment at retention wages. Evaluators learned from the site visits 

that the Los Angeles Center developed a relationship with the parole board in late 2006 to refer 

youth offenders for job placement services. Youth offenders often face additional constraints to 

employment, namely a support network of case management services and preparation to ensure 

that they are job-ready. BRCs would have been able to gain more placements by focusing on 

non-offenders, particularly since the largest number of job openings identified by BRCs in Los 

Angeles was in the security industry where it is difficult to place youth offenders. Overall, 

because the numbers of offenders placed in employment are so few relative to at-risk youth, it is 

difficult to make inferences about the relationship between status (at-risk or youth offender) and 

employment outcomes. 

Table VII.2. Placements and Placement Rates by Status 

Chicago D.C. Los Angeles New York Total 

Offender At-Risk Offender At-Risk Offender At-Risk Offender At-Risk Offender At-Risk 

Eligible 

Participants 7 145 16 111 35 238 9 389 67 883 

Number Placed in 

Employment at 

Retention Wages 

2 114 3 50 7 49 6 182 18 395 

Placement Rate .29 .79 .19 .45 .20 .21 .67 .47 .27 .45 
Chapter VII–Employment Outcomes 

McNeil Education, Training, and Research 67

B. Industries, Businesses, Referring Organizations and Placements 

It is useful to examine employment outcomes from the perspective of demand-side 

characteristics of industries and businesses in which youth were placed in employment. As 

discussed in Chapter IV, each Center recruited specific businesses within certain industries based 

on their ability to match their needs with the qualifications of youth referred to them. Targeted 

businesses typically represented growing sectors of the local economy, mostly mid-sized, 

service-oriented and in need of both skilled and unskilled labor to fill entry-level positions. 

Table VII.3 lists the industries in which youth were employed and/or placed. 

Table VII.3. Industry by Number of Employed, Placed and Terminated Youth 

Industry Employed Percent of 

Employed Placements Percent of Placements 

Information 125 24.5% 89 21.5% 

Accommodation and Food Services 89 17.4% 67 16.2% 

Health Care and Social Assistance 87 17.0% 80 19.4% 

Other Services (except Public Admin) 83 16.2% 69 16.7% 

Administrative and Support and Waste Management 

and Remediation Services 35 6.8% 35 8.5% 

Retail Trade 34 6.7% 28 6.8% 

Transportation and Warehousing 28 5.5% 19 4.6% 

Construction 8 1.6% 7 1.7% 

Manufacturing 7 1.4% 7 1.7% 

Professional, Scientific and Technical Services 5 1.0% 5 1.2% 

Finance and Insurance 4 0.8% 1 0.2% 

Public Administration 4 0.8% 4 1.0% 

Wholesale Trade 2 0.4% 2 0.5% 

Total 511 100.0% 413 100.0% 

Most youth were employed and placed in information, accommodation and food services, health 

care, and other services industries overall; however, the ranking differs slightly with respect to 

the proportion of placements within these industries. The table does not report all the youth that 

were placed; many youth employed in these top industries were placed with lower than retention 

wages or with fewer than full-time hours. This outcome harks back to our earlier discussion 

about the negotiations that took place between BI staff and employers to secure employment for 

youth while building rapport with employers. BRCs in New York, for example, reportedly made 

concessions with large employers, like King Teleservices, to secure full-time positions for youth 

at salaries below retention wages if youth were willing to accept them. Because these employers 

often made frequent requests for large numbers of candidates, the BRCs accommodated the 

employers to build rapport with the understanding that retention wage positions would be offered 

to youth from BI in the future. 

Given the close relationships established between the Centers and employers, the distribution of 

placements across industries is best understood at the site level. Table VII.4 shows the most 

popular industries for placements by site. The results suggest a much clearer correlation between 

the specific industries targeted by each Center and the results in terms of number of placements.
 Evaluation of DSYODP–II Final Report 

68 McNeil Education, Training, and Research 

Table VII.4. Most Popular Industries among Placements by Site 

 Chicago Washington, 

D.C. 

Los 

Angeles 

New 

York Total 

N 116 53 56 188 413 

Information 12 1 0 76 89 

Health Care and Social Assistance 69 6 5 0 80 

Other Services (except Public Admin) 0 6 33 30 69 

Accommodation and Food Services 8 5 0 54 67 

Administrative and Support and Waste 

Management and Remediation Services 13 8 0 14 35 

Retail Trade 2 21 5 0 28 

For each site, the number of participants placed in a given industry is positively correlated with 

the number of businesses recruited in that industry. The exception is New York, where two 

employers in the information industry accounted for the bulk of their placements. This suggests 

that in most cases, each business client employed only a few placements. Table VII.5 illustrates 

this phenomenon across each of the sites. 

Most placements were also employed in mid-sized firms with 50 to 400 employees and annual 

revenues between $500,000 and $5 million (data not shown). Washington, D.C. was the only 

exception in that 79% of its 53 placements were employed in retail chains and/or franchises with 

less than 50 employees. 

Although the industry outcomes were slightly different for Phase I of DSYODP, (the 

construction industry accounted for the highest proportion: 13.6% of all 616 placements in Phase 

I), all other characteristics of businesses and industries were the same in terms of the distribution 

of placements. 

The distribution and characteristics of ROs that referred youth who were placed in employment 

is also enlightening. There were 77 ROs out of 243 who referred youth resulting in placements. 

Table VII.6 shows the services provided by these ROs. Barring a few exceptions, most ROs 

associated with placements offered the same services. The most popular services offered were 

services to youth offenders, resume preparation, interviewing and case management. The 

Table VII.5. Number of Placements by Businesses 

Number of 

Placements Chicago Washington, D.C. Los Angeles New York Total 

 No. % No. % No. % No. % No. % 

1 13 44.8% 8 44.4% 15 60.0% 11 55.0% 47 51.1% 

2 to 4 9 31.0% 6 33.3% 6 24.0% 2 10.0% 23 25.0% 

5 to 9 4 13.8% 4 22.2% 4 16.0% 2 10.0% 14 15.2% 

10 to 19 2 6.9% 0 0.0% 0 0.0% 3 15.0% 5 5.4% 

20 or more 1 3.4% 0 0.0% 0 0.0% 2 10.0% 3 3.3% 

Total 

Businesses 29 100.0% 18 100.0% 25 100.0% 20 100.0% 92 100.0% 
Chapter VII–Employment Outcomes 

McNeil Education, Training, and Research 69

distribution of services offered differs within most sites. In Chicago, for example, computer 

training, pre-employment and employment specific training were prominent. This makes sense 

given that they worked with colleges and other organizations specifically geared towards training 

youth for healthcare professions. Case management was popular among ROs in Washington, 

D.C., Los Angeles and New York. New York also worked with the largest group of ROs. 

Table VII.6. Services Provided by ROs who Referred Placements 

Chicago Washington, 

D.C. Los Angeles New York Total 

N % N % N % N % N % 

Total 17 10 14 36 77 

Service Youth Offenders 14 82% 10 100% 14 100% 30 83% 68 88% 

Resume Preparation 14 82% 10 100% 11 79% 33 92% 68 88% 

Interviewing 12 71% 10 100% 9 64% 32 89% 63 82% 

Case Management 7 41% 10 100% 12 86% 33 92% 62 81% 

Pre-Employment 11 65% 9 90% 7 50% 28 78% 55 71% 

Computer Training 12 71% 8 80% 8 57% 26 72% 54 70% 

Communication 6 35% 8 80% 7 50% 23 64% 44 57% 

Professional Clothing 6 35% 9 90% 7 50% 20 56% 42 55% 

GED 7 41% 7 70% 5 36% 22 61% 41 53% 

Employment Specific Training 10 59% 7 70% 3 21% 17 47% 37 48% 

ESL 8 47% 3 30% 4 29% 18 50% 33 43% 

Drug Screening 1 6% 6 60% 3 21% 7 19% 17 22% 

Just as few business clients hired more than a few of BI’s referrals, most ROs only referred a 

small number of youth that resulted in placements. Table VII.7 shows the distribution of 

placements among ROs. Chicago, Washington, D.C. and New York had a few ROs responsible 

for referring the majority of their placed youth. That the incumbent Centers would have this 

outcome is not surprising. HRCs at each of the Centers spent a great deal of time establishing a 

pipeline for youth referrals with the most cost effective ROs. 

Table VII.7. Distribution of Placements among ROs 

Number of 

Placements Chicago Washington, 

D.C. Los Angeles New York Total 

 No. % No. % No. % No. % No. % 

1 5 29.4% 1 10.0% 5 35.7% 14 38.9% 25 32.5% 

2 to 4 7 41.2% 6 60.0% 6 42.9% 10 27.8% 29 37.7% 

5 to 9 2 11.8% 1 10.0% 3 21.4% 6 16.7% 12 15.6% 

10 to 19 3 17.6% 2 20.0% 0 0.0% 6 16.7% 11 14.3% 

20 or more 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 

Total ROs 17 100% 10 100% 14 100% 36 100% 77 100% 
 Evaluation of DSYODP–II Final Report 

70 McNeil Education, Training, and Research 

C. Hourly Wages and Hours per Week 

Table VII.8 shows the average hours and hourly wages for youth offenders and youth at-risk of 

court involvement placed in employment at retention wages. There was no significant difference 

in the average number of hours worked per week between status groups. Whether youth were atrisk or ex-offenders, all placements tended to work full-time as required by the project. Youth 

did, however, differ with respect to wages. At-risk youth tended to earn slightly higher wages on 

average than youth offenders. This was also the case in Phase I. Overall, the placed youth earned 

an average of $9.52 an hour, which is notably less than the average wages documented in Phase I 

of the project ($9.65 for youth offenders and $10.27 for at-risk youth). 

Table VII.8. Average Hours per Week and Average Hourly Wages 

by Status of Placed Participants

Chicago Washington, 

D.C. Los Angeles New York Total 

Offender At-Risk Offender At-Risk Offender At-Risk Offender At-Risk Offender At-Risk 

Number Placed 

in Employment 2 114 3 50 7 49 6 182 18 395 

Average Hours 

per Week 40 38.6 40 40 40 39.9 38.3 34.5 39.4 37 

Average Hourly 

Wages $8.31 $9.62 $9.00 $9.25 $9.26 $9.36 $9.33 $9.61 $9.13 $9.53 

It is worth reiterating that wages are based on those recorded at time of employment with the 

exception of a small number of participants who saw recorded changes in their wages within the 

first six months of employment. In this case, the most recent record of employment for these 

youth was used for the purposes of the analyses. 

It is also worth reiterating that some of the outcomes reflected in these data are driven by 

demand-side requirements of the labor market. In other words, staff at each of the Centers 

negotiated the needs of employers with the supply of available job-ready candidates. Staff at the 

Centers noted that some youth were willing to take a job regardless of whether or not it met 

retention wage standards. 

Table VII.9 provides information on the distribution of wages among participants based on 

several demographic characteristics. On average, males earned less than females, married 

participants earned higher wages than singles, although there were few married youth placed in 

employment, and Hispanics earned higher average wages than African-American and other 

ethnic groups. The table also shows that participants with higher levels of education earned 

higher average wages than participants with less formal education. 

Within sites there are particular differences in contrast to the general outcomes. Males in 

Chicago earned significantly higher wages on average relative to females. The opposite is true 

for males and females in Washington, D.C. and New York. With respect to average wages for 

married and single placements, Chicago shows the highest average wages went to single 

participants. Chicago accounts for 13 of the 22 married participants and has roughly a 3:1 ratio 

of females to male participants placed in employment. 
Chapter VII–Employment Outcomes 

McNeil Education, Training, and Research 71

Table VII.9. Average Wages for Placed Participants by Gender, Marital 

Status, and Ethnicity*

Characteristic 

Total Number 

of Placed 

Participants 

Chicago Washington,

D.C. 

Los 

Angeles 

New 

York Total 

Male 198 $10.72 $9.05 $9.38 $9.09 $9.39 

Female 214 $9.18 $9.38 $9.31 $10.40 $9.64 

 

Married 22 $8.99 $9.50 $12.81 $10.92 $9.74 

Single 390 $9.67 $9.24 $9.29 $9.55 $9.51 

African American 225 $9.20 $9.20 $9.14 $9.19 $9.18 

Asian/ Pacific Islander 5 $8.86 $12.81 $10.00 $9.88 

Caucasian 8 $7.57 $8.75 $9.67 $9.29 

Hispanic / Latino-a 162 $9.96 $9.48 $9.60 $10.27 $10.04

Multiracial 3 $9.00 $8.00 $8.33 

Native American 3 $7.57 $10.25 $9.36 

Other 6 $9.40 $8.25 $8.94 

 

High School Graduate 167 $8.94 $9.01 $9.37 $9.68 $9.37 

In College 67 $9.46 $9.87 $9.84 $10.31 $9.86 

GED Completed 44 $8.56 $9.00 $8.84 $9.67 $9.37 

Vocational Graduate 41 $8.57 $10.75 $9.00 $10.55 $8.83 

GED Preparation 41 $8.52 $8.88 $9.60 $8.64 $8.70 

Out-of-School/ No Degree 30 $13.88 $10.50 $8.70 $8.61 $10.84

College Graduate 16 $11.92 $9.75 $11.98 $11.67

In High School 6 $8.50 $8.75 $9.00 $8.88 

*One participant is missing demographic information. 

African-American and Hispanic youth composed the bulk of all placed participants. Within each 

site, Hispanics earned higher average wages than their African-American counterparts. It is no 

surprise that college graduates earned the highest average wages overall. In Chicago and 

Washington, D.C., however, participants who were out of school with no degree earned the 

highest wages. Also, participants in D.C. and New York with vocational certificates earned high 

wages. This phenomenon may be explained in terms of industries in which youth at these 

particular sites found employment. 

D. Placements at Retention versus Non-retention Wage Levels 

At the core of BI’s mission under DSYODP is placing youth in positions that pay retention 

wages. As shown in Table VII.10, wage standards were established for each site based on the 

cost of living in the respective cities. (The hourly rates were calculated by dividing the annual 

income by 2,080 hours/year.) In general, the retention wage placement standards do not differ 

substantially across the four sites. 
 Evaluation of DSYODP–II Final Report 

72 McNeil Education, Training, and Research 

Table VII.10. Retention Wage Placement Standards (updated 6/06)

City Number Of 

Dependents 

Annual Income 

Level *Retention Wage 

Chicago 

0 $15,745 $7.57/hr 

1 $18,824 $9.05/hr 

2 $21,902 $10.53/hr 

3 $24,980 $12.01/hr 

4 $28,059 $13.49/hr 

5 $31,137 $14.97/hr 

Washington, D.C. 

0 $15,267 $7.34/hr 

1 $18,345 $8.82/hr 

2 $21,424 $10.30/hr 

3 $24,502 $11.78/hr 

4 $27,580 $13.26/hr 

5 $30,659 $14.74/hr 

Los Angeles 

0 $16,036 $7.71/hr 

1 $19,116 $9.19/hr 

2 $22,196 $10.67/hr 

3 $25,276 $12.15/hr 

4 $28,356 $13.63/hr 

5 $31,436 $15.11/hr 

New York 

0 $16,307 $7.84/hr 

1 $19,385 $9.32/hr 

2 $22,464 $10.80/hr 

3 $25,542 $12.28/hr 

4 $28,620 $13.76/hr 

5 $31,699 $15.24/hr. 

*Divide the annual income by 2,080 hours/year to obtain retention wages/hour. 

Annual Income Levels are rounded down. 

A total of 511 youth achieved employment through the four Centers (regardless of whether or 

not they qualified as placements.) The most children any employed youth had was three. The 

retention wage level increases considerably as the number of dependents increases. This can 

have an impact on whether any particular individual gaining employment will be considered “a 

placement” at a retention wage level. The vast majority of participants do not have dependents, 

and for those who do, in most cases there is one child. Thus, the majority of participants were 

classified as placements if they gained employment at the lowest wage category, since they 

typically did not have children. 

Table VII.11 shows the distribution of youth employed as placements and non-placements. The 

gray areas indicate wage categories that were too low for employed participants to have qualified 
Chapter VII–Employment Outcomes 

McNeil Education, Training, and Research 73

as placements. In order for participants to be labeled as placements in the MIS, they would have 

passed several approval processes at the Center, including meeting eligibility requirements, 

verification of employment, verification of wages in relation to the number of dependents, and 

verification by the QCA of completed paperwork. 

Washington, D.C. had four youth employed below retention wages (non-placements) and 32 

youth employed in New York were non-placements. Three youth in Chicago seemed to qualify 

as placements but were not labeled as such. Los Angeles had five non-placements in addition to 

20 youth that may have qualified as placements with respect to their wage earnings but were not 

labeled as placements. This leaves a total of 44 youth who might have qualified as placements 

according to their wage category and number of children but were not labeled as such for reasons 

unknown. 

Table VII.11. Number of Employed Youth (Placements and Non-Placements) 

by Retention Wage Category, by Number of Children

Sites No. of 

Children Retention Wage Category 

Chicago 

less than 

7.57 

7.57 - 

9.04 

9.05 - 

10.52 

10.53 - 

12.00 

12.01 - 

13.48 

13.49 - 

14.96 

14.97 and 

up 

0 2 72 17 11 0 0 5 

1 0 0 5 7 1 0 1 

2 0 1 0 2 0 0 0 

Total 2 73 22 20 1 0 6 124 

Washington, D.C. 

less than 

7.34 7.34-8.81 8.82 -10.29 

10.30-

11.77 

11.78-

13.25 

13.26-

14.73 

14.74 and 

up 

0 1 25 16 4 1 0 1 

1 0 3 0 3 1 0 0 

2 0 0 0 1 0 0 0 

Total 1 28 16 8 2 0 1 56 

Los Angeles 

less than 

7.71 7.71-9.18 9.19-10.66 

10.67-

12.14 

12.15-

13.62 

13.63-

15.10 

15.11 and 

up 

0 1 33 24 1 2 1 0 

1 0 3 9 4 0 0 0 

2 0 0 0 0 1 0 0 

3 0 1 0 0 0 0 0 

Total 1 37 33 5 3 1 0 80 

New York 

less than 

7.84 7.84 -9.31 9.32 -

10.79 

10.80 -

12.27 

12.28-

13.75 

13.76-

15.23 

15.24 and 

up 

0 25 86 76 28 0 0 1 

1 0 1 14 9 0 0 1 

2 0 0 5 3 0 0 1 

3 0 0 1 0 0 0 0 

Total 25 87 96 40 0 0 3 251 

Note: Three children was the maximum number of children among all youth employed by BI. 

Table VII.12 shows the number of participants who were labeled as placements in the MIS 

database. The gray areas indicate wage categories that were too low for employed participants to 

have qualified as placements. 
 Evaluation of DSYODP–II Final Report 

74 McNeil Education, Training, and Research 

Table VII.12. Number of Placements in Each Retention Wage Category 

by Number of Children

Sites No. of 

Children Retention Wage Category 

Chicago 

less than 

7.57 

7.57 - 

9.04 

9.05 - 

10.52 

10.53 - 

12.00 

12.01 - 

13.48 

13.49 - 

14.96 

14.97 and 

up 

0 0 69 17 11 0 0 5 

1 0 0 3 7 1 0 1 

2 0 0 0 2 0 0 0 

Total 0 69 20 20 1 0 6 116 

Washington, D.C. 

less than 

7.34 7.34-8.81 8.82 -

10.29 

10.30-

11.77 

11.78-

13.25 

13.26-

14.73 

14.74 and 

up 

0 0 25 16 4 1 0 1 

1 0 1 0 3 1 0 0 

2 0 0 0 1 0 0 0 

Total 0 26 16 8 2 0 1 53 

Los Angeles 

less than 

7.71 7.71-9.18 9.19-10.66 10.67-12.14 

12.15-

13.62 

13.63-

15.10 

15.11 and 

up 

0 0 27 13 0 2 0 0 

1 0 1 9 3 0 0 0 

2 0 0 0 0 1 0 0 

Total 0 28 22 3 3 0 0 56 

NY 

less than 

7.84 7.84 -9.31 9.32 -10.79 

10.80 -

12.27 

12.28-

13.75 

13.76-

15.23 

15.24 and 

up 

0 1 80 56 26 0 0 1 

1 0 0 10 8 0 0 1 

2 0 0 2 2 0 0 1 

Total 1 80 68 36 0 0 3 188 

Note: Two children was the maximum number of children among placements. 

Overall, most placements were labeled appropriately. Only five placements were employed at 

wage levels that should not have qualified them as such in the data. Note that no placements had 

more than two children. Also note that youth with fewer children tended to fall into the lower 

wage categories. 

Table VII.13 shows the average wages for 98 employed youth that were not considered 

placements. As expected, the average wages for these youth were lower than those for 

placements. 
Chapter VII–Employment Outcomes 

McNeil Education, Training, and Research 75

Table VII.13. Number of Participants Employed but Not at Retention Wage Levels 

 Number Percent Average Wage Std Dev Min Max 

Chicago 8 8.2% $8.63 $1.15 $7.00 $10.00 

Washington, D.C. 3 3.1% $7.50 $1.09 $6.25 $8.25 

Los Angeles 24 24.5% $9.45 $1.56 $7.25 $15.00 

New York 63 64.3% $8.99 $1.40 $6.75 $12.00 

Total 98 100.0% $9.03 $1.44 $6.25 $15.00 

E. Retention and Termination among Placements 

Retention was achieved when youth remained with the same employer for at least 6 months. The 

goal of the DSYODP project was a 90% retention rate for placements. Termination from 

employment was defined as voluntary, involuntary or layoffs from employment. In the event that 

youth were terminated, employers were asked to report the reason for their termination. 

Data on retention and termination are based on the ability of Centers to track youth over time. 

For Phase II of the project, none of the Centers was able to maintain full staffing levels according 

to the model. Due to the absence of a RS, three of the four Centers reported that participants 

were only tracked intermittingly and for reasons other than to capture retention, such as when 

BRCs were anticipating a youth’s promotion to retention wage. New York is the exception in 

that it hired and maintained the same RS throughout the entire course of Phase II. Washington, 

D.C. and Los Angeles utilized other staff members to perform RS duties in addition to their other 

job responsibilities. The result of this staffing shortage is that retention and termination data on 

placements are inconsistent across the four sites. 

New York and Los Angeles also reported wage increases for more than 20 participants. In the 

majority of these cases, the youth began employment at wages below retention wage levels and 

later became re-categorized as placements once they received the raise in pay. Chicago and Los 

Angeles also recorded a few changes in wages in cases where youth were reportedly terminated 

from employment at a wage different from their wage at the start of employment. 

Table VII.14 shows the number of placements, reported terminations of placements and number 

of weeks employed before termination. The following totals are subject to interpretation with 

caution due to staffing shortages in Centers. Also, termination counts only include those youth 

terminated up through December 31, 2006. Overall, it’s likely that there were more terminations 

than reported across the sites. 
 Evaluation of DSYODP–II Final Report 

76 McNeil Education, Training, and Research 

Table VII.14. Number of Placements, Terminated Placements, and Weeks Employed 

Before Termination*

Placements Terminations 

Reported 

Termination 

Rate 

Avg. Number 

of Weeks 

Employed Before 

Termination 

Std Dev Min Max 

Chicago 116 2 1.7% 1.32 0.57 0.92 1.72 

Washington, 

D.C. 53 22 41.5% 8.89 7.33 0.12 22.4 

Los Angeles 56 15 26.8% 7.78 5.99 0 18.4 

New York 188 43 22.9% 7.67 6.7 0.12 22.4 

Totals 413 82 19.9% 7.32 6.39 0 22.4 

*For placements terminated voluntarily, involuntarily or laid off within 6 months of employment start date. 

Placed youth who were recorded as terminated from employment within six months stayed 

employed for an average of 7.32 weeks. This is a slightly shorter period than the average of 9.35 

weeks reported during Phase I of the project and may reflect the impact of RSs being present 

during Phase I. These averages are not very telling, however, since there was considerable 

variation among youth within each site with respect to how long they remained employed before 

termination. 

In terms of the types of termination for those reported by the Centers, there were roughly equal 

proportions of voluntary and involuntary terminations within and across each of the four sites 

(data not shown). Unlike Phase I, where layoffs were the primary reason for termination, there 

were no reported layoffs among placed youth in Phase II. The top five reasons for termination of 

the 82 placements reported were lateness or attendance problems (18.3%), poor performance 

(18.3%), disapproval of job or pay (14.6%), other job opportunities (9.8%), and personal, homerelated or school conflicts with the job (9.8%). These reasons do not differ markedly from the 

reasons for turnover given by recruited businesses before their involvement with BI (see Table 

IV.9). 

 

DSYODP Phase II did not last long enough to determine if youth retained employment beyond 

the six-month period. Many youth would not have been placed long enough to gather some 

information on their persistence and wages. Phase I youth, however, were placed a sufficient 

amount of time before our data gathering to provide some information on their job retention and 

wages. The next chapter describes the process and outcomes of analyzing the UI wage data for 

Phase I youth in the two states for which UI wage data became available. 
Chapter VIII–Further Analysis - UI Wage Comparison Study 

McNeil Education, Training, and Research 77

Chapter VIII 

FURTHER ANALYSIS - UI WAGE COMPARISON STUDY 

In this chapter, the argument is tested that working with an intermediary, like BI, affects wages 

and length of employment for youth offenders and youth at-risk of court involvement. This is 

done by utilizing and linking MIS and UI wage data for youth placed in employment and youth 

who were eligible but not placed in employment by BI during the first phase of DSYODP. 

Descriptive analyses were used to compare the characteristics and wages of the youth prior to 

and during their involvement with the Centers. The assumptions and constraints involved in 

modeling such a comparison with the data available are discussed and use OLS regression to 

look at the effects of program involvement on employment outcomes, controlling for 

demographic differences in the youth. 

A. Hypotheses 

Based on the goals of this evaluation McNeil ETR worked with DOL/ETA in developing a series 

of hypotheses to frame the evaluation (see DSYODP-II Evaluation Design Plan, Section III, pgs. 

12-16). Among the primary research questions presented in the Evaluation Design Plan for 

DSYODP-II was: “Does the use of BI’s services reduce unemployment spells and otherwise 

improve employment outcomes for youth ex-offenders and at-risk youth?” Specifically, the 

intermediary model’s impact on outcomes for employment, earnings, and retention of youth 

participants is evaluated. This can be phrased in terms of the following hypotheses: 

Hypothesis #1: Youth placed by the Centers have higher wages than similar youth who 

gained employment through other means. 

The descriptive analysis of employment outcomes presented in Chapter VIII suggested that the 

Centers were successful in identifying employers with jobs that paid retention wages. Youth 

enrolled in the project had access to the services of BI staff and the referring organizations that 

sent them. While youth with similar characteristics to placements may gain access to 

employment, it is expected that the retention wage standard of the intermediary model will yield 

significant differences in terms of wages for youth placed by BI. 

Hypothesis #2: Youth with higher average wages are associated with greater persistence in 

employment (job tenure). 

There are several assumptions underlying the concept of a “retention wage” as defined by BI. 

Basic to the idea is that higher wages ensure that youth stay longer at that place of employment. 

In other words, wages should be high enough such that jobs encourage rates of retention among 

the youth that are comparable to the norm for those particular jobs. In the Final Report for 

DSYODP-I, the relationship between wage rates and length of employment for youth prior to 

termination was discussed. Data from the MIS for DSYODP-I suggested a positive relationship 
 Evaluation of DSYODP–II Final Report 

78 McNeil Education, Training, and Research 

for youth placed by the Centers. Data also suggested that rates of termination and reasons for 

termination among placed youth in unskilled and skilled jobs were similar to the rates reported 

by employers in general. Given the lack of a basis for how firms determined their retention rates, 

evaluators could only speculate on the significance of this similarity for youth in the project. The 

combination of MIS data and UI wage records for youth participants in DSYODP-I allows for 

testing aspects of the assumption that higher wages, particularly wages among placed youth, 

affect job tenure and persistence in employment. In the next section, the data, methods and 

measures used to test this relationship are described. 

B. Data, Methods & Measures 

Chapter II of this report described the process by which the four BI Centers collected participant, 

business and referring organization data in the MIS and how the evaluators went about obtaining 

both the MIS data and UI wage records (or administrative data) for this analysis. Here, some of 

the qualities of MIS and UI wage records with respect to the following comparison study are 

discussed. 

Data 

Since the early 1980s, state agencies have worked with university researchers through programs 

like the Administrative Data Research and Evaluation (ADARE) program. In these partnerships, 

researchers utilize UI wage records to analyze labor market trends and manage the performance 

of public workforce development initiatives through program evaluations and tests of program 

effectiveness. UI wage data are still largely underutilized by the broader research community, 

particularly for conducting multi-state studies where data have to be requested by following 

protocols unique to each state of interest for the study (King and Schexnader, 1998). Researchers 

have noted the benefits of linking UI wage records data with other data sources to explore issues 

such as changes in wages and job tenure over time, industry expansion and contraction, and 

mobility of labor (see Hammida, 2004; Stevens and Simonetta, 2004). In this comparison study 

of DSYODP youth, evaluators were able to link UI wage records to participants in the MIS data 

using individual social security numbers for participants in the project. This allowed for the 

exploration of relationships between individual characteristics of youth and employment 

outcomes. 

Of the four sites in this study solicited for UI wage records, only Florida and Maryland provided 

DOL/ETA with the approval for McNeil ETR to gain access to their data for the purposes of 

evaluating DSYODP. Maryland and Florida provided McNeil records matched to youth enrolled 

in Phase I. Data from Florida were composed of employee-level data containing the 

employee/participant’s last name, social security number, employer, employer ID number, 

quarterly wages, and year/quarter in which each record of wage data was collected. Maryland 

provided data on youth placed by the Washington, D.C. business resource center. These data 

contained the social security number, employer’s UI identification number, the 

employee/participant’s quarterly wages, and the year/quarter in which each record of wage data 

was collected. Table VIII.1 shows the quarters of UI wage data matched to youth enrolled and 

placed during Phase I from the MIS. 
Chapter VIII–Further Analysis - UI Wage Comparison Study 

McNeil Education, Training, and Research 79

Table VIII.1. Quarters of UI Wage Data Requested and 

Number of Enrolled and Placed Youth from Phase I by Quarter and UI Wage State 

Year 2002 2003 2004 2005 2006 

Qtr 4th 1st 2nd 3rd 4th 1st 2nd 3rd 4th 1st 2nd 3rd 4th 1st Tot 

Phase I Quarters requested* 

Pre Pre Prg Prg Prg Prg Prg Prg Prg Prg Prg Post Post Post 14 

Number of Youth Enrolled with UI wage records from Maryland and Florida 

49 105 48 60 38 27 4 4 5 340 

Number of Placements in Phase I 

Total 2 45 114 58 101 104 74 49 59 10 616 

Washington 

D.C. 19 15 25 33 19 5 3 2 121 

Miami, FL 2 34 47 28 49 16 6 182 

Number of Washington, D.C. placements with UI records from Maryland 

13 11 16 20 18 4 3 2 87 

Number of Miami, FL placements with UI records from Florida 

2 33 44 28 46 16 6 175 

*Abbreviations are defined as follows: Pre: the period prior to the beginning of that Phase; Prg: the period of 

program operation of that Phase; Post: the period after the Phase (grant) has been completed 

Note the three distinct periods in the data with a possibility of two quarters for each participant 

prior to entry in DSYODP and a minimum of three after exiting the program. The two quarters 

prior to any individual’s enrollment would be their “pre” period for the sake of analysis. The first 

“program” quarter for Phase I would be the second quarter of 2003. The distinction between 

“program” and “post” quarters is a function of the length of operation for the project during 

Phase I, not of the quarters that any individual participant may have been recording UI wages. 

For example, in Phase I, participant A could have begun employment in the 2nd quarter of 2003 

and have 12 quarters of program and post data through the 1st quarter of 2006, whereas 

participant B could have begun employment in the last quarter of Phase I operation (2nd quarter 

of 2005) and only have a total of 4 quarters of program and post data combined. The maximum 

amount of quarters any youth can have in the data is 14, ranging from the fourth quarter of 2002 

through the first quarter of 2006. The maximum amount of quarters any placement could have in 

the data is thirteen, from first quarter 2003 through first quarter 2006. 

Placement activity for Washington, D.C. lagged slightly behind that of Miami, Florida during 

Phase I. The lag is also reflected in the UI wage records from Maryland and Florida. Note that 

not all youth placed in employment by the Washington, D.C. and Miami Centers were included 

in the UI wage records obtained from Maryland and Florida. The difference is, of course, more 

drastic for Washington, D.C. given the likelihood of youth being placed in other nearby 

metropolitan areas. More of the strengths and weaknesses of UI wage data records with respect 

to this study are discussed below. 
 Evaluation of DSYODP–II Final Report 

80 McNeil Education, Training, and Research 

Strengths and Weaknesses of UI Wage Data in this Study 

Like any data set, UI wage data has unique characteristics along with strengths and weaknesses 

(see King and Schexnader (1998) and Stevens and Simonetta (2004)). Stevens and Simonetta 

(2004) summarize the following characteristics of UI wage records: 

• All UI wage record files are continuous since beginning year/quarter. 

• All UI wage record files are routinely updated quarterly within four months of the end 

of the most recent reference quarter. 

• All UI wage record files are maintained by employee social security number. 

• Each quarterly wage record includes a dollar amount and the reporting employer’s 

state UI tax account number. 

• UI wage record information is confidential. The identity of employees and employers 

cannot be disclosed to the public. 

Source: The Administrative Data Research and Evaluation Project (ADARE), Background and Opportunities to 

Partner. Stevens, David W. and Jonathan Simonetta. 2004. 

Enough data were provided to control for youth holding multiple jobs in a given quarter. 

Changes in employers for youth were indicated as duplicate records for individuals containing 

repeated year/quarters with different wages in the UI wage records. Repeated quarters of data 

could also mean that employers changed ownership and received a different employment number 

in the UI wage records. According to King and Schexnader (1998), both are feasible scenarios 

for consideration in the use of administrative data. 

For this study, Florida provided the names of employers such that they could be matched for 

consistency with employer identification numbers. Florida data did not show any changes in 

ownership throughout the data collection period. Maryland did not provide additional 

information on employers to match with employer numbers but it is assumed that any changes in 

employer numbers resulting from changes in ownership are likely to be few. 

UI wage data do not include specific information on industries in which the youth were 

employed although Florida did provide information about employers that could be useful for 

more detailed analyses. Because of the lack of industry data, industry differences in employment 

outcomes for youth participants could not be controlled for. 

SESAs do not typically maintain data on the number of weeks worked within a quarter or the 

number of hours worked per week; thus, an aggregate amount for earnings in a quarter masks 

other information such as whether the individual worked full-time and for the full quarter. In 

addition, some participants may have accepted initial employment with employers who pay 

unreported wages, employment outside the state in which the Center operated (e.g., Virginia for 

participants from the Washington, D.C. Center) or they may have moved to another state during 

or after the period of the demonstration (whether in the group of participants placed through 

DSYODP or in the comparison group), and their earnings may not be fully counted. 

While other administrative data sets may contain data on DSYODP participants in Phase I, UI 

wage records provided us with accurate and unbiased data that was cost-effective for the 
Chapter VIII–Further Analysis - UI Wage Comparison Study 

McNeil Education, Training, and Research 81

purposes of this evaluation. A primary advantage of UI wage records provided was that actual 

wages earned by participants were given that covered periods of employment before, during, and 

after each youth’s participation in DSYODP-I. Given that BI staffs in Phase I monitored 

employment outcomes for a period of six months after an individual’s employment start date, UI 

wage data covered the initial six-month follow-up period for participants. 

Methods 

As discussed in the Evaluation Design Plan for DSYODP-II, there were a number of factors to 

take into account in determining which methods were feasible for conducting a comparative 

analysis of outcomes for DSYODP participants. After considering several options, a quasiexperimental study of outcomes using a comparison between placed and non-placed participants 

from Phase I of DSYODP was proposed. The initial study group was to encompass 616 

participants from DSYODP Phase I who were placed in employment at retention wages and 269 

participants referred to each of the four Centers who were determined eligible for DSYODP but 

were not placed in employment through the efforts of the Centers. Because McNeil ETR only 

received UI wage data from Florida and Maryland, the study was limited to participants from the 

Washington, D.C. and Miami, Florida Centers. As Table VIII.1 above shows, this yielded 340 

youth enrolled with UI wage records for inclusion in the study. 

It is recognized that with any partially-experimental comparison study, questions can be raised 

about the degree to which the control group is truly comparable to the group receiving the 

“treatment” offered through the program being evaluated. In many respects, the proposed 

comparison group seems to offer a particularly appealing means to addressing those questions 

since it contains the same type of participants as those who actually received the treatment; if it 

is assumed the principal treatment was their eligibility and enrollment in DSYODP-I. 

As described earlier in Chapter II, Ordinary Least Squares (OLS) multiple regression was used to 

model the effects of DSYODP involvement on youth outcomes. Preparation for the analysis 

entailed compiling comma-delimited (csv) files in SAS and merging data sets through a series of 

sequential iterations. Demographic and geographic characteristics of each youth were maintained 

throughout each data run for descriptive comparisons and use in the regression models. 

The assumptions underlying the regression models are as follows: 

- the pre-program characteristics of all eligible participants are similar (minimal selection 

bias), 

- wages for all youth were similar prior to enrollment in the program (average wages are 

similar for both groups) and approximate average wages in the population of youth atrisk and youth offenders (reduces the undesirable tendency of regression toward the 

population mean), 

- youth who were employed but not considered placements did not have outcomes as 

successful as youth who were placed at retention wages (the portion of participants not 

included in this analysis), and 

- any youth with characteristics similar to participants in DSYODP will have outcomes 

similar to DSYODP participants if enrolled in an intermediary program like DSYODP. 
 Evaluation of DSYODP–II Final Report 

82 McNeil Education, Training, and Research 

Measures 

The employment periods of interest were calculated based on youths’ Employment Start Dates 

and Eligibility (Enrollment) Start Dates in the MIS data. These dates were converted into 

year/quarters and matched to the UI quarters for the purposes of combining UI wages with 

participants in the MIS. Wages within each quarter were summed noting the frequency of 

employers in any given quarter for each youth. These data at the year/quarter level were then 

summed to the level of individual participants taking into account the number of quarters of UI 

data for each youth, the number of pre- and post-enrollment quarters for each youth, and the 

number of pre- and post-placement quarters for each youth placed by BI. 

Many youth reportedly placed by the partnership did not have UI wage records that corresponded 

with the quarter in which they were placed. These youth were employed for such a brief period 

of time that employers may not have reported the amount of wages. 

Dependent Variables 

Log (ln) of Average Weekly UI Wages After Employment and After Enrollment: Quarterly 

aggregations are more suited to the reporting schedules used by employers or for macro-analyses 

of broad labor market trends. For the sake of this study, UI wages were calculated as the natural 

log of weekly averages with the idea that earnings data for individual employees is conceptually 

more meaningful as smaller increments of time. The income variable was logged to generate 

units that are more normally distributed thus providing a better fit of the model to the data. 

UI wage data pose one basic limitation on our ability to analyze individual wages: the duration of 

time under analysis is short relative to the quarterly aggregation periods provided in UI wage 

records. If looking at outcomes over a longer period of time, quarterly data might be less of an 

issue. Given that DSYODP is expected to have an impact on youth’s wages, the “bulkiness” of 

UI records is accounted for by counting quarters containing youth’s enrollment and employment 

start dates as part of the period prior to participation in DSYODP. This allows us to keep the data 

from these quarters for analysis, rather than dropping them from the study. It also provides more 

credibility to any significant findings since the higher wages assumed to be associated with 

enrollment and employment start dates are stacked against our hypotheses. 

Proportion of Enrollment Quarters with UI wage Data Records: Calculated as the number of 

quarters beginning with the quarter of enrollment (containing the participant’s 

Eligibility/Enrollment Start Date) divided by the total number of reported UI wage quarters for 

each youth from enrollment forward, this variable attempts to capture youth’s persistence in 

employment. The fewer the number of quarters from the date the youth were enrolled, the greater 

the chances that the ratio will approximate 1. 

There were Eligibility/Enrollment Start Dates in the MIS database that were dated after the youth 

began employment according to the Employment Start Date. This was an unanticipated outcome 

resulting from BI’s incentive program, which rewarded points to each Center for improvements 

in its placement rates, or the proportion of youth employed out of the total number enrolled at the 

Center. This phenomenon made for an interesting conundrum in the calculation of time periods. 
Chapter VIII–Further Analysis - UI Wage Comparison Study 

McNeil Education, Training, and Research 83

Fortunately, it did not affect the analyses to a great extent other than close proximities in the 

calculation of average wages from time of youths’ enrollment and employment going forward. 

Another measure of job tenure was created using a ratio of the total number of employers over 

the total number of enrollment quarters with UI wage records. The goal was to account for 

changes in employers in the definition of tenure. No significant relationships resulted from the 

use of this measure and it was omitted from the analyses. 

Independent Variables 

The independent variables used in the OLS regression models include both continuous and 

categorical variables. Continuous variables include the log number of employers in each youth’s 

UI wage records and the log of average weekly wages for modeling wage effects on job tenure. 

Categorical variables include the following with their respective coding: 

• Gender (male=0, female=1), 

• Race/ethnicity (African-American=0, Non-African-American=1), 

• Education level (high school completion=0, GED completion=1, out-ofschool=2,GED preparation=3, in college=4, vocational certification=5, college 

graduate=6, and in high school=7),

• Marital status (single=0, married=1), 

• Offender classification (at-risk=0, offender=1), 

• Presence of children (youth without children=0, with children=1), 

• State (Maryland=0, Florida=1) for modeling effects on all placed youth, and 

• Placement category (placements=0, non-placed youth=1). 

Each of the categorical variables was transformed into dummy variables using the coding 

scheme above whereby the largest groups (i.e. males, African-Americans, high school graduates, 

single participants, at-risk youth, and youth placements) represented the reference category 

(equal to zero) against which its alternatives (equal to one or more) were compared. 

Table VIII.2 summarizes the characteristics of youth at Washington, D.C. and Miami with UI 

wage records in Maryland and Florida according to the variables used in the models. 

Participants who were placed and those who were eligible but not employed by BI (nonplacements) were similarly distributed in terms of age (not shown), gender, ethnicity, educational 

attainment, marital status, offender classification, and number of children. Youth were balanced 

in terms of gender across the two sites, and largely African-American in both cities. Miami did 

have a substantial proportion of Hispanic youth participants placed in employment. 

The majority of youth in either city were also high school graduates with notable numbers of 

youth either preparing for or having completed their GED. Youth at both sites also contained a 

fair amount of college or vocational training, particularly among placed youth in Maryland. 
 Evaluation of DSYODP–II Final Report 

84 McNeil Education, Training, and Research 

Table VIII.2. Characteristics of Youth Enrolled at Washington, D.C. and Miami Centers: 

Placements and Non-Placed Youth with Maryland and Florida UI Wage Records 

Washington, D.C. Center Youth in 

Maryland UI Records (N=157) 

Miami Center Youth in Florida UI 

Records (N=183) 

Placed at 

Retention Wages

Eligible but not 

Employed by 

DSYODP 

Placed at 

Retention Wages 

Eligible but not 

Employed by 

DSYODP 

N 87 70 175 8 

Gender 

 Female 36 (41.4%) 33 (47.1%) 93 (53.1%) 4 

 Male 51 (58.6%) 37 (52.9%) 82 (46.9%) 4 

Ethnicity 

 African-American 83 (95.4%) 69 (98.6%) 137 (78.3%) 7 

 Asian-Amer / Pacific Islander 0 0 1 0 

 Caucasian 1 0 4 0 

 Hispanic / Latino/a 3 1 33 (18.9%) 1 

Education of Participants Placed 

 High School Graduate 38 (43.7%) 30 (42.9%) 110 (62.9%) 3 

 GED Completion 9 8 22 1 

 In College 8 2 13 2 

 Out-of-School/ No Degree 6 12 14 1 

 GED Preparation 10 11 11 0 

 Vocational Graduate 10 3 3 0 

 College Graduate 4 2 2 0 

 In High School 2 2 0 1 

Marital Status 

 Married 5 1 7 0 

 Single 82 (94.3%) 69 (98.6%) 168 (96%) 8 

Classification 

 Youth Offender 16 (18.4%) 26 (37.1%) 38 (21.7%) 1 

 At-Risk 71 (81.6%) 44 (62.9%) 137 (78.3%) 7 

 

Reason for eligibility if “At-Risk” 

 Poverty 66 (77.0%) 41 (58.6%) 136 (77.7%) 7 

 Other 9 9 4 1 

 
Chapter VIII–Further Analysis - UI Wage Comparison Study 

McNeil Education, Training, and Research 85

Table VIII.2. Characteristics of Youth Enrolled at Washington, D.C. and Miami Centers:

Placements and Non-Placed Youth with Maryland and Florida UI Wage Records (cont’d)

Washington, D.C. Center Youth in 

Maryland UI Records 

Miami Center Youth in Florida UI 

Records 

Placed at 

Retention Wages

Eligible but not 

Employed by 

DSYODP 

Placed at 

Retention Wages 

Eligible but not 

Employed by 

DSYODP 

N 87 70 175 8 

Number of Children: 

0 58 (66.7%) 32 (45.7%) 158 (90.3%) 8 

1 24 23 16 0 

2 5 10 1 0 

3 0 4 0 0 

4 0 1 0 0 

Number of Participants 

Employed at Time of Referral 

 0 0 0 1 

Number of Employers for each 

youth in UI wage dataa 

 Average 

 (Std Deviation) 

5.56 

(5.41) 

4.27 

(3.46) 

11.45 

(5.58) 

12.63 

(4.24) 

 Median 3 3 12 13.5 

 Min/Max 1 - 26 1 - 13 1 - 34 6 - 18 

Overall Weekly UI Wagesb ($) 

 Average 

 (Std Deviation) 

181.89 

(162.88) 

121.81 

(114.42) 

233.33 

(148.77) 

277.60 

(142.50) 

 Median 140 88.35 222.99 227.98 

 Min/Max 7.83 – 790.13 1.50 – 503.04 9.58 – 869.88 143.04 - 549.17 

 by Number of Children: 

0 182.38 

(164.62) 

97.10 

(80.27) 

224.63 

(138.25) 

277.60 

(142.50) 

1 169.55 

(149.46) 

159.36 

(118.95) 

307.31 

(218.21) -- 

2 235.44 

(226.40) 

87.60 

(134.11) 

424.94 

(na) -- 

3 -- 168.97 

(224.75) -- -- 

4 -- 201.97 

(na) -- -- 

a. The number of employers is determined by the number of employer ID numbers reported in UI wage records for each youth. 

b. Average weekly wages were calculated from quarterly UI wage data and only based on the number of quarters for which 

each youth had UI data. Thus, quarters with no wage data are not included in these calculations. 

Youth in the comparison were mostly single and at-risk of court involvement although eligible 

youth in Maryland who were not placed had a larger proportion of youth offenders relative to at-
 Evaluation of DSYODP–II Final Report 

86 McNeil Education, Training, and Research 

risk youth. Poverty was the predominant reason for eligibility among the youth and most of the 

placed youth had fewer children than non-placed youth in the sample. 

Only one of the youth in the comparison sample was employed at the time of referral to BI. 

Youth in Florida generally had more employers throughout their UI wage records relative to 

youth in Maryland irrespective of whether or not they were placements. Most interesting are the 

differences in UI wages for both groups, particularly those participants in Maryland. Placed 

youth in Maryland seemed to have higher median wages on average relative to non-placed youth. 

Data also suggest that youth with children earned higher average wages as their number of 

children increased regardless of whether they were placed or not by BI. 

In sum, these data provide a good picture of the similarities between comparison groups and the 

individual factors that are likely to play a part in differences in outcomes within and between 

sites. The small number of non-placed participants from the Miami Center suggests that youth in 

Maryland make for a better comparison group for a case study of placement vs. non-placement 

outcomes. 

Employment and Wages 

Regional differences in wage levels might be expected given the differences in cost of living 

between regions. The discussion above also pointed to differences in the length of employment 

for youth between sites during the time period of data collected. Figure VIII.1 shows the 

distribution of UI quarters provided for youth participants in Washington, D.C. and Miami, 

Florida. 

Youth placed in Florida had more quarters of employment data during the 14 periods under 

observation. Placements made in Maryland had fewer quarters of UI wage records. The same is 

true for youth who were not employed by the D.C. Center but who obtained employment by 

other means at some point during the period. These quarters do not equate to continuous 

employment, because youth changed jobs often within quarters and UI wage records were not 

always consecutive from one quarter to the next. 
Chapter VIII–Further Analysis - UI Wage Comparison Study 

McNeil Education, Training, and Research 87

0

5

10

15

20

25

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Number of Quarters Employed

Florida Placements Florida Non-DSYODP Employed

Maryland Placements Maryland Non-DSYODP Employed

Figure VIII.1. Number of Youth by Number of Quarters Employed 

Based on UI Wage Records 

Table VIII.3 provides a comparison of the average number of UI wage quarters for DSYODP-I 

youth obtained from the UI wage records. Not only did youth in Florida have more employers, 

but also more quarters of employment relative to youth in Maryland records. This suggests that 

youth placed in Florida were more likely to have been employed longer periods of time. The data 

are not as suggestive with respect to differences in the number of quarters between placements 

and non-placements. Youth placed in Maryland had more wage-earning quarters on average than 

their eligible counterparts but the median numbers of quarters are the same for these two groups. 

There were strong correlations for youth in the UI records between offender status and average 

weekly wages (-0.3), between the number of employers and average weekly wages for youth 

(0.52), and between placement status and the number of employers per quarter (-0.32). (A 

Pearson correlation matrix for Maryland and Florida youth in the sample is presented in 

Appendix D). This suggests that frequent changes in employment correspond with differences in 

wages between placed and non-placed youth and that non-placed youth may have been less 

mobile. Number of Youth 
 Evaluation of DSYODP–II Final Report 

88 McNeil Education, Training, and Research 

Table VIII.3. Quarters of UI Wage Data by State and Placement Status 

Washington, D.C. Center Youth in 

Maryland UI Records 

Miami Center Youth in Florida UI 

Records 

Placed at 

Retention Wages

Eligible but not 

Employed by 

DSYODP 

Placed at 

Retention Wages 

Eligible but not 

Employed by 

DSYODP 

N 87 70 175 8 

Number of Quarters in UI 

Records 

 Average 4.51 3.84 8.99 9.63 

 Median 3 3 9 9.5 

 Std Deviation 3.51 2.94 3.5 3.11 

 Minimum - Maximum 1 - 14 1 - 11 1 - 14 4 - 14 

One of the underlying assumptions in the study was that youth referred to BI were entry-level job 

candidates. If this were the case, very few of the youth enrolled in DSYODP to have consistent 

records of earnings prior to their involvement with the project. Table VIII.4 represents a 

comparison of UI wage record quarters and quarters associated with the employment start date 

for placed youth from Washington, D.C. and Florida. 

Table VIII.4. Comparison of UI Wage Quarters Relative to Employment Start Date for 

Placed Youth 

Number of Youth 

Washington, D.C. Center 

Youth in Maryland UI 

Records 

Miami Center Youth in 

Florida UI Records 

 Placed in Employment by DSYODP 87 175 

 with Two Consecutive Quarters of UI Wage 

 Data Prior to Employment Start Date 

23 

(26.4%) 

75 

(42.9%) 

 with Two Consecutive Quarters of UI Wage 

 Data After Employment Start Date 

60 

(69%) 

171 

(97.7%) 

with Employment Start Date contained in First 

Quarter of UI Wage Data Provided 

17 

(19.5%) 

34 

(19.4%) 

 where Employment Start Date is not contained 

in any UI wage quarter 

49 

(56.3%) 

44 

(25.1%) 

A relatively small number of youth have consecutive quarters of UI wage data prior to placement 

by DSYODP. Most youth in the sample were not employed at all or employed intermittently 

prior to employment through DSYODP. A larger proportion of youth placements did have 

consecutive quarters of UI wage records following placement by BI. This does not entail that 

they stayed with the same employer, but that they were more actively employed after their 

involvement with DSYODP. 

For about 19% of youth placed at either site, their first quarter of UI wage records corresponded 

with the quarter containing their employment start date. About 56% of youth in Washington, 

D.C. and 25% of youth in Florida lacked a quarter for UI wage data that corresponds to the 

quarter in which they were employed. This suggests several things: (1) that they did not have 
Chapter VIII–Further Analysis - UI Wage Comparison Study 

McNeil Education, Training, and Research 89

enough documented hours of employment in that quarter to warrant a UI wage record, and (2) 

that youth in Florida could have stayed employed longer after their placement. 

Considering that none of the placed youth in this UI wage analysis was reported as being 

employed at the time of enrollment in DSYODP, their average wages for the two consecutive 

quarters prior to involvement with DSYODP are expected to be less than wages in the quarters 

after placement. In some cases, youth only had UI wage records for one quarter prior to or after 

their employment start date. It is expected that quarters containing the employment start date 

have higher average wages than those that do not, all other factors being equal. But this may not 

be the case if youth did not stay employed very long after placement. Table VIII.5 is a 

comparison of average wages for placed youth for the two consecutive quarters prior to and after 

their placement. As expected, the average wages for youth with two consecutive quarters of UI 

wage data are higher after placement versus before placement. 

Table VIII.5. Comparison of Placement Wages for Two Consecutive Quarters 

Prior to Placement and After Placement (Retention Period) 

Two Consecutive 

Quarters before 

Placement* 

Two Consecutive Quarters after 

Placement** 

Washington, D.C. (Maryland UI 

data) Same Youth Total 

 Number of Placed Youth 23 18/23 60 

Average Weekly Wages 

(Std Deviation) 

153.73 

(130.69) 

213.11 

(173.48) 

223.05 

(193.96) 

 Median 88.42 144.19 154.83 

 Min/Max 9.17 – 466.71 6.17 – 558.33 3.33 – 790.13 

Miami (Florida UI data) 

Number of Placed Youth 75 74/75 171 

Average Weekly Wages 

(Std Deviation) 

177.63 

(138.18) 

242.63 

(170.93) 

221.11 

(169.85) 

 Median 131.88 225.60 196.38 

 Min/Max 7.00 – 632.96 8.04 – 688.83 4.67 – 763.33 

*Average wages prior to employment are calculated by summing UI wages for the two quarters up to and/or 

including the employment start date for youth employed by DSYODP. 

** Average wages after employment in DSYODP are calculated from the quarter containing each youth’s 

Employment Start Date forward. 

OLS Regression Models 

Up to this point, the characteristics of youth participants in the comparison sample, the 

similarities and differences in their employment outcomes with respect to the number of 

employers, average wages, and wage differences for placed versus non-placed youth have been 

discussed. All data seem to indicate that the intermediary approach, particularly the placement of 

youth and its emphasis on retention level wages for those youth, has some impact on outcomes 

for youth offenders and youth at-risk of court involvement. What follows is a series of three 

models to test the significance of this assertion: 
 Evaluation of DSYODP–II Final Report 

90 McNeil Education, Training, and Research 

Model #1 tests the effects of individual factors and geographic differences between sites on 

average wages among placements. The model includes data for placed youth in both Maryland 

and Florida records. 

• Model #1: Log (ln) of Average Weekly UI Wages after Employment Start Date = 

 f (youth demographic characteristics, state, and logged number of employers) 

The period of time from youths’ enrollment in DSYODP onward provides a good basis for 

setting up a comparison of outcomes for both placed and non-placed youth because both groups 

are participants in the “treatment.” Table VIII.6 provides a comparison of average wages from 

enrollment date forward for placed youth against average wages for eligible youth who were not 

employed by DSYODP. Data are limited to youth with Maryland UI wage records. 

Table VIII.6. Average UI Wages for Maryland from Enrollment Quarter Forward between 

Placements and Eligible Non-Placed Youth in DSYODP - Washington, D.C. Center 

Placed Youth Enrolled but Not Placed 

by DSYODP 

Number of Placed Youth 73 61 

Average Weekly UI Wages* 

(Std Deviation) 

210.54 

(182.20) 

123.48 

(116.39) 

Median 153.07 87.67 

Min/Max 3.33 – 790.13 1.50 – 485.67 

*Differences in means for placed and non-placed youth are significant at p<0.05 using a two-tailed 

Satterthwaite T-test of unequal variances. T value= -3.35. 

Average weekly UI wages for the two groups from enrollment onward are significantly different 

based on a T-test of independent means. Knowing this, a model of the effect of placement status 

can be constructed, net of individual factors and movement between employers. Model #2 tests 

the hypothesis that placed youth have higher wages than similar youth who gained employment 

through other means, controlling for individual characteristics. Data are based on a comparison 

of placed and non-placed youth in Maryland UI records only. 

• Model #2: Log (ln) of Average Weekly UI Wages after Enrollment = f (youth 

demographic characteristics, logged number of employers, and placement status) 

Based on the preliminary findings, it is expected that placed youth differ from non-placed youth 

with respect to persistence in employment. This difference in job tenure may be associated with 

the differences in wages between the two groups. Model #3 tests the hypothesis that youth with 

higher average wages are associated with greater persistence in employment (job tenure), net of 

individual characteristics and placement status. 

• Model #3: Proportion of Enrollment Quarters Employed = f (youth demographic 

characteristics, placement status, and logged average weekly UI wages) 
Chapter VIII–Further Analysis - UI Wage Comparison Study 

McNeil Education, Training, and Research 91

C. Results

Table VIII.7 shows the regression coefficients and T-values for each of the three models 

described above. 

Table VIII.7. OLS Regression Coefficients and T- Values 

for Measures of Average Wages and Job Tenure 

Model #1 Model #2 Model #3 

Log (ln) Average UI 

Weekly Earnings After 

Employment Start Date 

(For All Placed Youth) 

Log (ln) Average UI 

Weekly Earnings After 

Enrollment in DSYODP 

(For Maryland Youth) 

Proportion of Enrollment

Quarters Employed 

(For Maryland Youth)a

Female 0.083 0.080 0.044 

 (0.73) (0.33) (0.70) 

Non-African-American 0.243 0.478 -0.266* 

 (1.61) (0.81) (-1.79) 

Non High School Graduates -0.048 -0.086 -0.015 

 (-1.53) (-1.65) (-1.11) 

Married 0.09 -0.451 0.26* 

 (0.32) (-0.77) (1.72) 

Offender -0.14 0.117 -0.158** 

 (-1.00) (0.43) (-2.27) 

Youth w/ Children 0.181 0.208 -0.061 

 (1.13) (0.88) (-1.01) 

Florida -0.090 -- -- 

 (-0.60) -- -- 

 

Log (ln) Number of Employers 0.533*** 0.481*** -- 

 (6.88) (3.88) -- 

Youth Not Placed in 

Employment by DSYODP 

-- 

-- 

-0.66*** 

(-2.87) 

-0.058 

(-0.95) 

Log (ln) Average UI Weekly 

Wages After Enrollment 

-- 

-- 

-- 

-- 

0.059*** 

(2.71) 

Adjusted R-squared 0.2059 0.1516 0.1302 

Intercept 4.11*** 

(22.31) 

4.22*** 

(13.84) 

0.306** 

(2.48) 

Number of Participants 246 134 134 

a. Calculated as the number of quarters beginning with enrollment divided by the total number of reported UI wage 

quarters for each youth. 

-- = not applicable.

 *Significantly different from zero at the .10 level, two-tailed test. 

 **Significantly different from zero at the .05 level, two-tailed test. 

****Significantly different from zero at the .01 level, two-tailed test. 

Model #1 shows the regression coefficients for average weekly wages for placed youth in both 

Maryland and Florida. With respect to variations in income among placed youth after their 

employment start date, there were no significant differences between demographic groups. 
 Evaluation of DSYODP–II Final Report 

92 McNeil Education, Training, and Research 

Holding demographic characteristics and geography constant, a 1% increase in the number of 

employers led to a 53% increase in average weekly income for placed youth. This effect 

indirectly captures how average wages increase with employment. The average wages for youth 

after their employment start date were highly correlated (.52) with their total number of 

employers. Youth frequently changed employers such that movement from one employer to the 

next could have resulted in significant increases in their wages. All other things being equal, the 

longer youth stayed employed, the greater their average wages were likely to be relative to youth 

with longer lapses between employment. 

Maryland UI wage data were best suited for comparing differences in average weekly wages 

between placed and non-placed youth after enrollment. Both groups were of comparable size and 

demographic characteristics whereas Florida only contained eight youth who were eligible but 

not placed through participation in DSYODP. Model #2 shows coefficients and T-values for 

average wages from Maryland UI wage records after youth were enrolled in DSYODP-I. 

Holding other factors constant, a 1% increase in the number of employers led to a 48% increase 

in average weekly income for youth participants. Also, average weekly earnings for non-placed 

youth were about 48% (anti-log of -0.66) less than youth placed by BI, holding all other factors 

constant. 

Model #3 shows coefficients and T-values for the proportion of quarters after enrollment during 

which youth were employed. These comparisons are also based on UI wage records from 

Maryland. Coefficients for differences between groups by race, marital status and offender 

classification were significant. For non-African-American youth, the proportion of enrollment 

quarters in which youth had UI wage records would be 0.266 points less than AfricanAmericans, net of other factors. Given that there were only 5 out of the 157 youth with Maryland 

UI wage records who were not African-Americans, little emphasis should be placed on this 

outcome. 

There were also very few married youth, six in total. They faired 0.26 times better with respect to 

job tenure relative to their single counterparts. Not enough youth in the data were married to 

warrant serious consideration of this outcome even though five of the six were also placed in 

employment. 

Youth offenders faired 0.158 times worse than at-risk youth in the proportion of enrollment 

quarters in which youth had UI wage records. This finding is substantive and has implications 

with respect to the focus and application of future interventions like the intermediary model. 

Wages had a significant impact on persistence in employment. Every $100 increase in average 

weekly wages for participants led to an increase of 5.9 times for the proportion of enrollment 

quarters in which youth had UI wage records, holding other factors constant. This suggests that 

youth with higher average wages stayed employed more than youth with lower wages. 

Although, on average, placed youth in Maryland had more quarters of UI wage data during their 

enrollment in DSYODP, the difference in tenure relative to non-placed youth was not great 

enough to be significant. Adding wages to the employment tenure models had a general effect of 

increasing the differences associated with youth’s demographic characteristics and lessening the 
Chapter VIII–Further Analysis - UI Wage Comparison Study 

McNeil Education, Training, and Research 93

differences in outcomes between placements and non-placed youth. (Preliminary nested models 

not shown.) This suggests that, by itself, placing youth in employment is not enough to 

strengthen tenure. Placing youth should be accompanied with higher wages to ensure that youth 

persist in employment. 

D. Discussion and Summary 

In the analyses presented in this chapter, the general argument is tested that youth involvement in 

intermediary programs like BI impacts the employment outcomes for youth offenders and youth 

at-risk of court involvement measured in terms of wages and persistence in employment. The 

analyses are framed in the form of two hypotheses: (1) that youth placed by the Centers have 

higher wages than similar youth who gained employment through other means, and (2) that 

youth with higher average wages are associated with greater persistence in employment. 

Data utilized for this study came from linked MIS participant data for Washington, D.C. and 

Miami and UI wage records for participants in DSYODP-I from SESAs in both Maryland and 

Florida. These data spanned 14 quarters from the fourth quarter of 2002 through the first quarter 

of 2006. UI wage records group wages by quarters. To compensate for this, the expected 

outcomes were stacked in favor of the null hypothesis. This allowed all the individual records to 

be retained while presenting conservative estimates for effects on employment outcomes. 

Given the limited data provided by each of the states, this study may be best viewed as a case 

study for the specific project sites for which data was received. Future studies like this involving 

multi-state UI wage data collection may want to consider incorporating partnerships with statelevel agencies in the design of the evaluation plan to maximize access to administrative records 

within each of the states of interest. 

Combining MIS data and UI wage quarters underscored the difficulty Centers must have faced in 

encouraging youth to stay with an employer for 6 months. UI wage records for the participants 

did not contain consecutive quarters of data for the most part. Quarters in which youth were 

placed by BI were frequently absent from UI wage records for youth. Because of the substantial 

proportion of placed youth (25 to 50%) in the sample without corresponding UI records, one 

could speculate as to how much and in what ways DSYODP actually impacted future wages 

youth were receiving in later quarters of employment. In other words, it could be that the 

unmeasured services and resources, the “casework” and employment readiness workshops 

provided by the intermediary over time, were just as impacting on participant outcomes as an 

initial job placement at retention wages. 

With regard to the results from the regression analyses, several items from the findings are worth 

consideration. Data suggest that the basic assumption was correct in that youth participants 

represented patterns of behavior characteristic of entry-level employees, regardless of their 

offender or at-risk status. These youth were highly mobile when it came to employers and higher 

levels of mobility were associated with higher wages on average. 
 Evaluation of DSYODP–II Final Report 

94 McNeil Education, Training, and Research 

Similar results from data collected during the YODP-III evaluation suggested that youth were 

able to increase wages over time by staying with an employer or by changing employers. In 

YODP-III, the St. Paul project operated an Employment Bonus program that rewarded youth a 

financial bonus of $1,000 for staying with an employer for at least 6 months. Youth were aware 

of employers’ policies for increasing wages based on their performance, timing of raises, and 

tenure in the job position. Some youth stayed at their employer beyond the 6 months needed in 

order to get the $1,000 bonus in DOL/ETA funds. Similarly during DSYODP-I, it’s possible that 

youth were judging whether they would gain wage increases at the same employer or could 

increase their wages by using the experience gained at their current employer at a new place of 

employment. 

The strongest effects on wages were associated with youths’ mobility in terms of the number of 

employers. This outcome runs counter to the importance placed on retention by BI and conceals 

the possibility of better wage outcomes for individuals who stay with the same employer longer 

rather than staying employed via multiple employers. 

This analysis also shows that placed youth faired better than non-placements with respect to 

wages. But the differences were not significant with respect to persistence in employment once 

average weekly wages were factored in. Adding an interaction term that combines average 

weekly wages and placement status to the model might provide a clearer picture of the net effects 

for wages and placement status. 

Youth offenders also faired worse than at-risk youth in this analysis with respect to persistence in 

employment. Perhaps it is no coincidence that Centers in Phase II of DSYODP saw a dramatic 

decline in the number of youth offenders enrolled. Given the presence of an incentive system 

that rewarded Centers for better retention rates with employers, Center staffs may have adapted 

their placement strategy in order to maximize the rewards associated with placing youth who, 

generally, have more favorable employment outcomes. 
Chapter IX–Conclusion 

McNeil Education, Training, and Research 95

Chapter IX 

CONCLUSION 

This evaluation encompasses a wide range of information with respect to Phase II of DSYODP. 

In its chapters, key literature and findings pertaining to employment of youth offenders and atrisk youth and the use of intermediaries for placing candidates in jobs were discussed. The 

evaluation approach used to guide the information gathered for the evaluation for both phases of 

DSYODP was outlined along with methods used to analyze the data collected and focused the 

analysis to answer the primary research questions of interest to DOL/ETA and BI. Using field 

work and MIS data queries, information was described on the characteristics of the Centers, the 

staffing patterns, and utilization of the MIS. The characteristics of businesses and referring 

organizations recruited by the Centers to coordinate the placing of participants in employment 

were described and the characteristics of participants, and the services they received. Both MIS 

data and UI wage records were utilized to explore the employment outcomes for businesses and 

youth participants and deduce significant findings. 

There were two primary sets of research questions of interest to DOL/ETA for the evaluation of 

DSYODP Phase II The first set of questions was: 

• How does the business intermediary approach identify and recruit employers and 

broker job placements for youth ex-offenders and youth at-risk of court or gang 

involvement? How well does BI identify and recruit employers who are willing to 

hire this target population? Does the use of the intermediary satisfy employers’ 

needs? What criteria does BI utilize to hire BRC staff? What is the turnover rate of 

such staff? What are the performance requirements of such staff? Do they receive 

placement bonuses? If so, what are the results? 

Staff at the Centers used a variety of inductive approaches to identify and recruit employers. 

They considered companies that looked for entry-level workers, were hiring and willing to pay 

the local retention wage. Centers used a combination of trade media, such as Crain’s publication. 

They researched local trends within industries to determine areas of growth. They attended and 

marketed their services at career fairs to make new contacts and made use of existing contacts 

with local businesses to generate leads. 

All indicators suggest that staffs’ methods were successful in generating leads for brokering job 

placements. Center staff worked together on a daily basis to plan their approach for securing job 

positions. Once they established rapport with employers through personal contact, Center staffs 

uncovered more than enough leads to job vacancies. Because they were personally vested in the 

mission of BI, they overcame the lack of corporate stability and structural limitations (i.e. lack of 

office equipment, staff turnover, lack of RSs, etc.) on their ability to perform tasks. With respect 

to the model, the more relevant and difficult challenges to staff performance often came in the 

form of their ability to identify an adequate supply of job-ready candidates to meet the demand 

of their recruited businesses. Several employers reported needing large numbers of employees on 

a regular basis and Center staffs struggled to meet these demands. 
 Evaluation of DSYODP–II Final Report 

96 McNeil Education, Training, and Research 

Employers expressed a willingness to adopt the ideas underlying the intermediary approach. 

BRCs were effective at selling their services in a professional manner, which made a strong 

impression on many employers and convinced hesitant employers to give youth a chance to 

succeed. Employers, like many of the RO contacts, expressed a sense of gratification in their 

ability to play a role in servicing the community while satisfying their need to fill vacancies with 

prescreened job candidates that matched the requirements of the position. Prescreening was one 

of the main selling points noted by employers in addition to the free hands-on service provided 

by BI. Small business clients were particularly happy for the free service because they had no 

money for their hiring processes. 

BRC also provided employers with a willingness to negotiate hours, shifts and wages with 

employers so that both partie’s goals were met. Once youth were placed, BRCs followed up with 

the employer to assess if the employer felt their needs were being met. This combination of 

“hands on” services put BI in a position that rivaled temporary placement agencies and other 

staffing services. 

The priority for BRCs was to focus on developing business relationships. Each Center’s BRCs 

were charged with generating 5 placements per month with a 90% retention rate. In order to be 

effective, BRCs had to have specific skills including good listening habits, an assertive and selfmotivated approach to their position, flexible schedules to accommodate the needs of employers, 

access to transportation, professional attire and etiquette, timeliness and consideration of the time 

constraints faced by their business clients. 

BI utilized a point system by which Centers were rewarded or penalized relative to meeting their 

recruitment and placement goals. Center staff also created their own goals for recruitment and 

placement. Despite the corporate incentives offered for meeting recruitment goals for businesses, 

many of the businesses utilized during Phase II were holdovers from the first Phase. Center staff 

members simply found it more cost effective to focus their energies on identifying job ready 

youth for placement rather than spending as much time on expanding their portfolio of 

employers. Placement bonuses were rewarded on occasion. The most notable occasion was the 

third quarter of 2006 when the New York Center placed 95 participants and received a large cash 

reward from BI headquarters as a result. Otherwise, Center staff members rarely indicated 

interest in receiving the monetary incentives. Given the environment characterized by regular 

staff overhauls, many staff expressed more concern over punitive consequences for not 

performing well enough to meet the approval of executive staff members. 

The turnover among BRCs in Phase II was rare. Vacancies tended to result from executive 

interventions in which entire Centers were re-staffed. This was the case in Los Angeles and 

Washington, D.C. to some extent. Staff overhauls at these two Centers slowed the momentum of 

recruitment activity and BRCs lost rapport with employers as a result of abrupt changes in 

personnel. 

The second set of primary research questions posed by DOL/ETA for the evaluation was: 

• How does BI connect youth ex-offenders and at-risk youth to the services of the 

demonstration project and to One-Stop Career Center services and other services 
Chapter IX–Conclusion 

McNeil Education, Training, and Research 97

available in the community? How well does the intermediary connect youth exoffenders and at-risk youth to services? Does the use of BI’s services reduce 

unemployment spells and otherwise improve employment outcomes for youth exoffenders and at-risk youth? Does BI help youth avoid criminal activity and retain 

employment? 

Referring organizations represented the organizational link between BI and job candidates. By 

establishing rapport with contacts in agencies and organizations that trained and catered to exoffenders and at-risk youth populations, BI was able to connect these youth to the services 

offered by the demonstration project. HRCs had this as their primary responsibility. They 

attended career fairs, researched programs at community colleges, government agencies, NGOs 

and the like to identify pools of potential job-ready youth fitting the description of the model. 

HRCs employed cost-effective strategies similar to that of BRCs in that they focused on 

developing rapport with a consistent and reliable source of ROs to supply them with youth for 

prescreening and placement. This often proved difficult for HRCs. Despite the range of services 

offered by ROs that BI worked with, ROs often had their own agendas and priorities. Some 

offered training programs that lacked any emphasis on resume preparation, communication skills 

and other soft skills which ensured that once youth were hired they could stay employed. Staff at 

ROs frequently sent unprepared or favored students to BI to see them obtain employment 

regardless of their job-readiness. 

Referring organizations that referred youth who were placed by BI were pleased at the kind of 

jobs BI found for their referrals and the level of the wages they received. However, few 

traditional referring organizations, that is, youth development agencies and public employment 

services, provided reality-based preparation of entry level workers in this demonstration. There 

were several ROs in the MIS that made numerous referrals but had no or few placements. This 

was another source of frustration expressed by both the HRCs and contacts within the RO. HRCs 

were often the catalyst to help ROs understand the needs and perspective of employers. 

Providing additional services to the ROs and youth (i.e. resume workshops, coaching, etc.) was 

often necessary in order for HRCs to overcome the apparent disconnect between ROs and 

businesses. This has implications for future efforts to improve labor market outcomes for these 

youth since there was a real need for improving work readiness models among existing agencies, 

including One Stop Centers. 

Youth were connected with many services through ROs who provided a wide range of services 

across each of the four cities. These included job-related services, skills training services, job 

placement services, and some personal development services. For those supportive services the 

ROs did not directly provide, they referred youth to other agencies. This included services like 

childcare, housing and health-related services. 

Overall, ROs were pleased with the efforts of BI to place youth in employment. Good 

placements generally reflected well on the RO, and may have counted towards their own goals. 

ROs also appreciated the feedback provided by BI staff, particularly HRCs, about youth’s 

experiences with employment. 
 Evaluation of DSYODP–II Final Report 

98 McNeil Education, Training, and Research 

Evidence from the employment outcomes and UI wage analyses suggest that BI was effective in 

improving employment outcomes for youth ex-offenders and at-risk youth. Youth expressed 

appreciation for the opportunities provided by the program. Most of the youth participants in 

Phase II were at-risk youth rather than offenders and few offenders received placements. In New 

York, for example, few offenders were sent by referral organizations. Whether it is the demandside model or just the limitations of this particular demonstration is not clear with respect to why 

the proportions of youth offenders in the demonstration declined during this phase. Perhaps 

future programs should bias the incentive structure in favor of increasing participation among 

this population rather than rates of placements overall. 

It is difficult to assert that BI’s efforts worked to curtail illicit behavior among youth. There were 

no measures for this outcome built into the evaluation. The basic assumption, however, is that 

job placement was the primary alternative to other preoccupations for youth but there is no 

indication that youth would have engaged in illicit activities had they not been involved in the 

project. 

With respect to retaining employment, youth in Phase II of DSYODP did not have the benefit of 

consistent follow-ups by a RS at each of the sites. Only New York and Washington, D.C. 

maintained this position while other Centers satisficed or simply did not attempt to perform the 

tasks associated with this position. Retention outcomes might have improved to some degree 

with the consistent presence of a full-time RS in all sites. Termination and retention data were 

weak for this Phase as a result of their absence. 

Below are some additional observations from this evaluation: 

• Phase II had a short time period under which each of the Centers was fully operational. 

This made it difficult to evaluate the model. Evaluators had to constantly assess the 

outcomes with respect to influences on the part of the management structure versus 

outcomes inherent to the intermediary model. 

 

• The project had to start over in Phase II and had extensive turnover. Because the 

intermediary model depends heavily on relationship building, these fluctuations diluted 

what BI could have possibly accomplished. 

• For demonstration grants, sustaining the activities of an initiative beyond the period of 

the grant presents one of the major challenges that grantees face. BI’s management did 

not adequately prepare to deal with the issue of sustainability. This factored into staff 

turnover and employment outcomes towards the end of the project as executive staff had 

difficulty filling vacant positions. There were sustainability options that were not 

pursued, but the question remains whether it was a sustainable model as a free-standing 

intermediary or as a component of a work readiness effort. 

• BI did prove that there are jobs for low- and medium-skilled workers at retention wages. 

Every site demonstrated the error of thinking that only minimum wage jobs were 

available for this population. Youth outcomes with respect to retention were not 

dissimilar to trends among entry-level workers in similar jobs. Many industries with high 
Chapter IX–Conclusion 

McNeil Education, Training, and Research 99

demand for entry-level workers expect high turnover so six months retention is a savings 

for them. 

• Even with retention level wages, many workers did not get benefits, and that is a 

limitation for workers with families. This may have factored into the differences in 

placement rates for eligible youth with children. 

• Youth development issues, punctuality, solving transportation and childcare problems, 

completing high school, and acquiring soft skills continue to affect the labor force 

experience of these young workers—even with better paying jobs. 

• Employment Bonus programs of YODP, especially at St. Paul, found that youth 

offenders and at-risk youth took longer to become work-ready than program 

administrators expected. This is consistent with findings from DSYODP, both phases, 

where BI staff found that most youth referred by ROs were not work-ready. This means 

more staff time – whether at an RO or an intermediary like BI – needs to be allocated for 

individual and group activities that strengthen work readiness. 

• A major challenge for St. Paul was finding employers willing to take offenders. Given 

that BI did not place many offenders, not much information is available on the hurdles 

faced at the four BI sites. St. Paul and a couple of the BI sites were able to identify 

businesses who would give offenders and at-risk youth a chance; and the promise of 

careful screening before placement and retention support after placement encouraged 

employers in St. Paul and BI sites to try more of these youth. 

Recommendations 

The lessons of the DSYODP need not be a detriment to trying it with other ways of organizing 

the effort. The model could be integrated into One-Stop Career Centers or other configurations, 

franchises, etc. Does the model for DSYOSP require a central headquarters, or could city Centers 

be independent units? An in-between model might offer specific roles for a central office that 

would recognize its potential efficiencies: (1) develop a standard set of forms and staff job 

descriptions, which was very time-consuming for each site to develop in YODP; (2) develop a 

standard database design that could be operated in Access (again, very time-consuming for 

YODP grantees to each create their own); (3) facilitate information sharing whereby a central 

office facilitates the dissemination of successful practices among the sites. Each site, as a largely 

independent unit, would develop its own plan for recruiting businesses and ROs, set its own 

goals, hire and train its own staff, and be accountable to a local board of directors. There may be 

an additional role for a central office to provide technical assistance to Centers for sustaining 

operations (e.g., grant writing and fundraising) and possibly sharing grant funds obtained by a 

national office. 

A possible “next step” would be to marry the strengths of YODP/EB and DSYODP – 

Employment Bonus provided highly-effective retention support, DSYODP identified businesses 

with lots of openings that paid a decent wage, and both programs recognized the importance of 

getting youth to reach a minimum standard of work readiness before sending youth on any 
 Evaluation of DSYODP–II Final Report 

100 McNeil Education, Training, and Research 

interviews. This last factor is important in two respects: youth who are not well-prepared are 

more likely to get discouraged about job seeking and to be unsuccessful if hired for a job, and 

employers will not look favorably on future referrals if the youth are not well-matched to the job 

and work-ready. 
Chapter IX–Conclusion 

McNeil Education, Training, and Research 101
 Evaluation of DSYODP–II Final Report 

102 McNeil Education, Training, and Research 

REFERENCES 

Bartik, T.J. (2001). Jobs for the poor: can labor demand policies help? New York, NY: Russell 

Sage Foundation. 

Bernstein, L., and Houston (2000). Crime and work: What we can learn from the low-wage labor 

market. Washington, D.C.: Economic Policy Institute. 

Buck, M. (2000). Getting back to work: Employment programs for ex-offenders. New York, NY: 

Public/Private Ventures. Fall. www.ppv.org. 

Bushway, S. and Reuter, P. (1999). Labor markets and crime risk factors. In L.W. Sherman, D. 

Gottfredson, D. MacKenzie, J. Eck, P. Reuter, and S. Bushway (eds.). Preventing Crime: 

What Works, What Doesn’t, What’s Promising. College Park, MD: University of 

Maryland. 1999. 

Business Interface Inc. 2005. Demand-Side Youth Offender Demonstration Project Technical 

Assistance Manual-Volume 2. Unpublished document. 

Chao, E. (2003). Opening remarks at the 2003 Workforce Innovations Conference. Washington, 

D.C.. Retrieved October 9, 2003 from the U.S. Department of Labor Web site: 

http://www.dol.gov/_sec/media/speeches/20030708_workforce_innovations.htm. 

Fleischer, W. (2001). Extending ladders: Findings from the Annie E. Casey Foundation’s Job 

Initiative. Baltimore, MD: Annie E. Casey Foundation. 

Freeman, J. and Taylor, J. (2002). Beyond welfare-to-work: helping low-income workers 

maintain their jobs and advance in the workforce. Boston, MA: Jobs for the Future. 

Retrieved October 7, 2003 from the Jobs for the Future Web site: 

http://www.jff.org/jff/PDFDocuments/BeyondWTWReten.pdf

Furstenberg, F. S., Kennedy, V., McLoyd, et al. (2004). Growing up is harder to do. Contexts, 

Vol.3, Number 3, Summer. 

Hamilton, G. (July, 2002). Moving people from welfare to work: Lessons from the national 

evaluation of welfare-to-work strategies. Washington, D.C.: U.S. Department of Health 

and Human Services and U.S. Department of Education. 

Hammida, Mustapha. 2004. Job mobility and hourly wages: is there a relationship? Monthly 

Labor Review. May 2004. Vol. 127(5):23-30. 

Monthly Labor Review. May 2004. http://findarticles.com/p/articles/mi_m1153/ 

is_5_127/ai_n6155625
References 

McNeil Education, Training, and Research 103

Harer, M. (1994). Recidivism among federal prisoners released in 1987. Washington, D.C.: 

Federal Bureau of Prisons, Office of Research and Evaluation. 

Holzer, H. and Stoll, M. (2000). Employer demand for welfare recipients by race. Washington, 

D.C.: Urban Institute. 

Holzer, H., Raphael, S., and Stoll, M. (2002). Can employers play a more positive role in 

prisoner reentry? A paper prepared for the Urban Institute’s Reentry Roundtable, 

Washington, D.C., on March 20-21, 2002. 

Javar, J. and Wander, S. (2004). The use of service providers and brokers/consultants in 

employment and training programs. In Christopher J. O’Leary, Robert A. Straits and 

Stephen A. Wander (eds.). Job Training Policy in the United States (pp. 135-175). 

Kalamazoo, MI: W.E. Upjohn Institute for Employment Research. 

Jenks, S., MacGillivray, L., and Needels, K. 2006. Youth Offender Demonstration Project 

Evaluation Final Report. McNeil Research and Evaluation. Unpublished report. 

Jenks, S., Robinson, C., Stone-El, R., Vernon, A., and MacGillivray, L. 2006. Process 

Evaluation for the Welfare-to-work Partnership’s U.S. Department of Labor DemandSide Youth Offender Demonstration Project (DSYODP) – Final Report. McNeil Research 

and Evaluation. Unpublished report. 

Jobs for the Future. (January, 1998). Business participation in welfare-to work: lessons from 

the United States. Boston, MA: Jobs for the Future. 

Jobs for the Future. (May, 2002). How to help welfare recipients and other low wage workers 

secure—and keep—better jobs. Boston, MA: Jobs for the Future. 

King, Christopher T. and Deanna T. Schexnager. 1998. The Use of Linked Employer-Employee 

UI Wage Data: Illustrative Uses in Texas Policy Analysis. Ray Marshall Center for the 

Study of Human Resources, The University of Texas at Austin. 

http://www.utexas.edu/research/cshr/pubs/pdf/useoflinkeddata.pdf

Lerman, R. I. (April 2000). Improving career outcomes for youth: Lessons learned from the U.S. 

and OECD Experience. Washington, D.C.: U.S. Department of Labor. 

Lerman, R. I. (November 2000). Are teens in low-income and welfare families working too 

much? Washington, D.C.: The Urban Institute. 

Lynch, L. (2000). Labor market intermediaries for the training of low-wage workers. Paper 

prepared for the Emerging Market Institutions for the 21st Century Conference, NBWE, 

August 4-5, 2000. Cambridge, MA. 
 Evaluation of DSYODP–II Final Report 

104 McNeil Education, Training, and Research 

Mueller, E. and Swartz, A. (2002). Creating change: Pushing workforce systems to help 

participants achieve economic stability and mobility. Evaluation of the Annie E. Casey 

Foundation Jobs Initiative. Cambridge, MA: Abt Associates and New School. 

Mukamal, D. (June 2001). From hard time to full time. Washington, D.C.: U.S. Department of 

Labor Employment and Training Administration, Office of Adult Services, Division of 

Welfare-to-Work. 

Pager, D. (2002). The mark of a criminal record. Paper presented at the annual meeting of the 

American Sociological Association, Chicago, August 16-19. 

Partnership for Employer-Employee Responsive System (PEERS). (November, 2003). 

Workforce intermediaries: generating benefits for employers and workers. PEERS. 

Pavetti, L., et. al. (September, 2001). Changing the culture of the welfare office: The role of 

intermediaries in linking TANF recipients with jobs. FRBNY Economic Policy Review, 

63-76. 

Public/Private Ventures (September, 2002). Serving high-risk youth: Lessons from research and 

programming. New York: Public/Private Ventures. 

Roberts, B and Padden, J. (1998). Welfare to wages: strategies to assist the private sector to 

employ welfare recipients. Flint, MI: Charles Stewart Mott Foundation. 

Stevens, David W. and J. Simonetta. 2004. The Administrative Data Research and Evaluation 

Project (ADARE), Background and Opportunities to Partner. PowerPoint presentation. 

www.ubalt.edu/jfi/adare/presentations/HUD%20stevens%20ppt%203-3-04.ppt

Taylor, J. and Rubin, J. (June, 2005). Engaging employers to benefit low-income job seekers: 

lessons from the job initiative. Boston, MA: Jobs for the Future. 

W. K. Kellogg Foundation. (2001). Logic model development guide. Battle Creek, MI: Author. 
References 

McNeil Education, Training, and Research 105
 Evaluation of DSYODP–II Final Report 

106 McNeil Education, Training, and Research 

APPENDICES 
A-1 

APPENDIX A 

Review of Prior Research
 

A-2 
A-3 

APPENDIX A – REVIEW OF PRIOR RESEARCH

In this section of the appendices, selected areas of prior research are presented to inform our 

understanding of the context of the DSYODP. The principal areas are: 

 A. Employment of Youth Offenders and Youth At-Risk of Court Involvement 

 B. Demand-side Approaches and the Use of Intermediaries 

 C. Evaluation of Workforce Intermediaries 

A. Employment of Youth Offenders and Youth At-Risk of Court Involvement 

1. Programs as Prevention 

While most of the discussion of employment and youth offenders concerns what happens after 

the period of confinement, a broader perspective would take into account efforts to prevent youth 

from engaging in activities that result in involvement with the criminal justice system. Since this 

is a very expansive issue, this segment reviews the major evaluations and studies of programs 

that have attempted to address the relationship between crime and employment among adults and 

youths. 

Based on their review of previous evaluations, Bushway and Reuter (1999) assessed the merits 

of various employment programs. They looked at both supply-side programs oriented toward the 

youth themselves and demand-side programs targeted to employers. In terms of supply-side 

programs for job training and education for at-risk youth, the authors identified three types of 

programs: (1) provision of summer work or other forms of subsidized employment, (2) shortterm training with job placement for out-of-school youth, and (3) long-term, intensive residential 

programs providing vocational and life skills training, general education and job placement (e.g., 

Job Corps). In reviewing studies of these types of programs, they found that short-term programs 

did not provide enough time to measure and/or overcome a failed academic career that finds 

individuals so far behind they cannot catch up quickly, and that employment by itself is not 

enough to stop crime. 

Bushway and Reuter found that the most effective programs were Job Corps and other intensive 

long-term programs that deal with all aspects of treating at-risk youths. The authors concluded 

that the real, long-term answer for the vast majority of at-risk youth lies not with the after-thefact job training but rather with an effort to make schooling more meaningful in order to avoid 

drop-outs. As an example, they cited the Quantum Opportunities Program, which was a 

demonstration project that offered extensive academic assistance, adult mentoring, career and 

college guidance, a small stipend and money set aside for a college fund. They also noted that 

the most important part of this program was the adult mentoring; apparently the mentors 

provided the necessary focus and motivation for students to change their behavior and perform 

better in school. 

On the demand-side, Bushway and Reuter looked at three types of programs: bonding and wage 

supplements, enterprise zones, and Community Development Block Grants. They found that 
 

A-4 

enterprise zones seem to have some positive effects in creating jobs for the target population and 

are most promising when designed with a crime prevention objective in mind. 

2. Barriers to Employment 

The challenges of youth ex-offenders and at-risk youth in seeking employment opportunities are 

certainly well-documented, and consequently in this segment a brief overview of some of the key 

findings is presented. 

• Furstenberg, et al. (2004) reported that it takes much longer to attain the kind of 

employment that pays enough for young people to support themselves because the 

labor market demand for skills is more rigorous than in previous years, and it 

requires more years of schooling and part-time jobs to be prepared for the 

available full-time positions that pay benefits. In the weak economy, 

Public/Private Ventures (2002) added that an economic downturn means that 

youth are competing for scarce jobs with adults with good employment records. 

• Lerman (November 2000) reported that a substantial proportion of adolescent 

Americans lack the basic skills that all employers require and that the United 

States has a weak system connecting education and careers. These factors are 

exacerbated for vulnerable youth, those who come from neighborhoods with high 

unemployment and social dysfunction. In a separate paper, Lerman (April 2000)

reported that disadvantaged youth are not only disproportionately less successful 

in school, but they were also less likely to gain employment, even during the 

unprecedented economic boom of the 1980s and 1990s. He listed among the 

factors limiting access to jobs: high illiteracy rates among 16-25 year old 

American youth (25% compared to 5% in Germany and 10% in Canada) and the 

higher incidence of economically disadvantaged and minority youth with criminal 

records. 

• Bernstein and Houston (2000) gathered research to show that released prisoners 

have a lower prospect for secure employment and decent wages throughout their 

lifetimes. Upon return to the community, former prisoners face a number of 

significant barriers to securing employment, particularly employment outside of 

the low-wage sector. 

Several other studies reinforce findings on the extent of the challenges facing this population: 

• Employers are more reluctant to hire former prisoners than any other group of 

disadvantaged workers. (Holzer et al. 2002); 

• Job applicants with a criminal record are substantially less likely to be hired. 

According to a recent audit, the likelihood of getting hired was 40 percent lower for 

the applicant with a criminal record. (Pager 2002);

• Individuals with felony convictions are statutorily barred from many jobs. The list of 

employment bans has increased over the past decade. (Mukamal 2001); 
A-5 

• The availability of criminal records online, and changing public policies regarding 

access to those records, make it easier for employers to conduct criminal background 

checks on potential employees. (Holzer et al. 2002);

• The kinds of jobs for which employers have historically been more willing to hire 

individuals who were formerly incarcerated—blue collar and manufacturing jobs—

are diminishing in the national economy. (Holzer et al. 2002). 

3. Overcoming Barriers 

Despite what seems to be a discouraging litany of barriers to employment for these youth, the 

experiences of the Youth Offender Demonstration Projects and DSYODP projects indicate that a 

significant number of youth, especially those beyond school-age and thus available for full-time 

employment, have been successful in gaining and retaining employment with the assistance of 

demonstration projects throughout the country. Two sources that reinforce what evaluation of the 

demonstrations has found are highlighted below. 

• Research conducted by Harer (1994) focused less on evaluating the success of 

programs and more on evaluating the success of former offenders. This research 

reported that offenders who had worked or been in school were less likely to reoffend. Those who had jobs before leaving incarceration were also less likely to 

re-offend. 

• The third round of the Youth Offender Demonstration Projects provided 

substantial evidence that youth offenders and youth at-risk of court involvement 

could obtain employment as participants in programs that provided case 

management and comprehensive support services (Jenks, MacGillivray, and 

Needels, 2006). For example, ninety percent of those youth who were active or 

had completed their individual service plans achieved a placement in 

employment. Further, youth offenders were as likely to obtain employment as 

non-offenders, but their wages were significantly less. Two projects that received 

special grants for employment bonus incentives and retention support reported 

very high persistence rates, with those youth placed in full-time employment 

persisting for the six months required for receipt of the full $1000 bonus, resulting 

in retention rates averaging over 90 percent for the two programs combined. 

• “Serving High-Risk Youth: Lessons from Research and Programming” 

(Public/Private Ventures 2002) echoed prior youth employment literature that 

found that “nothing works.” The author’s concern was, however, that few 

programs had been rigorously evaluated. The paper focused on ten that had been 

subject to rigorous evaluation, and still concluded that few had produced enduring 

impacts for their clients. Yet site-by-site assessments led to a less discouraging 

conclusion because some projects did have lasting impacts—even when other 

pilots or demonstration projects were not considered successful. The authors 

observed the difficulty in implementing such complex projects and reviewed a list 

of practices that they believe make a difference: 
 

A-6 

o A desire by participants to change their lives, 

o Prompting from a trusted adult, 

o An opportunity to earn income, 

o Supportive staff-youth relationships, 

o A sense of belonging, and 

o Getting needs identified by the youth met. 

B. Demand-side Approaches and the Use of Intermediaries 

1. Demand-side Approaches 

Secretary of Labor Elaine Chao, in her opening remarks at the 2003 Workforce Innovations 

Conference, co-sponsored by the U.S. Department of Labor and the National Association of 

Workforce Boards, called for a “demand-driven workforce investment system” that responds 

rapidly to changing economic conditions and helps workers gain greater self-sufficiency (Chao 

2003). She said that workforce development programs need to “better fit the hiring needs of 

employers so trainees can have a better chance of securing jobs.” 

Hamilton (2002) used data from the National Evaluation of Welfare-to-Work Strategies 

(NEWWS). This data-based study examined the long-term effects on welfare recipients and their 

children in eleven mandatory welfare-to-work programs. Their conclusions about what worked 

best in successfully leaving public assistance for employment are interesting. 

They first identified three strategies used by the programs: 

• Short-term job search assistance that emphasized getting a job quickly (known as 

Labor Force Attachment, or LFA programs); 

• Longer-term skill-building activities (mostly finishing basic education) that should be 

completed before entering the labor market (known as Human Capital Development, 

or HCD programs); and 

• A mix of the two. 

Overall, they found that all programs, regardless of which strategy was used, increased earnings 

for participants and decreased welfare receipt. However, the short-term programs or LFA’s were 

far more successful than the longer-term strategies that focused on education and training first. In 

other words, between the two, it was far more beneficial to focus on securing employment as 

soon as possible. Nevertheless, the most successful programs in this study combined both 

strategies and far out-performed the other programs. Why did this work? 

It worked because its focus was on employment first and foremost. But underlying that emphasis 

were the notions that short-term education and training and on-going education and training were 

important components to securing and retaining jobs. Moreover, these mixed programs also 

emphasized holding out for a “good” job—not just the first job that was offered. This increased 

earnings and retention as well. 
A-7 

Bartik (2001), in his analysis of the potential role of demand-side policies in increasing the 

employment and earnings of the poor, argued that U.S. antipoverty policy would be more 

effective if it made greater use of labor demand policies. Using empirical evidence, he asserted 

that labor supply policies alone have significant limitations resulting in only modest effects in 

helping low-income Americans gain employment. Labor demand policies are needed because 

even when overall U.S. unemployment is low, jobs are in short supply for the most 

disadvantaged. He outlined a labor demand program with two main components: (1) wage 

subsidies to employers, particularly small businesses and nonprofits, who increase their overall 

employment; and (2) a focus on those who are outside the labor force or otherwise unlikely to be 

employed because of education, race, and/or residence in high-unemployment cities. 

In an examination of the determinants of employer demand for welfare recipients, Holzer and 

Stoll (2000) found a labor market for welfare recipients quite strong, but one that could be 

significantly weakened during an economic downturn. They cited a broad range of factors, 

including skill needs and type of industry, which affect the potential demand for welfare 

recipients among employers. Other characteristics that affect the relative supply of welfare 

recipients to employers—such as spatial location and employer use of local agencies or welfareto-work programs—influence the extent to which such demand is realized in actual hiring. The 

authors suggested that interventions by local agencies aimed at improving the accessibility of 

employers to welfare recipients could broaden and thereby improve their employment prospects. 

Current workforce development programs have largely been shaped by the “work-first” approach 

that emphasizes rapid job placement. Yet research on employers suggests a need to reassess the 

work-first model (Roberts and Padden 1998). Businesses are unwilling to assume the burden of 

preparing the unprepared or unmotivated for work, expecting government to invest in basic 

human resource development. Creating a more effective welfare-to-work system requires 

developing relationships with the business sector, and doing so in ways that fundamentally 

restructure the public policy process to better understand the perceptions, capabilities and 

limitations of business involvement. Potential advantages of making business more central to the 

process can accrue to all partners—recipients, public agencies, business and training partners—

involved in workforce development. 

Fleischer (2001) discussed the importance of focusing on sectoral employment strategies. 

Targeting a specific industrial sector that is important within a certain geographical area has 

definite advantages, especially if the industrial sector has a difficult time recruiting workers due 

to difficult training objectives. Fleisher recommends the following to make a sectoral 

employment strategy more effective: 

• Design a training curriculum that targets a certain sector that has employment needs in 

the area. 

• Hire the training graduates targeted to this sector. Ideally, an employer should promise to 

hire participants before they enroll in training, thus giving an added incentive for 

employer and participants to make training a success. 

• Establish a mentor program. This helps new hires and training graduates adjust to the 

realities of employment in this industrial sector. 
 

A-8 

• Create employee documents, orientations and shop-floor training. By becoming this 

involved with work sites, program staff can learn what the employer really needs and 

wants—beyond just what they say they want. Often employers will make a list of basic 

requirements, but forget underlying skills or tangential skills—for instance, the need for 

someone who has basic math skills or is highly literate. 

Moreover, Fleischer notes the pivotal importance of soft skills—skills such as interpersonal 

communication, ability to take direction, customer service, etc. These skills are more important 

than technical skills when it comes to success in the workplace. Because so many of these 

disadvantaged workers have had so little exposure to workplaces or the relevant workplace 

cultures, they need training on the importance of timeliness, dress codes and social manners 

among other skills. A lack of soft skills is one of the most important reasons that employment 

does not continue. On the other hand, Fleischer notes that employers must also broaden their 

expectations and have the communication skills to work with those that come from different 

backgrounds. 

Demand-driven workforce development acknowledges that employer demand for certain types of 

workers (those that increase productivity or profitability) significantly drives the labor market. 

As Freeman and Taylor (2002) maintained in their report on demand-led retention, workers 

who succeed are those who can meet employers’ demands, and those lacking the necessary skills 

or attributes are in danger of losing out. The authors identified four key characteristics of 

demand-driven workforce development: 

• Dual-customer focus. 1) Underemployed job seekers and 2) the employers that will 

hire them. Both needs have to be met. 

• New partnerships and working relationships between employment and training 

providers and local employers. These relationships incorporate employer input into 

program design, curriculum development and program operation. 

• Stimulation of collective action by employers. Demand-driven systems should be 

responsible to, and influenced by, clusters of firms with similar workforce needs. 

• Coordination of the workforce development system with economic priorities.

Developing a skilled workforce supply can help a region attract businesses. 

By the nature of their mission, structure and location, many social service program providers are 

removed from the business world. They are generally unfamiliar with business or industry 

policies and practices, and they lack concrete information about the companies or occupations of 

interest to their clients (Freeman and Taylor 2002). For these organizations, demand-driven 

workforce development is a serious challenge. The authors recommended several strategies for 

engaging employers: 

• Offer concrete benefits to the employer. 

• Supervisors are essential—don’t ignore them. Many providers reach out to upper 

management and are greeted with enthusiasm, only to be met with hostility by directline supervisors. Directly addressing supervisor concerns is essential. 

• Involve employers in program design and implementation. 
A-9 

• Recruit employers from among existing contacts. 

• Recruit from civic organizations (such as Chamber of Commerce or Rotary Club). 

2. Welfare-to-Work Approach 

The $3 billion Welfare-to-Work grants program established by Congress as part of the Balanced 

Budget Act of 1997 provided funds to over 700 state and local grantees. Approximately $2 

billion was awarded in formula grants to the states. In addition, the Department of Labor 

awarded $712 million in three rounds of competitive grant funding. 

DOL/ETA used similar criteria to evaluate applicants in all three rounds of funding. Its 

solicitation for Welfare-to-Work Round three indicated that the following criteria would be used: 

(1) the relative need for assistance in the area proposed to be served; (2) the extent to which the 

project proposed innovative strategies for moving welfare recipients into lasting work; (3) the 

quality of the proposed outcomes of the project; (4) the degree to which the project was 

coordinated with other services; and (5) the demonstrated capacity of the grant applicant. 

Applicants that agreed to participate in a random assignment evaluation and applicants that 

proposed to serve Empowerment Zones or Enterprise Communities were given special 

consideration. 

Title VII of the Consolidated Appropriations Act for FY2000 amended the Welfare-to-Work 

program expanding the population eligible for services under the Welfare-to-Work program. 

Under these amendments, the Welfare-to-Work program would continue to serve long-term 

welfare recipients, but up to 30% of funds could be spent on individuals in other groups, which 

included: (1) welfare recipients with significant barriers to self-sufficiency as defined by their 

local Workforce Investment Board (WIB); (2) youths between 18 and 24 years of age who were 

in foster care before their 18th birthday; and (3) custodial parents with incomes below the poverty 

line who were not recipients of assistance under their state Temporary Assistance to Needy 

Families (TANF) program. 

The Department of Health and Human Services contracted for an evaluation of the Welfare-toWork Program with the goal, as defined in the grant solicitation, to “expand the base of 

knowledge about programs aimed at moving the least job-ready welfare recipients into 

unsubsidized employment.” The process and implementation analysis component of the national 

evaluation was based on information collected through two rounds of site visits in 1999 and 2001 

to 11 study sites, and MIS data maintained by the programs in participants and services. 

Highlights included: 

• Three general program models for delivering services to the hard-to-employ were 

implemented in the study sites: (1) Enhanced Direct Employment Models, (2) 

Developmental/Transitional Employment Models, and (3) Intensive PostEmployment Skills Development Models. 

• Grantees focused on the most disadvantaged, as specified in Congressionally 

established provisions, but most programs faced difficulties enrolling eligible 

individuals. 
 

A-10 

• Welfare-to-Work programs went beyond job readiness and self-directed job search 

assistance in the sense that they provided intensive individualized case management, 

coaching or support; and many programs also included more intensive developmental 

components and activities. 

• Programs generally provided some type of post-employment services, primarily to 

help individuals retain their jobs. 

• Grantees reported that about half of their program participants entered regular 

unsubsidized employment. 

• Welfare-to-Work participants followed four different “pathways to employment”: (1) 

The Basic Employment Preparation or Work First Model, (2) The Transitional 

Employment Pathway, (3) The Educational or Training Pathway, and (4) The Mixed 

Activities Model. 

3. Intermediaries 

For purposes of the evaluation of the DSYODP, intermediaries, like BI, are entities that link 

individuals with jobs. The scope of the activities entailed in that “link” varies greatly across 

intermediaries, and can include: job search, work experience, education, training, subsidized 

employment, placement, retention support, and advancement support. Intermediaries can be nonprofit or for-profit organizations, educational institutions, or public or quasi-public organizations. 

Pavetti, et al. (2001) offer a two-pronged definition of an intermediary. An organization 

functions as an intermediary when a) it provides services that help link welfare recipients with 

jobs and b) it has a formal relationship with the welfare office or other administrative entity that 

has the responsibility of moving welfare recipients to the job market. Intermediaries can also be 

categorized as either primary intermediaries that bear the primary burden of connecting 

employers and welfare offices and secondary intermediaries. These secondary intermediaries are 

often specialized organizations that may focus on specific types of training or work with specific 

industries. Many types of organizations can function as intermediaries: for-profit, non-profit, 

educational or public organizations can all fulfill the intermediary function. Interestingly enough, 

however, for-profit organizations, such as staffing agencies, comprise the smallest percentage of 

intermediaries, but serve the largest number of welfare recipients. 

Given the limitations of an approach that focuses primarily on the supply-side of persons seeking 

jobs, increasingly, both employers and advocates for low-income job seekers are looking to a 

range of intermediary organizations to make the connections. Such organizations can be 

community-based organizations, employer-led organizations and associations, labor unions, 

community colleges, the public sector, or temporary help and staffing firms. Whatever their 

institutional form, they perform a number of general functions (Jobs for the Future 1998), 

including: 

• Organizing and representing employers in workforce development planning and 

policy; 

• Serving as brokers between employers and local players in workforce development, 

including educational institutions and social service agencies; and 
A-11 

• Delivering services directly to employers and their workforce (rather than acting as a 

broker that connects and refers employers to community-based providers). 

In a subsequent study for Jobs for the Future, Taylor and Rubin (2005) examined data on the 

nine-year Jobs Initiative (JI) to investigate two areas: (1) the types of employers that will do 

business with workforce intermediaries and employ low-income workers and what factors 

determine this, and (2) employer attitudes towards workforce intermediaries themselves. The 

data come from over 65 employers across the U.S. and range from small to large employers. 

Fundamental findings covered six topic areas: 

• Screening: The JI intermediaries benefited their participants by screening 

employers and they benefited employers by screening job applicants. Employers 

valued intermediaries for a distinctive role: reducing the risk of hiring JI 

participants. Because JI participants were viewed as risky, employers felt that 

intermediaries reduced the risk to an acceptable level. 

• Training I: Employers worried that low-income individuals represented 

significant business risks and they believed soft skills training was a solution. 

Data of JI participants bore this out: participation in soft skills training was the 

main predictor for short-term labor market retention of JI participants. 

• Training II: Technical skills were essential for longer-term retention and 

advancement. Employer data supported the perception that technical skills 

training was essential for low-income people, but it also suggested that the impact 

of training did not become evident until about six months into the job. In other 

words, soft skills counted most for surviving the first few months of the new job, 

but technical skills were essential for long-term retention and advancement. 

• Support: A large number of JI employers implemented practices to support their 

entry-level workers. Supportive practices ranged from ad-hoc offerings (e.g. 

transportation subsidies) to comprehensive career ladders that included in-house 

training. The immediate intent was to stabilize and even upgrade the skills of 

internal workforces whose reliability, experience or skills fell short. 

• Motivation: Two factors, acting in combination, shaped employers’ development 

of supportive practices: a desire to benefit the larger community by hiring people 

who needed help and recognition that the “bottom line” constrained the extent of 

the support the firm could provide. 

• Improvement: Employers valued the advice of trusted intermediaries on how they 

could strengthen or expand their supportive practices. The JI intermediaries 

recommended changes to support practices that would both benefit firms and help 

workers stay on the job and advance. Intermediaries often supplied technical 

assistance on how to implement those changes. 
 

A-12 

Pavetti et. al. (2001) note that for an administrative entity, there are important questions to 

consider when choosing an intermediary. Focusing on these issues should provide for a more 

effective and efficacious relationship. The questions are: 

• How much responsibility should be transferred to the intermediary? For instance, 

should case management as well as job placement responsibilities be transferred? 

• Is this responsibility transferred to one or multiple intermediaries? 

• How are the intermediaries reimbursed and how much are they reimbursed? Is costreimbursement or pay-for-performance a better choice? 

The exact process of how intermediaries interact with welfare participants, employers and 

administrative entities also has significant variations and ranges from simple to complex. The 

models examine how many referrals occur in the movement from welfare-to-work. Four major 

models are detailed: 

1. Welfare office case management model: 

 welfare office → other service provider or intermediary 

In this model, the welfare office keeps the case management aspect of the process, but 

outsources job search and placement. 

2. Intermediary case management model: 

 welfare office → intermediary → intermediary or other service provider 

This has the welfare office outsourcing case management as well as all job searching and 

placement. 

3. Workforce development system case management model: 

welfare office → workforce development system → intermediary 

The third model focuses on the use of a workforce development system, which then employs an 

intermediary. 

4. Workforce development system progressive model: 

 welfare office → workforce development system → intermediary 1 (assessment) → 

intermediary 2 (case management, job search) → intermediary 3 (placement). 

This model focuses on using specialized intermediaries for assessments, case management and 

job searching and placement. Clearly, the most involved of the models may also offer, however, 

the most targeted services by the most qualified providers. 

At its best, workforce intermediaries take a dual customer approach, addressing the needs of 

employers and low-income and less-skilled workers and job seekers. In short, workforce 
A-13 

intermediaries act as a “center of gravity” in developing relationships, enforcing accountability, 

and leveraging resources. 

C. Evaluation of workforce intermediaries 

Mueller and Schwartz (2002), using a case-study approach, looked at the Annie E. Casey 

Foundation’s Jobs Initiative demonstration project to help low-income residents find jobs that 

pay family-supporting wages. The project began in 1995 and was to run for eight years. The 

general goal of the initiative was to be a catalyst for systems change, primarily by bringing 

together employers, elected officials, community-based organizations, low-income residents and 

other stakeholders in the design and creation of new job-related strategies. Although the initiative 

resulted in only modest changes grounded in fragile relationships, the authors emphasized that 

the reforms reflect an increased understanding of the complex barriers to work, acknowledge that 

employers should be engaged stakeholders, addressed post-placement supports for workers and 

employers to enable workers to persist and advance, and were pushing systems toward new 

policy goals. 

 

The experience of existing intermediaries suggests several key implementation challenges and 

opportunities (Jobs for the Future, 1998): 

• Targeting priority employers or sectors in a community; 

• Sorting out long-standing and complex interrelationships among organizations, 

institutions, and individuals in a community, and conducting effective outreach; 

• Deciding on short- and long-term goals and activities that yield maximum benefits to 

employers, job seekers, and the community; 

• Matching organizational and/or community capacity to proposed workforce 

development activities; 

• Developing viable funding strategies over the long-term, from general membership 

fees, to fee-for-service activities, to public sector and foundation grants; and 

• Developing and agreeing on common benchmarks to ensure that intermediary 

initiatives undertaken will lead toward desired results (i.e., job placement rates, 

credentials awarded, postsecondary completion rates, and long-term retention in an 

employment track). 

Lynch (2000) identified two major challenges in her study of the role of labor market 

intermediaries in the training of low-wage and low-skill workers. First is the capacity of 

emerging intermediaries to achieve a stable source of funding. Many programs examined in her 

study were sponsored by venture capital money or private foundation support, with the hope that 

once the programs were up and running, alternative sources of funding would be identified. A 

second issue is employer involvement. While it is critical to ensure employer involvement, there 

is also tension about how much of a role employers should play in governing intermediaries. 
 

A-14 

The study by Javar and Wander (2004) noted important differences between intermediaries 

depending on their function and size. They clearly identified larger intermediary organizations, 

whether for-profit or non-profit, as the most effective intermediaries. This is due simply to their 

larger pool of resources and full-time staff and their ability to offer more specialization. 

Additionally, non-profit organizations are more commonly chosen as the intermediary 

organizations for adult and youth services and have a strong presence in the Once-Stop Career 

Centers. 

But what may be most interesting is the competition to be named intermediaries for youth 

service. Contrary to the overall trends (large organizations, mostly non-profit), youth services are 

almost exclusively dominated by smaller community-based organizations. For-profit entities 

almost never compete for youth services—there tends to be either too much competition or not 

enough profit within this arena. 

A study conducted by the Partnership for Employer-Employee Responsive System (PEERS, 

2003) on the benefits to employers and workers of workforce intermediaries provides several 

valuable findings. First, the study offered a list of types of intermediary functions, which 

supplies a useful classification system: 

o Obtain new workers, 

o Plan and/or provide training, 

o Measure and/or redesign jobs, 

o Obtain labor market information, 

o Identify or obtain public funding, 

o Identify, compare and/or broker among vendors, and 

o Assist with child care and transportation. 

 The PEERS study also reported four major findings: 

• All workforce intermediaries were not created equal. Simply using an 

intermediary, regardless of the purpose for which the intermediary was 

used, did not have a significant effect on either productivity or wages. 

Different kinds of intermediary functions had different effects on wages 

and productivity. 

• Using a workforce intermediary to measure and/or redesign jobs led to 

higher productivity and higher wages, benefiting both employers and 

employees. 

• Using a workforce intermediary to plan and/or provide training reduced 

labor turnover and that led to higher productivity and higher wages. 

• Using a workforce intermediary only to obtain new workers led to lower 

wages, to the detriment of workers and, possibly, to employers. 

The fundamental finding in the PEERS study offers an interesting perspective on the use of 

intermediaries: 
A-15 

When employers use workforce intermediaries to improve or link jobs or to 

locate or provide skills training and employment supports, both workers 

and employers benefit. However, when employers use workforce 

intermediaries only for placement and/or job matching, workers’ wage 

prospects can be hindered and whether or not any benefit accrues to 

employers is unclear (emphasis in original). (p. 4) 

In summary, the substantial body of research on youth employment, youth involvement in the 

justice system, and the roles of employers and intermediaries provide evaluators with a strong 

base of understanding the goals of the DSYODP. Any evaluation of such programs must address 

their complexity by looking closely at the stakeholders involved and the various components of 

the model. 
 

A-16 
B-1 

APPENDIX B 

Study Questions 
 

B-2 
B-3 

APPENDIX B – STUDY QUESTIONS

A. Development of a Network of Employers

1. What types of approaches have BI’s Centers used to identify and recruit employers and broker job 

placements for youth ex-offenders and youth at-risk of court or gang involvement? 

2. Which approaches at each Center have been most effective identifying and recruiting employers 

who are willing to hire this target population? 

3. To what extent are employers willing to pay a “retention wage” to DSYODP participants? What 

challenges do Center staff encounter in identifying employers with positions that pay a retention 

wage and what strategies have been effective in gaining a retention wage for as many participants 

as feasible? 

4. To what extent has each Center worked with employers to involve them in development and ongoing operation of BI’s business intermediary model as implemented in each city? 

5. Over the period of the demonstration, to what extent has each Center been able to expand the pool 

of employers who have hired this target population? 

6. What is the full range of employers that has been contacted by each Center? What types of 

industries are represented among those contacted? 

7. What types of industries are represented among those employers that have vacancies listed with 

the Center compared to those that actually hire referred youth? 

8. To what extent are high-growth industries represented? 

9. What is the number of placements, by employer and by industry? What is the number and percent 

of repeat placements where after hiring from BI an employer returns to fill more vacancies? 

B. Employer Outcomes

1. To what extent are employers’ needs for workers being met by the Center in each city? What types 

of businesses are not having their needs met? What is the number and percent of each Business 

Clients’ vacancies that are being filled by referrals from the Center (“fill rate”)? Has the Center 

been able to make adjustments over time to meet those businesses’ needs for workers? Have 

Business Clients indicated that they expect to have vacancies in the future that might require new 

skill sets different from the skill sets for current vacancies, and if so, what types of new skill sets? 

2. How satisfied are employers with the services provided by the Centers? What services are of the 

most value? For what reasons do employers choose to work with the Center in contrast to other 

private or public recruitment (placement or staffing) organizations? What changes would 

employers recommend? 

3. What types of services were requested or of interest to Business Clients that would be provided by 

the Center or ROs (e.g., training, education, support services)? 

4. What specific skills and requirements do Business Clients have for the vacancies that they may be 

willing to fill through referrals from a Center? 

5. Are employers satisfied with the quality of the youth being referred to them by BI? 

6. What has been the experience of the employers with the persistence of BI participants? Why was 

there turnover of BI participants after placement? 

C. Project Organization and Staffing [Business Resource Centers] 

1. Does the basic staffing design (RS, HRC, BRCs, and Program Assistant) appear to be appropriate 

for each of the four Centers? Does each Center have the needed numbers of staff in each type of 

position? What criteria are used by BI in the recruitment and selection of staff? 

2. Has BI established performance requirements and expectations for each staff position/person? 

How does each Center evaluate staff performance against the requirements? Do staff receive 

placement bonuses? Is there evidence that placement bonuses affect performance results? What is 

the rate of turnover, by Center and position, over the period of the demonstration? What are the 

reasons that staff may not persist in employment? 

3. Does each Center have a fully operational management information system? Have enhancements 

been made to the MIS to reflect any changes in data elements for Phase II? Have staff been trained 

appropriately? 

4. How does each Center and BI use MIS data to assess progress toward goals and program 

effectiveness? For example: number of placements; placements by types of employers who hire 

DSYODP-II participants; wage rates by industry and SOC; wage progression and other 

advancements of participants. 

5. To what extent has each Center and BI been able to develop and implement planned components 
 

B-4 

for sustainability, such as linkages to Once-Stop Career Centers, fee-for-service arrangements, and 

leveraging of resources within the community? 

D. Service Delivery to Youth 

1. How does each of BI’s Centers assess the needs of each youth offender and at-risk youth for 

services? 

2. To what extent have youth offenders and at-risk youth actually received the services – whether 

through the Center for work preparedness or through other sources – that were identified as 

needed by each youth? 

3. What resources and networks has BI tapped into to help develop baseline employability (“soft”) 

skills for all youth? Overall, what strategies seem to be most effective in getting participants to be 

job-ready? 

4. To what extent do participants need pre-employment services beyond the basic services 

encompassed by the DOL/ETA funding for DSYODP? Do participants need such services as 

assessment, substance abuse treatment, mental health care, routine health care, housing, GED 

preparation, etc.? 

5. To what extent has each Center been able to gain access to services for its participants from other 

sources, such as Once-Stop Career Centers, education systems, community and faith-based 

organizations, public agencies, and other local resources? 

6. What strategies used by Center staff appear to positively affect persistence (i.e., retention) of 

participants in employment? 

E. Participant Outcomes 

1. What are the basic characteristics (age, gender, offender status, ethnicity, etc.) of the youth 

referred to each Center? 

2. Of the youth referred to each Center, what proportion of youth are placed? Do the characteristics 

of youth who are placed differ from those who are not placed? 

3. What proportion of participants is placed at or above the retention wage as established for each 

Center’s city? What benefits do participants receive? 

4. What are the wage progressions of participants over the period of the evaluation? What number 

and percent of participants are not initially placed at retention wage levels but subsequently reach 

such levels? 

5. What is the duration of employment for participants? How many participants who are placed lose 

employment during the period of the evaluation, and for what reasons? 

6. To what extent do participants achieve advancements beyond wage increases? For example: job 

promotion, more desirable title, more desirable shift/hours, improved benefits or conditions. 

7. To what extent do participants achieve certifications and licensures during employment? 

8. To what extent do participants achieve education outcomes (e.g., high school completion)? 

9. To what extent do participants avoid involvement with the criminal justice system? Do rates of 

involvement differ for ex-offenders compared to youth at-risk of court involvement? 

10. At the time of referral, what types of skills are most frequently present? Which types of skills are 

most frequently missing? 

F. Referring Organizations 

1. What types of ROs has each Center recruited? What populations do the ROs serve? What types of 

services (e.g., job-related, skills training, supportive) do the ROs provide? Which ROs have the 

most referrals? 
C-1 

APPENDIX C 

Site Visit Protocols 

Preliminary Site Visit Field Guide 

Follow-up Site Visit Protocols (Second Visits) 

Business Resource Center Manager 

Business Resource Consultant 

Human Resource Consultant 

Retention Specialist 

Executive Manager at Business Interface 

Businesses 

Referring Organizations 

Youth Participants 

Quality and Compliance Auditor/ Manager 

Headquarters Site Visit Protocols 

Third Visit Protocols

Center Staff Focus Group Questions 
 

C-2 
C-3 

Field Guide – Preliminary Site Visit 

Demand-Side Youth Offender Demonstration Project – Phase II 

Evaluator Name: _________________________________ Date of Visit: ________________ 

DSYODP-II Site: ________________________ 

Street Address: ______________________________________________________________ 

Principal Contact Person: __________________________ Position: ___________________ 

Telephone: ____________________ Fax: _____________________ 

Email: __________________________________________________ 

Individuals Interviewed Organization Position in Project 

 
 

C-4 

1. Description of operational status for Phase II 

a) Describe the implementation (Los Angeles) or resumption (New York, Chicago, and 

Washington, D.C.) of activities and relationships [Include a timeline to the date of preliminary 

visit]. What are the overall goals and priorities for the Center? Have goals or priorities changed 

since initial implementation/resumption of activities at this site? 

Staffing 

Position Name Date 

Hired 

Training Background/ Workforce 

Experience 

Business Resource 

Manager

 

Business Resource 

Consultant 1

 

Business Resource 

Consultant 2

 

Business Resource 

Consultant 3

 

Human Resource 

Consultant

 

Retention Specialist 

Quality & 

Compliance Auditor

 

b) For each position, what are the goals and priorities? 

Position Goals Priorities 

Business Resource 

Manager 

 

Business Resource 

Consultant 

 

Human Resource 

Consultant 

 

Retention 

Specialist 

 

Quality & 

Compliance 

 

c) What are the duties and roles of each staff person? [Bring job descriptions back to 

headquarters.] 

Position Name Roles and Duties 

Business Resource 

Manager

Business Resource 

Consultant 1

Business Resource 

Consultant 2

Business Resource 
C-5 

Consultant 3

Human Resource 

Consultant

Retention Specialist

Quality & 

Compliance Auditor

d) What is the general process used at the Center for entering data in the MIS? 

What types of reports are generated from the MIS? 

[To the evaluator: as feasible, observe use of MIS by staff; obtain samples of completed input 

documents and reports] 

2. Interactions with business clients, referring organizations, and youth

a) Depending on whether site is new (Los Angeles) or continuing (New York, Chicago, and 

Washington, D.C., what strategies are being used for recruiting businesses and referring 

organizations as clients? What has been the degree of success to date? 

 Strategies Successes 

Recruiting 

Businesses 

Recruiting 

Referring 

Organizations 

b) With which ROs and Service Delivery Partners does your Center have an established working 

arrangement (MOU)? 
 

C-6 

Referring Organizations Service Delivery Partners 

c) What has been the experience to date with recruiting and placing youth in employment? 

d) What is the nature of the working relationship between the HRC and the BRCs in identifying 

youth who might fill positions at businesses? 

e) What role does each person play in determining the qualifications and skills needed? 

 

f) For youth, provide approximate counts of: 

 Role in Determining Business Needs 

Human Resource 

Consultant 

Business 

Resource 

Consultant 
C-7 

Participant category Count 

Total youth referred to the Center since Phase II operations have 

begun 

Of total youth referred, number who are eligible for DSYODP 

Of those eligible for DSYODP, number placed in retention 

wage positions 

Of those eligible for DSYODP, number placed in positions but 

not qualifying as retention wage placements (based on wage 

level and/or number of dependents) 

3. Preparation for follow-up visits 

To the evaluator: Collect hard copy documents as needed to prepare for later site visits or data 

analyses: 

Documents Collected During Visit √

Job Descriptions 

MIS Data Entry Screens 

Business Contact List 

Referring Organizations Contact List 

Other Partner Organizations Contact List 

Local Employment Conditions Report 

Local maps marked with target areas 
 

C-8 

DSYODP II 

Follow-up Site Visit Protocols 

(Second Visit) 

for 

Business Resource Manager 

Business Resource Consultant 

Human Resource Consultant 

Retention Specialist 

Division Manager at Business Interface 

Businesses 

Referring Organizations 

Youth Participants 

Quality and Compliance Auditor/ Manager 
C-9

Site Visit Protocol 

Demand-Side Youth Offender Demonstration Project – Phase II 

Field Guide—Business Resource Manager 

NOTES: Under “Evaluation Item,” the numbers refer to the original Study Questions; only those Study Questions that involve field 

listed, but the numbering system was retained for reference back to the full set of Study Questions. Abbreviations for Data Sources: BRC – Business Resource Consultant; BRM – Business Resource Manager; RS – Retention SpeciConsultant 

EVALUATION ITEM PROMPTS AND REMINDERS 

A. Development of a network of employers 1.a. What approaches has your Center used to identify and recruit employers and broker job placements for youth ex-offenders and youth at-risk of court or gang involvement? 

• Linkages w/ CBOs, etc. • First contact, subsequent follow-up • What are their selling points? • How did they mine preexisting leads? 

2.a. Which approaches have been the most effective for identifying and recruiting employers 

who are willing to hire this target population? 

• How is effective defined? • Why is it effective—what aspect of 

approach is most important? 

3.a. To what extent are employers able to pay a “retention wage” to DSYODP participants? 

3.b. What challenges do your staff encounter in identifying employers with positions that 

pay a retention wage. What strategies have been effective in gaining a retention wage for as many participants as feasible? 

• Is there a career ladder to achieve a 

retention wage job? 

• What are the range of pay scales and jobs 

that provide retention wages? 

• What resources are used to identify 

employers who pay enough? 

• What selling points are used to convince 

employers? 

• Is there a difference between employers w/ 

retention wage vs. other employers? 

4.a. To what extent has your Center worked with employers to involve them in development and on-going operation of BI’s business intermediary model as implemented in each city?

• Specify model used • Who is involved? • Has regular contact facilitated other 

solutions? 

• Regular contact or more sporadic? 
C-10 

• How important is this contact? • Does this help institutionalize the program? • Is there a synergy that is created within this 

contact group? 

B. Employer outcomes 1.a. To what extent are employers’ needs for workers being met by this Center? 1.b. What types of businesses are not having their needs met? 1.c. Has this Center been able to make adjustments over time to meet those businesses’ needs for workers? 1.d. Have Business Clients indicated that they expect to have vacancies in the future that might require new skill sets different from the skill sets for current vacancies, and if so, what types of new skill sets? 

• If employer needs are not met, why? Is it 

quality of the workers? 

• Or is it in/ability to match workers w/ 

needs? 

• Do employers who require unskilled labor 

have different results than those who require skilled labor? 

• What proportion of the vacancies of each 

business have been filled through referrals from the Center? 

2.a. How satisfied are employers with the services provided by this Center? 

2.b. What services did they want that they did not receive? 

2.c. What services are of the most value? 2.d. For what reasons do employers choose to work with this Center in contrast to other private or public recruitment (placement or staffing) organizations? 

2. e. What changes would employers recommend? 

5.a. Are employers satisfied with the quality of the youth being referred to them by BI? Do Centers have a plan to increase the number of youth offender placements? 

• What about the youth quality worked well 

for employers? 

• What aspects of the youth were consistently 

unsatisfactory? 

6.a. What has been the experience of the employers with the persistence of BI participants? 

6.b. Why was there turnover of BI participants after placement? 

• Is lack of persistence a skills issue? If so, is 

it “hard skills” or “soft skills?” 

• Support issue? 

C. Project Organization and Staffing (Business Resource Centers) 1.a. Does the basic staffing design (RS, HRC, BRCs, and Program Assistant) appear to be appropriate for this Center? 1.b. Does this Center have the needed numbers of staff in each type of position? 

• What would you change about the design? • Are the numbers of staff and the criteria 

selection appropriate? Is it based on 
C-11

1.c. What criteria are used by BI in the recruitment and selection of staff? 

qualifications, experience, personality? 

2.a. Has BI established performance requirements and expectations for each staff position/person? 2.b. How does this Center evaluate staff performance against the requirements? 

2.c. Do staff receive placement bonuses? 2.d. Is there evidence that placement bonuses affect performance results? 2.e. What is the rate of turnover to date for each position? 2.f. What are the reasons that staff do not persist in employment? 

• Is there training to address challenges? 

What type? 

3.a. Does this Center currently have a fully operational management information system? 3.b. Have enhancements been made to the MIS since you’ve been with BI ? 3.c. Have staff been trained appropriately? 

• Is the MIS user-friendly? • Is information collected accurate? • Is data entry up to date? • What data are missing because of staff 

vacancies? 

• Does it provide an accurate picture of the 

work at BRC? 

• May depend on: 

• Understanding of MIS • Leadership • Organization support • Budget 

4.a. & b. How does this Center use MIS data to assess progress toward goals and program effectiveness? For example: number of placements; placements by types of employers who hire DSYODP-II participants; wage rates by industry and SOC; wage progression and other advancements of participants. 4.c. Does this Center receive reports produced by the evaluators? 

• Does staff understand the: 

• Importance of the data? • How it is used? 

• Do they realize what the data can help them 

achieve? 

• Success may depend on: 

• Ease of MIS • Technical expertise/staff capacity • Leadership 
C-12 

• Organization support • Budget • Environmental factors 

5.a. To what extent has this Center been able to develop and implement planned components for sustainability, such as linkages to Once-Stop Career Centers, fee-for-service arrangements, and leveraging of resources within the community? Could the business intermediary model function without grant money? How conscious do you think employers are that they are a part of an innovative model? Do you think they would pay for this service? What types of training programs have been implemented? At what sites? Who is conducting the trainings and how frequently? Do they use data to develop and revise the trainings? 

• How is sustainability defined? • How do the linkages work? • Why has sustainability worked or not 

worked? 

• Is there: 

• A written plan for sustaining the 

program? 

• Task force or other org to keep 

stakeholders together? 

• Dissemination of data and descriptive 

info to date? 

May depend on: 

• Early planning for sustainability • Visible involvement of stakeholders • Consistent outreach • Data-driven outcomes 
C-13

Site Visit Protocol 

Demand-Side Youth Offender Demonstration Project – Phase II 

Field Guide—Business Resource Consultant 

NOTES: Under “Evaluation Item,” the numbers refer to the original Study Questions; only those Study Questions that involve field 

listed, but the numbering system was retained for reference back to the full set of Study Questions. Abbreviations for Data Sources: BRC – Business Resource Consultant; BRM – Business Resource Manager; RS – Retention SpeciConsultant 

EVALUATION ITEM PROMPTS AND REMINDERS 

A. Development of a network of employers1.a. What types of approaches have you used to identify and recruit employers and broker job placements for youth ex-offenders and youth at-risk of court or gang involvement? 

• Linkages w/ CBOs, etc. • First contact, subsequent follow-up • What are their selling points? • How did they mine preexisting leads? 

2.a. Which approaches have been most effective identifying and recruiting employers who 

are willing to hire this target population/ 

• How is effective defined? • Why is it effective—what aspect of 

approach is most important? 

3.a. To what extent are employers able to pay a “retention wage” to DSYODP participants? 

3.b. What challenges do you encounter in identifying employers with positions that pay a retention wage and what strategies have been effective in gaining a retention wage for as 

many participants as feasible? 

• Is there a career ladder to achieve a 

retention wage job? 

• What are the range of pay scales and jobs 

that provide retention wages? 

• What resources are used to identify 

employers who pay enough? 

• What selling points are used to convince 

employers? 

• Is there a difference between employers w/ 

retention wage vs. other employers? 

4.a. To what extent have you worked with employers to involve them in development and on-going operation of BI’s business intermediary model as implemented in each city? 

• Specify model used • Who is involved? • Has regular contact facilitated other 

solutions? 

• Regular contact or more sporadic? • How important is this contact? • Does this help institutionalize the program? 
C-14 

• Is there a synergy that is created within this 

contact group? 

B. Employer outcomes1.a. To what extent are employers’ needs for workers being met by this Center? 1.b. What types of businesses are not having their needs met? 1.c. Has this Center staff been able to make adjustments over time to meet those businesses’ needs for workers? 1.d. Have Business Clients indicated that they expect to have vacancies in the future that might require new skill sets different from the skill sets for current vacancies, and if so, what types of new skill sets? 

• If employer needs are not met, why? Is it 

quality of the workers? 

• Or is it in/ability to match workers w/ 

needs? 

• Do employers who require unskilled labor 

have different results than those who require skilled labor? 

• What proportion of the vacancies of each 

business have been filled through referrals from the Center? 

2.a. How satisfied are employers with the services provided by the Centers? 

2.b. What services did they want that they did not receive? 

2.c. What services are of the most value? 2.d. For what reasons do employers chose to work with this Center in contrast to other private or public recruitment (placement or staffing) organizations? 

2. e. What changes would employers recommend? 

 

5.a. Are employers satisfied with the quality of the youth being referred to them by BI? • What about the youth quality worked well 

for employers? 

• What aspects of the youth were consistently 

unsatisfactory? 

6.a. What has been the experience of the employers with the persistence of BI participants? 

6.b. Why was there turnover of BI participants after placement? 

• Is lack of persistence a skills issue? If so, is 

it “hard skills” or “soft skills?” 

• Support issue? 

C. Project Organization and Staffing (Business Resource Centers)1.a. Does the basic staffing design (RS, HRC, BRCs, and Program Assistant) appear to be appropriate for this Center? 1.b. Does this Center have the needed numbers of staff in each type of position? 

• What would you change about the design? • Are the numbers of staff and the criteria 

selection appropriate? Is it based on qualifications, experience, personality? 

3.a. Does this Center currently have a fully operational management information system? • Is the MIS user-friendly? 
C-15

3.b. Have enhancements been made to the MIS since you’ve been with BI? 3.c. Have staff been trained appropriately? 

• Is information collected accurate? • Is data entry up to date? • What data are missing because of staff 

vacancies? 

• Does it provide an accurate picture of the 

work at BRC? 

• May depend on: 

• Understanding of MIS • Leadership • Organization support • Budget 

4.a. & b. How does this Center use MIS data to assess progress toward goals and program effectiveness? For example: number of placements; placements by types of employers who hire DSYODP-II participants; wage rates by industry and SOC; wage progression and other advancements of participants. 4.c. Does this Center receive reports produced by the evaluators? 

• Does staff understand the: 

• Importance of the data? • How it is used? 

• Do they realize what the data can help them 

achieve? 

• Success may depend on: 

• Ease of MIS • Technical expertise/staff capacity • Leadership • Organization support • Budget • Environmental factors 
C-16 
C-17

Site Visit Protocol 

Demand-Side Youth Offender Demonstration Project – Phase II 

Field Guide—Human Resources Consultant 

NOTES: Under “Evaluation Item,” the numbers refer to the original Study Questions; only those Study Questions that involve field 

listed, but the numbering system was retained for reference back to the full set of Study Questions. Abbreviations for Data Sources: BRC – Business Resource Consultant; BRM – Business Resource Manager; RS – Retention SpeciConsultant 

EVALUATION ITEM PROMPTS AND REMINDERS 

B. Employer outcomesDo you have any direct contact with employers? 2.a. How satisfied are employers with the services provided by this Center? 

2.b. What services did they want that they did not receive? 

2.c. What services are of the most value? 2.d. For what reasons do employers choose to work with this Center in contrast to other private or public recruitment (placement or staffing) organizations? 

2.e. What changes would employers recommend? 

 

5.a. Are employers satisfied with the quality of the youth being referred to them by BI? • What about the youth quality worked well 

for employers? 

• What aspects of the youth were consistently 

unsatisfactory? 

C. Project Organization and Staffing (Business Resource Centers)1.a. Does the basic staffing design (RS, HRC, BRCs, and Program Assistant) appear to be appropriate for this Center? 1.b. Does this Center have the needed numbers of staff in each type of position? 

• What would you change about the design? • Are the numbers of staff and the criteria 

selection appropriate? Is it based on qualifications, experience, personality? 

3.a. Does this Center have a fully operational management information system? • Is the MIS user-friendly? 
C-18 

3.b. Have enhancements been made to the MIS since you’ve been with BI? 3.c. Have staff been trained appropriately? 

• Is information collected accurate? • Is data entry up to date? • Does it provide an accurate picture of the 

work at BRC? 

• May depend on: 

• Understanding of MIS • Leadership • Organization support • Budget 

4.a. & b. How does this Center use MIS data to assess progress toward goals and program effectiveness? For example: number of placements; placements by types of employers who hire DSYODP-II participants; wage rates by industry and SOC; wage progression and other advancements of participants. 4.c. Does this Center receive reports produced by the evaluators? 

• Does staff understand the: 

• Importance of the data? • How it is used? 

• Do they realize what the data can help them 

achieve? 

• Success may depend on: 

• Ease of MIS • Technical expertise/staff capacity • Leadership • Organization support • Budget • Environmental factors 

D. Service Delivery to Youth1.a. How does this Center assess the needs of each youth offender and at-risk youth for services? 

• Mental health? • Medical? • Educational? • Support? • Employment? • What assessments are administered by 

referring orgs and what are done on-site? 

4.a. To what extent do participants need pre-employment services beyond the basic services encompassed by the DOL/ETA funding for DSYODP? 

• What types of additional services were 

needed? 

• And if they received the services, were the 

services successful in resolving the problem? 
C-19

4.b. Do participants need such services as assessment, substance abuse treatment, 

mental health care, routine health care, housing, GED preparation, etc.? 

• Dimensions to explore: 

• Funding • Advocacy • Collaboration • Partnerships 

2.a. To what extent have youth offenders and at-risk youth actually received the services – 

whether through the Center for work preparedness or through other sources – that were identified as needed by each youth? 

• What soft skills were the most problematic? • Was job training given? • Do youth show up for recommended 

services? 

3.a. What resources and networks has BI tapped into to help develop baseline employability 

(“soft”) skills for all youth? 

Overall, what strategies seem to be most effective in getting participants to be jobready? 

• Do referring organizations provide this type 

of skills training, or are other organizations also used? 

• Do the organizations provide both 

assessments and training? 

5.a. To what extent has this Center been able to gain access to services for its participants from other sources, such as Once-Stop Career Centers, education systems, community and faith-based organizations, public agencies, and other local resources? 

• Would an intermediary model work for 

dealing with other organizations involved? 

• May depend on: 

• Cross-service or cross-organization 

leadership or support 

• Consistent messages about project 

activities, successes (marketing) 

• Broad-based advocacy • Interlocking board memberships 

6.a. What strategies used by Center staff appear to increase participant retention in employment? 

• Is there a focus on skill level training? • Or do soft skills matter more in retaining 

employment? 
C-20 
C-21

Site Visit Protocol 

Demand-Side Youth Offender Demonstration Project – Phase II 

Field Guide—Retention Specialist 

NOTES: Under “Evaluation Item,” the numbers refer to the original Study Questions; only those Study Questions that involve field 

listed, but the numbering system was retained for reference back to the full set of Study Questions. 

Abbreviations for Data Sources: BRC – Business Resource Consultant; BRM – Business Resource Manager; RS – Retention Speci

Consultant 

EVALUATION ITEM PROMPTS AND REMINDERS 

A. Development of a network of employers3.a. To what extent are employers able to pay a “retention wage” to DSYODP participants? 

3.b. What challenges does this Center encounter in identifying employers with positions that 

pay a retention wage and what strategies have been effective in gaining a retention wage for as many participants as feasible? 

• Is there a career ladder to achieve a 

retention wage job? 

• What are the range of pay scales and jobs 

that provide retention wages? 

• What resources are used to identify 

employers who pay enough? 

• What selling points are used to convince 

employers? 

• Is there a difference between employers w/ 

retention wage vs. other employers? 

B. Employer outcomes2.a. How satisfied are employers with the services provided by this Center? 

2.b. What retention services did they want that they did not receive? 

2.c. What retention services are of the most value? 2.d. For what reasons do employers choose to work with this Center in contrast to other private or public recruitment (placement or staffing) organizations? 2.e. What changes would employers recommend? 

 

5.a. Are employers satisfied with the quality of the youth being referred to them by BI? • What about the youth quality worked well 

for employers? 

• What aspects of the youth were consistently 
C-22 

unsatisfactory? 

6.a. What has been the experience of the employers with the persistence of BI participants? 

6.b. Why is there turnover of BI participants after placement? 

What do RSs (or Center staff) do when youth are vulnerable to losing a job? 

• Is lack of persistence a skills issue? If so, is 

it “hard skills” or “soft skills?” 

• Support issue? 

C. Project Organization and Staffing (Business Resource Centers)1.a. Does the basic staffing design (RS, HRC, BRCs, and Program Assistant) appear to be appropriate for this Center? 1.b. Does this Center have the needed numbers of staff in each type of position? 

1.c. What criteria are used in the recruitment and selection of staff? 

• What would you change about the design? • Are the numbers of staff and the criteria 

selection appropriate? Is it based on qualifications, experience, personality? 

3.a. Does this Center have a fully operational management information system? 3.b. Have enhancements been made to the MIS since you’ve been with BI? 3.c. Have staff been trained appropriately? 

• Is the MIS user-friendly? • Is information collected accurate? • Is data entry up to date? • Does it provide an accurate picture of the 

work at BRC? 

• May depend on: 

• Understanding of MIS • Leadership • Organization support • Budget 

4.a. & b. How does this Center use MIS data to assess progress toward goals and program effectiveness? For example: number of placements; placements by types of employers who hire DSYODP-II participants; wage rates by industry and SOC; wage progression and other advancements of participants. 4.c. Does this Center receive reports produced by the evaluators? 

• Does staff understand the: 

• Importance of the data? • How it is used? 

• Do they realize what the data can help them 

achieve? 

• Success may depend on: 

• Ease of MIS • Technical expertise/staff capacity • Leadership • Organization support • Budget 
C-23

• Environmental factors 

D. Service Delivery to Youth1.a. How does this Center assess the needs of each youth offender and at-risk youth for services? 

• Mental health? • Medical? • Educational? • Support? • Employment? • What assessments are administered by 

referring orgs and what are done on-site? 

4.a. To what extent do participants need pre-employment services beyond the basic services encompassed by the DOL/ETA funding for DSYODP? 

• What types of additional services were 

needed? 

• And if they received the services, were the 

services successful in resolving the need? 

4.b. Do participants need such services as assessment, substance abuse treatment, mental health care, routine health care, housing, GED preparation, etc.? 

• Dimensions to explore: 

• Funding • Advocacy • Collaboration • Partnerships 

2.a. To what extent have youth offenders and at-risk youth actually received the services – 

whether through the Center for work preparedness or through other sources – that were identified as needed by each youth? 

• What soft skills were the most problematic? • Was job training given? • Do youth show up for recommended 

services? 

3.a. What resources and networks has this Center tapped into to help develop baseline employability (“soft”) skills for all youth? 

Overall, what strategies seem to be most effective in getting participants to be jobready? 

• Do referring organizations provide this type 

of skills training, or are other organizations also used? 

• Do the organizations provide both 

assessments and training? 

5.a. To what extent has this Center been able to gain access to services for its participants from other sources, such as Once-Stop Career Centers, education systems, community and faith-based organizations, public agencies, and other local resources? 

• Would an intermediary model work for 

dealing with other organizations involved? 

• May depend on: 

• Cross-service or cross-organization 

leadership or support 

• Consistent messages about project 

activities, successes (marketing) 

• Broad-based advocacy • Interlocking board memberships 
C-24 

6.a. What strategies are used by Center staff appear to positively affect persistence (i.e., 

retention) of participants in employment? • Is there a focus on skill level training? 

• Or do soft skills matter more in retaining 

employment? 
C-25

Site Visit Protocol 

Demand-Side Youth Offender Demonstration Project – Phase II 

Field Guide—Division Manager at Business Interface 

NOTES: Under “Evaluation Item,” the numbers refer to the original Study Questions; only those Study Questions that involve field 

listed, but the numbering system was retained for reference back to the full set of Study Questions. 

Abbreviations for Data Sources: BRC – Business Resource Consultant; BRM – Business Resource Manager; RS – Retention Speci

Consultant 

EVALUATION ITEM PROMPTS AND REMINDERS 

C. Project Organization and Staffing (Business Resource Centers)2.b. How does each Center evaluate staff performance against the requirements? 

2.c. Do staff receive placement bonuses? 2.d. Is there evidence that placement bonuses affect performance results? 2.f. What are the reasons that staff may not persist in employment? 

• Is there training to address challenges? 

What type? 

3.a. Does each Center have a fully operational management information system? 3.b. Have enhancements been made to the MIS to reflect any changes in data elements for Phase II? 3.c. Have staff been trained appropriately? Is there MIS data that is incomplete as a result of technical/infrastructure problems or staff turnover? Are the Centers receiving and using reports from the evaluators?Do Centers have a plan to increase the number of youth offender participants? 

• Is the MIS user-friendly? • Is information collected accurate? • Is data entry up to date? • Does it provide an accurate picture of the 

work at BRC? 

• May depend on: 

• Understanding of MIS • Leadership • Organization support • Budget 

Could the business intermediary model function without grant money?How conscious do you think employers are that they are a part of an innovative model? 
C-26 

Do you think they would pay for this service? What types of training programs have been implemented? At what sites? Who is conducting the trainings and how frequently? Do they use data to develop and revise the trainings? 
C-27

Site Visit Protocol 

Demand-Side Youth Offender Demonstration Project – Phase II 

Field Guide--Businesses 

NOTES: Under “Evaluation Item,” the numbers refer to the original Study Questions; only those Study Questions that involve field 

listed, but the numbering system was retained for reference back to the full set of Study Questions. 

Abbreviations for Data Sources: BRC – Business Resource Consultant; BRM – Business Resource Manager; RS – Retention Speci

Consultant 

EVALUATION ITEM PROMPTS AND REMINDERS 

A. Development of a network of employersCan you tell us a little bit about the kind of work you do here? • Get a sense of the work involved, then ease 

them into questions on intermediary model

1.a. How did the Center come to identify and recruit you as an employer for placing youth ex-offenders and youth at-risk of court or gang involvement? 

• Linkages w/ CBOs, etc. • First contact, subsequent follow-up • What are their selling points? • How did they mine preexisting leads? 

2.a. What was effective about their approach? • How is effective defined? 

• Why is it effective—what aspect of 

approach is most important? 

3.a. To what extent was your establishment able to pay a “retention wage” to DSYODP 

participants? 

• Is there a career ladder to achieve a 

retention wage job? 

• What are the range of pay scales and jobs 

that provide retention wages? 

• What resources are used to identify 

employers who pay enough? 

• What selling points are used to convince 

employers? 

• Is there a difference between employers w/ 

retention wage vs. other employers? 

4.a. To what extent has the Center worked with your establishment in developing and sustaining operations of BI’s business intermediary model? 

• Specify model used • Who is involved? • Has regular contact facilitated other 

solutions? 

• Regular contact or more sporadic? • How important is this contact? 
C-28 

• Does this help institutionalize the program? • Is there a synergy that is created within this 

contact group? 

B. Employer outcomes1.a. To what extent are your needs for workers being met by the Center? 

1.b. What needs are not being met? 1.c. Has the Center been able to make adjustments over time to meet those needs for 

workers? 1.d. Do you expect to have vacancies in the future that might require new skill sets different from the skill sets for current vacancies, and if so, what types of new skill sets? 

• If employer needs are not met, why? Is it 

quality of the workers? 

• Or is it in/ability to match workers w/ 

needs? 

• Do employers who require unskilled labor 

have different results than those who require skilled labor? 

• What proportion of the vacancies of each 

business have been filled through referrals from the Center? 

2.a. How satisfied are you with the services provided by the Center? 2.b. What services do you want that you do not receive? 

2.c. What services are of the most value? 2.d. For what reasons do you chose to work with the Center in contrast to other private or public recruitment (placement or staffing) organizations? 2. e. What changes would you recommend? 

 

5.a. Are you satisfied with the quality of the youth being referred to you by BI? • What about the youth quality worked well 

for employers? 

• What aspects of the youth were consistently 

unsatisfactory? 

6.a. What has been your experience with the persistence of BI participants? 

6.b. Is there turnover of BI participants after placement? If so, why? 

• Is lack of persistence a skills issue? If so, is 

it “hard skills” or “soft skills?” 

• Support issue? 
C-29

Site Visit Protocol 

Demand-Side Youth Offender Demonstration Project – Phase II 

Field Guide—Referring Organizations 

NOTES: Under “Evaluation Item,” the numbers refer to the original Study Questions; only those Study Questions that involve field 

listed, but the numbering system was retained for reference back to the full set of Study Questions. 

Abbreviations for Data Sources: BRC – Business Resource Consultant; BRM – Business Resource Manager; RS – Retention Speci

Consultant 

EVALUATION ITEM PROMPTS AND REMINDERS 

D. Service Delivery to YouthCan you tell us a little bit about the kind of work you do here? What services do you offer the youth? Do you have goals specific to workforce development? How did you come to refer youth to BI’s Center? 

• Get a sense of their work, then ease them 

into questions on intermediary model

1.a. How does the Center assess the needs of each youth offender and at-risk youth for services? 

• Mental health? • Medical? • Educational? • Support? • Employment? • What assessments are administered by 

referring orgs and what are done on-site? 

What types of services do youth participants referred to BI generally need? 4.a. To what extent do participants need pre-employment services? 

• What types of additional services were 

needed? 

• And if they received the services, were the 

services successful in resolving the need? 

4.b. Do participants need such services as assessment, substance abuse treatment, mental health care, routine health care, housing, GED preparation, etc.? 

• Dimensions to explore: 

• Funding • Advocacy • Collaboration 
C-30 

• Partnerships 

2.a. To what extent do youth offenders and at-risk youth actually receive the services for 

work preparedness, whether through your organization or the Center? 

• What soft skills were the most problematic? • Was job training given? 

3.a. Overall, what strategies seem to be most effective in getting participants to be jobready? Do you provide “soft skills” training and assessments of job-readiness? • Do referring organizations provide this type of skills training, or are other organizations 

also used? 

• Do the organizations provide both 

assessments and training? 

5.a. Does the Center gain access to services for its participants from other sources, such as Once-Stop Career Centers, education systems, community and faith-based organizations, public agencies, and other local resources? 

• Would an intermediary model work for 

dealing with other organizations involved? 

6.a. What strategies have you identified that appear to increase youth retention in employment? Are these strategies employed by BI staff members? 

• Is there a focus on skill level training? • Or do soft skills matter more in retaining 

employment? 
C-31

Site Visit Protocol 

Demand-Side Youth Offender Demonstration Project – Phase II 

Field Guide—Youth Participants 

EVALUATION ITEM PROMPTS AND REMINDERS 

D. Service Delivery to Youth How were you referred to BI’s Center? • Get a sense of their work, then ease them 

into questions on intermediary model 

What types of services have you received through the Referring Organization? What skills have you acquired through the Referring Organization or by participating in the program with BI? How often do you visit BI or connect with BI staff? Have you worked with other referral sources, such as Once-Stop Career Centers, education systems, community and faith-based organizations, public agencies, and other local resources? Do you have specific goals regarding your employment? Has working with BI helped you achieve your goals? B. Employer outcomes Are you currently employed as a result of working with BI? If employed:

How would you describe your current employment? Can you tell us a little bit about the kind of work you do? Have you received additional services since being placed in employment by the Center? Has working with BI helped you achieve your goals for employment? 
C-32 
C-33

Site Visit Protocol 

Demand-Side Youth Offender Demonstration Project – Phase II 

Field Guide—Quality and Compliance Auditor / Manager 

EVALUATION ITEM PROMPTS AND REMINDERS 

C. Project Organization and Staffing (Business Resource Centers)

2.b. How does this Center evaluate staff performance against the requirements? 

 

3.a. Does each Center have a fully operational management information system? 3.b. Have enhancements been made to the MIS to reflect any changes in data elements since you’ve been with BI? 3.c. Have staff been trained appropriately? What types of training programs have been 

implemented? At what sites? Who is conducting the trainings? How frequent? Do they use data to develop and revise the trainings? Is there MIS data that is incomplete as a result of technical/ infrastructure problems or staff turnover? Are the Centers receiving and using reports from the evaluators? 

• Is there training to address challenges? 

What type? 

• Is the MIS user-friendly? • Is information collected accurate? • Is data entry up to date? • Does it provide an accurate picture of the 

work at BRC? 

• May depend on: 

• Understanding of MIS • Leadership • Organization support • Budget 
C-34 
C-35 

DSYODP-II 

Headquarters Site Visit Protocols 

Interview with Mr. Rodney Carroll and Executive Staffs

 

We’d like to start out with a few background questions about your experiences and the beginnings of the 

project... 

How did the Welfare-to-Work Partnership begin? How did the focus move to youth and youth offenders, more 

particularly? 

What is your long-term vision for Business Interface, Inc.? 

Can you tell us a little bit about the work you do and the organizational activities maintained in the central office? 

What are the ways the central office connects and coordinates the work of the four Centers? 

Are there regularly scheduled meetings or conference calls? If so, how often are these? Who participates in these 

calls? 

For the next few questions, we would like to get a sense of your assessment of the project’s progress and the 

tools BI uses to assess progress... 

2.b. How does each Center evaluate staff performance against the requirements? 

How is the quarterly performance score calculated for the four Centers? How do you interpret the numbers? 

How do you assess the quality of the work beyond the numbers? 

How well do you think the basic staffing model (3 BRCs, 1 HRC, 1 RS, 1 Auditor) works for accomplishing your 

goals? 

What types of training programs have been implemented? At which sites? 

Who conducts (or has conducted) the trainings and how frequently? Do they use data to develop and revise the 

trainings? 

How are the Centers making use of information from the evaluators’ reports? 

3.b. What enhancements were made to the MIS? How do these enhancements reflect ongoing changes in BI’s 

needs? 

3.c. What types of training have staff received on using the new MIS? 

3.a. How operational is the enhanced management information system? 

Is there MIS data that is incomplete as a result of technical/ infrastructure problems or staff turnover? 

2.d. Is there evidence that placement bonuses affect performance results at the Centers? 

2.f. In your opinion, what factors account for the high turnover in both central office staff and staff at the Centers? 

Do you have expectations about the uniformity of BRC strategies to select employers? 

Do you have expectations about the uniformity of BRC strategies to choose industries and occupations to target? 
 

C-36 

What are the differences in working with youth offenders vs. youth at-risk? 

What is your definition of success for a youth? For an employer? 

How conscious do you think employers are that they are a part of an innovative model? 

What factors do you think keep the Workforce System (One Stop Centers) from using the intermediary model? 

From what we’ve gathered from the site visits, growth in the number of recruitments and placement activity 

seems to be both a blessing and a problem given the limitations on hiring additional staff at the Centers. 

Do you think the project was adequately funded to support the staff needed to fulfill DOL/ETA’s requirements? 

Are the expenses greater than anticipated for working with this group of constituents? If so, how? 

The Centers seem to have recruited plenty of employers and RO clients. Why the push to keep expanding the 

network of organizations and employers? 

For this final segment, we’d like to ask you some questions about sustainability and the future for BI... 

If you had it to do all over again, what changes would you make in structuring the organization? 

Centers seem to be under some financial strain, would you have allocated funds differently if you had it to do over? 

How would you add a more entrepreneurial dimension to the model so BI could continue its work? 

Could the business intermediary model function without grant money? 

Can you describe some of the steps BI is taking to achieve sustainability? What challenges does BI face in trying to 

accomplish this goal? What successes have you had thus far? 

Do you think employers would pay for this service? 

Do the Centers have a plan to increase the number of youth offender participants? 

Are there plans to conduct future trainings of staff at the Centers? 

Do you have any questions you would like to ask us? 

Thank you. 
C-37 

Interview with IT Consultant

How long have you been employed with BI? 

Can you tell us a little bit about your background and the work you do at BI? 

Are there particular staff members that you work with on a regular basis? 

In what ways do you connect or communicate with the staff at the Centers? 

Are there regularly scheduled meetings or conference calls? If so, how often are these? Who participates in these 

calls? 

3.a. Does each Center have a fully operational management information system? 

3.b. Can you describe the enhancements made to the MIS to ensure data accuracy and improved data collection? 

When were the changes introduced? 

How is the new system better or more user-friendly relative to the old system? 

3.c. Have staff at the Centers been trained in using the new system?

What types of training programs have been implemented? 

Do you use data to develop and revise the trainings? 

Based on information gathered during the site visits, we’ve received mixed reviews from center staff about 

the new MIS. Overall, how would you characterize staffs’ response to the new system of data collection? 

Is there MIS data that is incomplete as a result of technical/ infrastructure problems or staff turnover? 

From the site visits, we also learned that a Help Desk was set up for staff at the Centers. Can you tell us when 

the MIS Help Desk established and how the Help Desk system works? 

Staff at the Centers mentioned some problems with equipment, like faxes and copiers, not working properly.

Can you tell us about what infrastructure needs you’ve identified for the company? 

Of those infrastructural or technical needs, which do you identify as crucial to BI’s ability to accomplish its goals? 

In what ways is BI taking steps to address these needs? 

 

The evaluators for DSYODP have produced several reports based on data collected from the project. Have 

you seen any of the previous reports from the evaluators? 

Has information from the evaluators’ reports been incorporated in addressing the IT needs of BI? 

Do you have any questions for us? 

Thank you. 
 

C-38 
C-39 

DSYODP-II 

Center Staff Focus Group Questions 

Third Site Visit 

A. Development of a network of employers 

1.a. What types of approaches have you used to identify and recruit employers and broker job 

placements for youth participants? 

2.a. Which approaches have been the most effective in identifying and recruiting employers who are willing to hire 

participants? 

3.a. To what extent were employers willing and able to pay a “retention wage” to DSYODP participants? 

3.b. What challenges did you encounter in identifying employers with positions that pay a retention wage and what 

strategies have been effective in gaining a retention wage for participants? 

4.a. How were you able to involve employers in the development and on-going operation of BI’s intermediary 

model? 

Were some employers more receptive than others to the goals of BI? If so, why? 

B. Employer outcomes 

1.a. To what extent would you say that employers’ needs for workers have been met by this Center? 

1.b. What additional needs did businesses desire to have met by this Center? 

1.c. Have you made adjustments over time to meet those businesses’ needs for workers? What types of 

adjustments? 

Were there particular types of businesses that did not work out? Why not? 

1.d. Have Business Clients indicated that they expect to have vacancies in the future that might require new skill 

sets? If so, what types of new skill sets? 

2.a. How satisfied were employers with the services provided by the Centers? 

2.b. What services did they want that they did not receive from BI? 

2.c. What services were of the most value? 

2.d. For what reasons did employers chose to work with this Center in contrast to other private or public recruitment 

(placement or staffing) organizations? 

How conscious do you think employers were that they were a part of an innovative model? 

2.e. What changes to the model did employers recommend? 

5.a. Were employers satisfied with the quality of the youth being referred to them by BI? 

6.a. What has been the experience of the employers with the persistence of BI participants? 

6.b. Can you describe some of the reasons for turnover of BI participants after placement? 
 

C-40 

C. Project Organization and Staffing (Business Resource Centers)

1.a. Did the basic staffing design (RS, HRC, BRCs, and Program Assistant) appear to be appropriate for this Center? 

1.b. Did this Center have the needed staff in each type of position? 

3.a. Did this Center currently have a fully operational management information system? 

3.b. Have additional enhancements been made to the MIS since our last visit with BI? 

3.c. Have staff received any additional training on the MIS since our last visit? 

Are there incomplete or possibly incorrect data in the MIS? 

What factors might have contributed to incomplete or incorrect data? 

Has technical/ infrastructure problems or staff turnover affected the quality of data in the MIS? 

4.a. & b. How did you use MIS data to assess progress toward goals and program effectiveness? For example: 

number of placements; placements by types of employers who hire DSYODP-II participants; wage rates by industry; 

wage progression and other advancements of participants. 

What tools were helpful for you to perform your job duties? 

What additional resources would have been helpful to accomplish your Center’s goals? 

Can you describe the relationship between your center and BI headquarters? Between your center and other Centers 

in the project? How have these changed over time? 

How often did you correspond with or receive updates from BI headquarters? What were the most common methods 

of communication between your center and BI headquarters? (i.e. fax, email, phone) What issues were the most 

commonly addressed in your correspondence? (i.e. hiring of staff, placement goals, etc.) 

4.c. Did this Center receive reports produced by the evaluators? 

What types of staff development or training programs have been implemented at your center? 

What factors do you think would be necessary for the business intermediary model to be sustainable? 

5.a. To what extent has this Center been able to develop linkages to Once-Stop Career Centers, establish fee-forservice arrangements, and leverage resources within the community? 

Do you think employers would pay for this service? 

D. Service Delivery to Youth 

Overall, what strategies seemed to be the most effective in getting participants to be job-ready? 

3.a. What resources and networks has this Center tapped into to develop “soft skills”, like timeliness and 

professional etiquette, for youth participants? 

4.a. To what extent did participants need pre-employment services beyond the basic services encompassed by the 

DOL/ETA funding for DSYODP? What types of additional services were most often needed? 
C-41 

5.a. To what extent has this Center been able to gain access to services for its participants from other sources, such 

as Once-Stop Career Centers, education systems, community and faith-based organizations, public agencies, and 

other local resources? 

6.a. What strategies did you employ to increase participant retention in employment? Of these, which were the most 

effective? 

In conclusion, if you were to construct a project based on a demand-side intermediary model, how would you do it? 

What components of the model would you change? What components of the model would you keep? 

Thank you for your time. 
 

C-42 
D-1 

APPENDIX D 

Correlation Coefficients 
 

D-2 
D-3 

Pearson correlation coefficients for variables used in the OLS regressions. The data presented combine participants in the sample from 

Washington, DC and Miami linked to Maryland and Florida UI wage records.

Variable N Mean Std 1 2 3 4 5 6 7 8 9 10 11 12

Female 420 -- -- 1.00Non-African-American 420 -- -- .01 1.00

Education other than high school 

completion

420 -- -- .01 -.04 1.00

Married 420 -- -- -.01 .16* .00 1.00

Offender 420 -- -- -.3* -.10* .08 -.04 1.00

Has Children 420 -- -- .15* -.11* .09 .17* .02 1.00

Florida 420 -- -- .08 .29* -.19* .04 -.09 -.27* 1.00Non-placement 420 -- -- -.01 -.18* .10* -.08 .23* .16* -.46* 1.00

Log Average Weekly UI Wages 

after Enrollment 

324 4.97 1.09 .09 .15* -.19* .07 -.13* -.09 .34* -.31* 1.00

Log Average Weekly UI Wages 

after Placement

316 4.98 1.10 .07 .15* -.18* .06 -.13* -.09 .34* -.30* .99* 1.00

Proportion of Enrollment 

Quarters w/ UI Wage Records

340 0.64 0.34 .04 .13* -.16* .08 -.16* -.24* .54* -.33* .49* .49* 1.00

Log N of UI Employers 340 1.8 0.92 .05 .14* -.18* .05 -.10 -.23* .58* -.32* .53* .52* .93* 1.00

*p<.05

Pearson correlation coefficients for variables used in the OLS regressions. The data presented combine participants in the sample from 

Washington, DC and Miami linked to Maryland and Florida UI wage records.

Variable N Mean Std 1 2 3 4 5 6 7 8 9 10 11 12

Female 420 -- -- 1.00Non-African-American 420 -- -- .01 1.00

Pearson correlation coefficients for variables used in the OLS regressions. The data presented combine participants in the sample from 

Washington, DC and Miami linked to Maryland and Florida UI wage records.

Variable N Mean Std 1 2 3 4 5 6 7 8 9 10 11 12

Female 420 -- -- 1.00Non-African-American 420 -- -- .01 1.00

Education other than high school 

completion

420 -- -- .01 -.04 1.00 Education other than high school 

completion

Married 420 -- -- -.01 .16* .00 1.00

Offender 420 -- -- -.3* -.10* .08

420 -- -- .01 -.04 1.00

Married 420 -- -- -.01 .16* .00 1.00

Offender 420 -- -- -.3* -.10* .08 -.04 1.00

Has Children 420 -- -- .15* -.11* .09 .17* .02 1.00

Florida 420 -- -- .08 .29* -.19* .04 -.09 -.27* 1.00

-.04 1.00

Has Children 420 -- -- .15* -.11* .09 .17* .02 1.00

Florida 420 -- -- .08 .29* -.19* .04 -.09 -.27* 1.00Non-placement 420 -- -- -.01 -.18* .10* -.08 .23* .16* -.46* 1.00

Log Average Weekly UI Wages 

after Enrollment 

324 4.97 1.09 .09 .15* -.19* .07 -.13* -.09 .34* -.31* 1.00

Log Average Weekly UI Wages 

after Placement

316 4.98 1.10 .07 .15*

Non-placement 420 -- -- -.01 -.18* .10* -.08 .23* .16* -.46* 1.00

Log Average Weekly UI Wages 

after Enrollment 

324 4.97 1.09 .09 .15* -.19* .07 -.13* -.09 .34* -.31* 1.00

Log Average Weekly UI Wages 

after Placement

316 4.98 1.10 .07 .15* -.18* .06 -.13* -.09 .34* -.30* .99* 1.00

Proportion of Enrollment 

Quarters w/ UI Wage Records

340 0.64 0.34 .04 .13* -.16* .08 -.16* -.24* .54* -.33* .49* .49* 1.00

Log N of UI Employers 340 1.8 0.92 .05 .14* -.18* .05 -.10 -.23* .58* -.32* .53* .52* .93* 1.00

-.18* .06 -.13* -.09 .34* -.30* .99* 1.00

Proportion of Enrollment 

Quarters w/ UI Wage Records

340 0.64 0.34 .04 .13* -.16* .08 -.16* -.24* .54* -.33* .49* .49* 1.00

Log N of UI Employers 340 1.8 0.92 .05 .14* -.18* .05 -.10 -.23* .58* -.32* .53* .52* .93* 1.00

*p<.05