?*. DOCUMENT RESUME

ED 206:860 CE 029 968

TITLE Knowledge Development under the Youth

Initiatives-7Proceedings of an Oierviell'Conference

(Waltham, 'Massachusetts, October 5-6, 1978)."

Knowledge Development Framework. Youth Knowledge

Develdpment Report 1.3..

INSTITUTION 'Brandeis Univ., Waltham, Mass. . SPONS AGENCY 'Employment and Training Administration .006,

WashingtOn, D.C. Office of Youth Programs.

PUB DATE May-80'

NOTE - 243B.; For related documents see ED 182 454, ED 199

377, ED 203 056, CE 029 544, CE 029 570, and CE 029

969-980.40

AVAILABLE FROM Superintendent of Documents, U.S. 'Government Printing,

-. Office, Washington, 'DC 20402 (Stock No.

029-014-100118-0, s6.00.

ED RS PRICE.

DESCRIPWRS.

MF01/PC10 Plus Postage.

Aiency Cooperation; Community Services; Data

Analysis; Data Collection; *Dembnstration Programs;

Educational Needs; EdUcational Objectives;

Educational Planning; Educational Policy; Educational

Practices; *Educational Rese-arch; Educational Trends;

Employment Patterns; *Employment Programs; Field

Experience Programs; Job Training; Labor Force; Needs

Assessment;.Outcomes of Education; Prograi

DescriptiOns; *Program Development; *Program

Effectiveness; Public Policy; Unemployment; Work

Experience Programs; *Youth Employment; !oath

w Programs ,

*

. IDENTIFIERS Job Corps; Private Industry; Ventures in Community

fInvoliement Project; Youth Community Conservation

.. Improvement Project; *Youth Employment and ' Demonstration Projects Act; Youth Employmeil Training

Progiam; YoUth Incentive Entitlement Pilot

Projects

.ABSTRACT

These proceedings of a conference on .1978 youth

knowledge development activities implemented under the Youth

.Employment and Demonstration Projects Act (YEDPA) of 1977 consist of

. the proceedings of the five sessions of the conference"and the texts

of 16'conference papers. DiscuSsed in the first section of the

proceedings are demonstrations and research conducted under. the . auspi es of the Youth Incentive Entitlement Pilot Projects (YIEPP),

the Y uth Community Conservation and Improvement-Prbjects. (ICCIP),

and t e Youth Employment and Training Pro4rams.(Y2TP); basic i . research; and data collection and evaluation. Topics covered .n the

conference 'reports include the effectiveness of various research and

job training programs, YEDPA research and interagency cooperation and

discord, ah economic analysis of the patterns and trends in youth . unemploythent, the potential impact bf employment and job training

programs on youth unemployment, the social and economic significance

of teenage Unemployment, youth experiences asvpathways into the world

of work, and the economic impact of the Job Corps. (Related youth

knowledge and development reports are available separately through

ERICbee note. (MN)

A
c

t

YOUTH KNOWLEDGE DEVELOPMENT REPORT 1.3,

Knowledge DeVelopment Under The

-Youth Initiatives -- Proceedings

Of An Overiew Conference

Brandeis University

April 1980

Ir

l'or sale h.% the Superintendent of Documents. U.S. Government Printrnf, °thee

Washington. D 1'. 20402

1) ' 4 iGr

US. DEPARTMENT OF EDUCATION

NATIONAL INSTITUTE OF EDUCATION

EDUCATIONAL RESOURCES INFORMATION

CgNTER (ERIC)

loolihts document has been reproduced as

received from the person or organization

ongtnating it.

4.[ I Minor changes have been made to improve

reproduction quality

Points of view or opinions stated .1 this docu- ment do not necessarily represent official NIE

position or policy
U S. Department of Labor

Ray Marshall, Secretary

Employment and Training Administraftion

Erfiest G.-Green, Assistant Secretary for

Employment'and Training Administration

Office of Youth Programs

Material contained in this publication is

in the public domain and may be

reproduced, fully or partially, without

perMission of the Federal-Government,

Source credit is requested but not

required. Permission is required only

to reproduce-any copyrighted material . contained herein 6
OVERVIEW

Congress and the Administration have made a.major commit-,

mient to expanding, and improving employment, training, and

career development services for youth. On August 5, 1977,

the president signed the Youth Emproyment and Remonstration

Projects Act (YEDPA) which created for new programs, add provided significant discretionary authority to test new approaches

for aiding youth. ,The new programs were subsequently emended

through fiscal 1980 and backed up sizeable appropriatidris.

In additiOn to these new efforts, the Job Corps program which

provides comprehensive services in'a residential setting for , severely disadvantaged 4outh has been doubled in size to

44,000 slots. Finally, the Summer. Youth Employment Program

(SYEP) has been reassessed and refocuSed with the goal of substantially increasing its effectiveness,- ' Knowledge development activities are a critical element in

these youth initiatives, Re seatch, -evaluation...and _demonstration effOrts reldtedto youth were a major. priority in

the 1960's'but have received declining attention in recent

years. While much was learned, many critical questions remain unanswered about the scale and nature of the problems

and the effectiveness of public interventions. A primary

aim of YEDpei was, therefore, to improve the information base

for publicioolicymaking. It was designed as a limited duration "demonstration act" which mandated a range, of experimental, research and evaluation.efforts. Other longstanding 0 youth programs shared this emphasis: "Job Corps had traditionally been a laboratory-for experimenting with new,

approaches to serve economically disadvantaged youth. With

expansion, there was a need for innovative approaches as well

as improved. assessment of current operations. Basic information was also lacking about SYtP and possible improvements.

Congress and the Administration wanted to know morahbeforq

developing and implementing youth policies, for the longer

term.

An Office of-Youth Programs was established in the Employment and Training Administration in July 1977 to implement

these youth initiatives. A major dimension of its missioV

was to develop and coordinate knowledge development activ- - ities for YEDPA, Job Corps and SYEP while serving.as a nexus'

for linking with youth-related efforts of other agencies.

_ In fiscal 1978, an ambitious` agenda of demonstration, research

and assessment activities was implemented. A Knowledge Development Plan for the Youth tmpioyment and, Demonstration projects

Act structured an array of ,YEDPA discretionary efforts. which

would address the most basic questions of concern to national

policymakers. These efforts included a massive experiment with

A A

0.
a

job guaranteed as well asialmost $100 million in action pro-

' grams designed to yield information about what works best

for youth.; Demonstration programs were mounted as part Of

'thb Job Corps expansion effort, and previously commissioned

4,research and evaluation work was completed. Researgh and

evaluation related to the 1978 summer program was signif-

. icantly broadened. With the extension of YEDPA programs

through 1980, knowledge development plans were also developed

for fiscal 1979 aQ,cil 1980.

The knowledge development activities for the three years

represent an aggregate of more than half a billion dollars.

While most of these resqurces are used to provide quality

,,services and emplqyment opportunities to youth, and while

they also ,are a.me9haniSm for institutional change as well . as for achieving equity, they have been carefully structured

so --as`-to -yield--much_raare_than_the usual information about

What works for whom. Reseaich, assessment and technical

afftlstance components which.do not yiOld direct' benefits

to youth amount to less than 2 percerit of total youth

resources; yet with maximum structuring of action programs,

this investment may yield substantial long-run knowle ge

benefits.

The spe fic demonstration programs, the research work, and

the dval ations are designed to address 15 basic questions, , which Were specified in the legislation or were discussed'in

the debate over future youth. employment policies:

1. Does school retention and completion increase'-the

future employability of potential dropouts and

the disadvantaged, and are employment and training

services linked to educationan effective mechanism

for increasing school tdtention and completion?

2.- Can the school-towork transition process be improved?

This involved several related questions. Are new

NLinstitutional arrangements°Teasible and warranted?

Will increased labor market information and assistance

expedite the transition? Can new transition routes

be created?

. 3. Given the fact that work experience has becime the

iptImary, emphasis of, youth prograMs, are the jobs

productive, .which ones are most "meaningful" and how

can. they be improved?

t

ri;
Does structured, disciplined work experience have as

much or more impact on future employability than othei

human resoiarce development services or a combinatio4

of services and employment, i.e., should public policy

emphasize' straight work experience, combifiefions ofwork apd training and other services, or should training,

education, and supportive services be emphasized?

5. Are there better approaches and delivery mechanisms

for the types of career development-,--employment-and

training services which are currently being offered?

o 6. To what. extent are 'short-rueinterVentions and optcomes

k,eefied to longer-term impacti on employability during

adulthood? Put in another way, how much can public

interventions redirect the developme tal process?

\'N,

- :Whatworks-best-for-. 0 or ouLLuEte

standards are best to determine what does and does not

work-for youth? Which youth with what characteristics

bpnefit from which programs'and approaches?

What is the universe of need for youth programs? What

is the cost of fully employing youth? How many would

take jobs if they were available and how many hours of

f N emplOymentdo they require? . 9. ,sihat approaches and procedurpscan be'used to involve

the privatt sector in employment and training efforts

and to increase the placement of the participants in

private sector jobs? How effective are these approach

f'n accessing new jobs and providing bettet career tracks

or youth? Are they preferable to public sector approaches?

10. mat is the. hest mix of enrollees in terms of age and

_income status? Will poor youth benefit from interaction

with nondisadvaritaged youth or with older persons? Is

targeting,achieved and is it at worthwhile notion? . 11. What arrangements can,be made to increase the duration

of emplbyment and 4raining, interventions and to assure

that participants realize lifetime benefits?. Will youth

demonstrate the commitment and cqnsistency to make these

long-term investments ipay;off?

12: What strategies are most important at different points ' in.the lives of youth? Must training be delayed until

greater maturity is achieved? Are employment and training programs a way of inducing maturity-?

At rp
,

. 13. How can separate youth programs be better integrated to

improve administration and to provide more comprehensive

services to youth? To what extent, are the programs already

iritegrated at the.local.level2

14. How do the problems of significant youth segments differ

including those of migrants,` rural youth, the {handicapped',

offenders, young women with ohildren, runaways and the

like? Are special needs groups and special problems

better handled by mainstreaming or be separate programs

for these groups?

15. How can the lessons' from knowledge development activities

best be transferred to improve existing youth

How can the institutional change process be pr oted?

What are the learning curves on new programs and how

much can they be expected to improve witl,time?

These broad questions subsume many more. They will not be relye& by any--s-ingle-set-o-f-ac-tivi-tries,---but,--rathe-r

of a synthesis of a range of coordinated efforts

including basic research, ,process evaluations, impaCt assess-7_

ments of regular program long-term followup, structured expert

. ments, demonstrations testing the feasibility of concepts( and

better identification of currently existing model programend

components.

The scale and scope of knowledge development activities are

staggering. Each demonstration, research and evaluation project . had to be,separately and carefully designed in Very short

order to get'the activities underway. The clock,was running

in the sense that YEDPA was initially for one year only and

even with its extension through 1980, results were needed

immediately to redesign youth policies for the 1980s.

a/

, . Given the scale, complexity and haste, it is im ortant that

-the elements of this knowledge development,eff rt be continuously examined, that coordination be achieved wherever

possible,\and the results be synthesized as they become available. ConStant improvements andoadjustments are required in `

research designs and it is necessary to assure that the results will be available in as timely a fashion as is. feasible'

to, address critical policy isses.

To thelp assure these results, the Office of Youth'PrOgrams

commissioned the Brandeis University Center for Public Servibe

to organize a Conference of 19,78 Youth Knowledge Development

Activities which would bring together the researchers on 1,

the major projects and many of the key players ultimately

invol'ed jn translating the findings inte policy. The two-'

day conference was held October 5 and 6, 1978. Short background papers were preparedwith little lead time;--tut the

iv
O

major purpose was to lay the\conceptualgroundwork for a continuing assessment and modification process. The proceedings

of a rather rively conference have beeh substantially edited

to focus on the major, issues. The background papers are in-

_ cluded with little pretense that they reflect any rigorous

analyses.

The results are_important_in_several_ways: -First-, there-is

a good deal of basic, information about the projects and the

related research. Second, the discussions suggest ways in

which the projects are-interrelated. Third, the needs for

greater coordination and more careful time- sequencing of results are made apparent. Fourth, the complexity of demonstration, evaluation and research in this area becomes clear.

Fifth; the varying perspectives within the research community

exe_auggeatedi_ap-Itiell.-ap-the-di-fferent wayc of approaching

problems. Sixth, a realistic sense of the limitations on research and evaluation activity is demonstrated. Seventh,

areas where more work is needed are identified. Eighth, the

interrelationship between information, knowledge and policy

is highlighted.

This volume will be useful for those seeking to get some understanding of the knowledge development activities beyond the

specifics in the knowledge development plans, for 1978-19110.

The volude is also a foundation for continuing assessment and

modification of these youth demonstration, evaluation and researgh activities. Never before,hAs.sucha.large scale,

diversified set of activities been undertaken to address e,

single social problem. The respltgwill yield much know- i ledge not only about youth_emg1oyment problems and policies,

but also'about the potentials of research, assessment and

policymaking in our complex society. A followup conference

in June 1980, Makin Sense of the Knowledge Development Findings, assesses the progress overthe-year And a hhlf since

this initlalemeetinge as well as setting the stage for continuing work on knowledge development. However, the first

volume is important for more than historical reasons because

it raises many of the key issues which must be considered in

dealing with the research, evaluation and demonstration

fiidings. 4

. 6

.

. 'l This volume is just one of. the products of/the "knowledge

development" effort implemented under the mandate of the

Youth Employment and DemonStration Projects-Adt. of 1977.

The knowledge development effort consists of hundreds of

separate research, evaluation and demonstration activities

Which will result in literally thousands of written products.

The activities have been structured from the outset so that

each is self7btanding but also interrelated with a host of

other activities. lhe framework is presented inIA Knowledge

1.
O

Development Plan for the Youth Employment and Demonstration

Projects Act of,1977, A Knowledge Development Plan for the

Youth Initiatives Fiscal 1979 and Complet,ing the Youth Agenda:

A Plan for Knowledge Development, Dissemination and Applicati4

in Fiscal 1980.

Information is available or will be coming available from the

various knowledge development activities to help resolve an

almost limitlesarray of issues, but answers to policy

questions will usually require, integration and synthesis

from a number of separate products, which, in_turn, will depend on knowledge and availability, of these products. A

major shortcoming of past research, evalbation and demonstration activity has been the failure to organize and disseminat

the products adequately to assure the full exploitation of the

findings. The magnitude and structure of the youthknowledge

development effort puts a premium on_arganizationarld-d-izs-ernination of findings.

As part 'of its knowledge-,development mandate, therefore, the

Office of Youth Programs of the Department of Labor will

organize, publish and disseminate the written products of.

all major research, evaluation and demonst4tion activities

supported directly by or mounted in conjunction with the

knowledge'deveiopment effort. Some of the same products

may alSo be published and disseminated t4ough other channels,

but they will be included in the structured series of Youth

Knowledge Development Reports in order to facilitate access

and integration.

The Youth Knowledge Development Reports, of which this is one,

are divided into twelve broad categories:

1, Knowledge Development Framework! The.products in

this category are concerned with the structure of knowledge

development activities, the assessment methodologies which

are employed, validation of measurement instruments, the

translation of knowledge into policy, and th6 strategy for

disseminating findings. , , 2. Research on-Youth Employment and'Employability Development: The products in this category represent analyses of

existing data, presentation of findings from new data sources,

special studies on dimensions of youth labor market problems

and policy analyses.

3. Program Evaluations: The products in this category

include impact, process and benefit-cost evaluations of youth

programs including the Summer Youth Employment Program, Job

Corps, the Young Adult Conservation Corps, YOuth Employment

and Training Programs, Youth Community Conervation and

Projects, rOjects, and the Targeted Jobs Tax Ci"edit.
4, Service and Participant Mix: The evaluatiots and amonstretion6 summarized in this category concern the matching of

different types of youth with.different service combinations.

This involves experiments with work vs. work-plus reniediation

vs. straight remediatibn as treatment options. It also includes

attempts to mix disadvantaged and more affluent participants, as

well as youth wifh'older workers.

.

.

. ... r 5. Education and Training Approaches: The products in this

category present the findings of structured experiments to test

the impact and effectiveness. of various education and vocational

training approaches including.specific education methodologies

for the disadvantaged, alternative education.and advanced

career training.

6. Pre-Employment and T ansition Services: The products in

',this-category-present-the-findings-of structured experiments-to

test the impact and effectilleness of school-to-work transition

activities, vocational exploration, job-search assistance and

other efforts to better prepare youth for. labor market success.

-

..0. 7. Youth Work Experience: The pro ucts in this category

address the organization of work ectivit/

es, their output, pro

ductive roles,for youth and the impacts of various employment

appro?es.

8. Implementation Issues: This category includes crosscutting analyses of the practical lessons concerning "how-to

do-it'." fgsue,s such as learning curves, replication processes

and programmatic "batting averages" will be addressed under . this- category, as well as the comparative advantages of alternative delivery agents. .. , 9. Design and Organizational Alternatives: The product

in this category represent assessments of demonstrations of

alternative program.2nd delivery arrangements such'as consolidation, year-round geparation for summer programming, the use

of incentives and multi-year tracking of individuals.

JO. Special Needs Groups: The products in this category

present findings on the special problems of and adaptations.

needed for significant segmerits includingminbrities, young

mothers, troubled'youth, Indochinese refugees and the handicapped. . Innovative Approaches: The products in this category

prebent the findings of those activities designed to explore

new approaches. The subjects covered include the Youth.

Incentive Entitlement Pilot Projects, pfivate sector initi -'

.atives, the national youth service experiment, and energy

initiatives inyeatherization, low-head hydroelectric dam

restoration, windpower and the like.

Vii
t

12. Institutional Linkages: The products in this category

will include studies of institutional arrangements-and linkages

as well as assessments of demonstration activities to encourage

such linkages with education, volunteer groups, drug abuse

agencies and the like.

In each of these knowledge ,development categories, there will

be a range of discrete demonstration, research and evaluation

activitiesr.focused on different policy, progtam and analytical

issues. For insta:Ice, all experimental demonstration projects

haveboth process and impact evaluations, frequently undertaken

by different evaluation agents; Findings will be published as

they become available so that there will usually be a series

of reports as evidence acCumuiates. To organize these pro-,

ducts, each publication is classified in one of the twelve

broad knowledge development categories, described in terms of

the more specific issue, activity or cluster of activities to

which it is addressed, with an identifier of the product and

what A----iresents relative to other products in the demonstrgiio,n. /Hence, the multiple products under a knowledge

development activity are closely interrelated and the activities in each broad cluster have significant interconnections.

#

This volume I is closely interrelated with all the other products.

in the "knowledge development framework" categOry. In particular, it should be read in conjunction with the report on the

follow-up conference; Makin 7 Sense of the Knowledge, Development

F,i.ndings as well as the basic conceptual'outlines in The Knowledge Development Agenda and the Office of Youth Programs

synthesis of findings in Youth Knowledge Development Summary.

ROBERT TAGGART

AdMinistrator

Office of Youth Programs

viii

4

*La
r

4

TABLE QF CONTENTS

Overview

PART CQNFERENCE PROCEEDINGS:

INTRODUCTORY SESSION

4 e

4 PAGE'

c-

-

SESS'ON Youth Incentive Entitlement 4ilot

Projects (YIEPP) -- Demonstration

and Research

SESSION II: Youth Community Conservation and

Improvement_Projects

10

Demonstration and Research 29'

SESSION III: Youth Employment and Training'

Programs (YETP)--Demonstration

sand Research' 45

SESSION IV: Basic Research . .

. _ 63

SESSION V: Data Collection and Evaluation . 73 /I

CONFERENCE SUMMARY AND A LOOK FORWARD 92

PART II: CONFERENCE PAPERS

Summary of Research NY7tivities on tkYouth

Incentive Entitlement Pilot projects 107

.L. V

Replication/Va-uation/Impact: The Ventlires:s

in Community Improvement Proje9t (VICI) 129

YEDPA Research Via Interagency Agreement

(and Disagreement): A Study of Conflicting

. Objectives 140

0
Learning By Try14 Identifying the Strengthg

of Exemplary In-SchookYouth Employment

Projects ' -

( BackgSund: The Purpose .of, the Propose'

DOWNIE-Field Test of.,.the Career Intern

'Program -. 1-53

. J

CPRV'sPrivate Sector Initiatives

Demonstration .

. 159

Project Overview. Statement- -Youth Collimunity,

Service Proiect- 166 _ Service. Mix Alternatives DemonstratiOn

Project . t

. . 171

146

An Economic Analysis of the Patterns and

Trends in Youth Unemployment

The Potential'Impact ofEmployment and

Training- Pkagrams on YOUth Unemployment . t

- 174.

1.82.

Theocial and Economic Significance of

Teenage Unemployment 0 . 184

/ Measurement of Work Outp4of Youth Programs'. 190

-.- Continuous Longitudinal Manpower Survey k

(CLiMs. of YEDPA Programg) .... . .. "- . 197

.t. . . PathWays Into the World of Work: Experiences'

of Youth 203

it.

Nati9nal Council on Emplofinent Policy Youth

Evaluation Project 209,

Economic Impact or Gob Corps .215

PART III: APPENDICES los_ CONFERENCE AGENDA

CONFERENCE PARTICIPANTS

.%

. . . 221

-1A;4 225

.
Ji

INTRODUCTORY SESSION

, MR. TAGGART: The Office of YouthPrograms has been looking forward

to this, conference with great expectation and also some trepidation.

We have been busy in the last year and a'half, distributing money,

developing grants, implementing research designs, moving enormous

mountains of paper. -We relish this notionof getting everybodytpgether, taking stock of what we've done and getting adVice from

some of the best experts in the country.

On the other hand, we'r'e also looki4g to this conference with

some degre'e of anxiety. The anxiety comes from the fact that we

are opening up your research efforts at Avery /early stage, looking

at-what's ongding instead of waiting uhtirl the results are completed.

The volume of activity is so great that we are not sure of the

"quality of what we have ongoing,. Our effort has been concentrated

on developing research and demonstration projects rather than

montioring, coordinating and integrating them.

The reason we in the research, evaluation andpolicymaking

business ar here is because Congress has challenged the Administration and the research community to come up with answers abosrt

how better to serve them, and, for the first time, has provided'

the resources,and the flexibility to meet that mandate.

(.; The Youth Employment Demonstration Project Act,was signed

by President Carter on august 5, l'977:--an Act which was initially

for one year and now is going to be extended two more years--

but one which was definitely a/short-term demonstration, Aat. It

provides massive demonstration'authority.and gives specific

research, demonstration, and evaluation mandates.

First, it creates what' is, I helieVe, the largest structured'

social exReriment in our history: Youth Incentive Entitlement

Pilot Prof,s which will eventually amount to about $306 million

in funding over three yers.

Secondly, under the YouthCommunity Conservation and Improvement t.

Projects portion of the bill, it gives discretionary authority to

test the "sweat theory",--that if you put kids to hard work under

careful supervision, with a stress on output, it will have an , impact on their life. '

ti Under another portion of the,bill, Youth Employment and

Training Programs, Mach will fund any of a broad range o? approaches,

the Secretary of Labor issgiven a large amount of ditbretionary

authority to test everything under the sun for youth. But this

is not all. At the same time, we're doubling the size pf Job

Corps. With the resources that are provided for this expansion,

there is the opportunity to use Job Corps. once again as a laboratory.

4

.7

1.
In Job Corps, you have more contiol, better records And more

tracking than in any other network. It is the only program where

we have an intensive investment in human resources fo'i youth. So,

we have possibilities there as well as under YEDPA.

, At the same time, it Was clear that Cong ess was no longer

satisfied with our knowledge of the Simmer P ogram for Economically

Disadvantaged Youth (SPEDY). We were reporting back year after

year about how4mAq ,outh we' served and not about tile 'impacts or

content.of services, Congress wanted to know what was happening,

whether the participants were going back to school, whether their

summer jobs were good and whether they were supervised.

4 Congress is asking us to tome up with answers to, questions

about SPEDY,.about Job Corps, and about the pDPA programs.

Accompanying these manda.Ces and requirements is a great deal of

money., , In pursuit of this,legislative'mandate, we have been supported

'by substantial commitment within the Administration from the highest

levels. When we first came into this effort, almost a year and a

half ago, there was a meeting at he Nhite HouSe in which all the

federal- agencies were called together and were told to cooperate,

to share resources, ideas and responsibilities in a coordinated effott.

I presume that happens in every administration, but in this case

thei.e was follow-through, containing commitments, and result6.

aps it means very little tQoutsiders, but we .have some 17

in ragency agreements under the youth initiatives. The Department

`of t borOlas worked cooperatively with almost every federal agency.

There's a Vice President's Task Force on-Youth EMployment that's

been established to ftrther this coordination, as well as effo2ts

within the White House. There,is going tole a, Presidential Policy

Review Memorandum., There is a very conscientious effort from the

top to pull this thing together and to Itorok at the research and

demonstration issues and the 'results as'an intergral part of the

policy'process.

a Under YEDPA, the Labor Department had the lead action. We

are lucky ta have a Secretary and Assistant Seretary who, are

very ,much committed to doing what is necessary in order to learn

what works and what doesn't. They have stood by that commitment.

For instance, they have stood by it when we were choosing Entitlement

sites by a competitive process which provided as .much as $20-mi1liOn.

to-e single area and where political pressures were simply enormous.

We had a Secretary and Assistant Secretary who didn't waiveron the

selecticin on the basis of objective criteria.

That's very impoitant in mounting any type of knowledge

development effort- At the same time you had a Secretary who

ha$ been willinq'to go forward with attempts at wage subsidy

experiments, knowing there were going to be reservations on the

part of the labor ntvement. He, stood 14' the experimental effort

because,he believed we have to learn what works and what doesn't.

2
An office of Youth Programs (OYP) was established in the

Employment and Training Administration specifically to emphasize,

research, demonstration, and evaluation activities, as well as to

coordinate and improve youth programs across the board. Early in

the game, we in OYP developed a Knowledge Development.Plan that

outlined all the efforts that we would launch and all the' issues

we would address. This plan was approved and every element has

been implemented. I doubt that ever before discretionary

activity has been planned so carefully or delivered so completely.

It's a testament to the support from the Secretary and the

Assistant Secretary for employmerit and training.

f.

You will be examining all the major pieces of 1978 discretionary

activity. There is now:a 1979 plan for another $215 million or so

coming down the pike. So we are talking about almost half a billipn

dollars ift:thediscretionary projects related to youth, all of which

have been structured .to learn more about wham works and what doesn't.

It's a massive array of research, evaluation, and demonstration.

What we will be reviewing in.the\,4ext two days are those elements

we had already initiated in fiscal 1978, and only the largest of . `those elements. But in the few minutes that I have I do not want to

talk about those projects, because I think the experts who will be on

the panels will be much-better prepares -to do that. I would rather

review # few of the broader issues.

Perhaps the "most critical' issue is why, after ,roughly 15 years

of evaluatii.on, demonstration and research activity related to youth;

we have a Bill in which Congress says we do not know/what works and

what doesn't, and that we haveto experiment in order to provide,the

answers. Given this pa'st record,- why do we,think that anything will - be different .15 years from now? Why do we think that we will

up wekthe answer this time around? Without being critical ofo f 4

past efforts, I think we can1all agree thatz,there. were some shortfalls- -

almost all of Which were unadyoidable7-which might be corrected.'

f'irst, because there were,fewer institutions that delivered

employment and training programs in the4arly 1960'S7, most of our

efforts in the'past decade had to dp with testing deA.very approaches,

institutions", and.servicepackages. We spent a large amount of

dollars on demonsttation projects-that were not meant to measure

impacts, but to assess the feasibility of ideas.

In effect there was no real research community in 1960. We

could not have called a group,like this together and have expected

the same wealth of experience. There were just not as many persons

out there with the talents that we have in this room. Also in 'the

1960's, there was very much less emphasis irk the administration or

in the Congress on testing, evaluation, assessment and utilization

of the results.in policymaking. /This Congress in particular seems

..to be directed toward wanting facts' and figureS and information

as a basis for policy: Thg administration has that same perspective,
contrasted with the Great Society of Lyndon Johnson, where evaluation

results were often a mechanism for selling programs rather than

assessing where dollars,should be allocated.

We, therefore,. had in the 1960's a number of isolated demi\istration

projects focusing on administrative feasibility issues with very little

pre- or post-testing. Even in the few isolated cases where results

were assessed, the sample sizes and demonstration conditions were tod

limited. We tried out 10 or 15 different projects with different ideas

rather than 10 or 15 projects all testing the same idea in order to

create a large enough sample size and large enough variability'of

conditions to find out whether or not a simple notion had promise.

., In the 19'60's, when evaluation and assessment efforts began, I

think we were naive about the complexities. One reason was the anticipation that the benefits of-social intervention would be so -large

tbat they'd overwhelm the technical uncertainties, such as the control

group selection issues. We believed that there would, be a 50 percent

return on our investments so that technical issues would be largely

irrelevant. When you are trying to assess a 5 or 10 percent return,

then the variabilities of those issues becomes cpiticallfimportant.

In the 1960's, when we did have control groups, we did very little

in the way of random assignments to alternative intervention. We,

always chased after the Holy Grail of assessing overall, rather than

relative impacts. The relative impacts, which are much easier to

test than the absolute ones, are probably more important for, policymaking where the alternative of doing nothing is rarely considered.

Pt was not until the end of the 1960's that we began 'to realize

the importance of the longitudinal aspects of impacts. Programs were

rarely followed up for 6 months, much less for 12or 18 months% Again,

it was not realized that longer-term impacts would have to be considered

to justify the investments..

The problem was that the longitudinal evaluations raised the

ante. It ,was so expensive to do a longitudinal evaluation and so

difficult to design the programs from the start with such a long -'

rangtk focus that this approach was rarely adopted. Except for the

National Longitudinal SUrvey and Supported Work, we were rarely

willing to spend the dollars f long-term follow-up and careful

experimentation. CETA--whatev4Wits delivery benefits--certainly

underminded knowledge .developments. CETA is an excellent prograpf

but it's very hard to evaluate what happens in a program in whi6tg,-,

you don't,have set categorical regulations defining sets of act-"

ivities that'are consistenacross areas. It is difficult to find

out what happens in.that decategorized box to determine the marginal

effects of different types of interventions. You alpost need categorical

apprpaches of some type if.you are going to do large demonstrations.

If you are not going to do large scale demonstrations, th n you fall

back on the scattered demonstrations which do not yieidslarge samples

OT indications of replicability.

17 - 4 - --411

ts
I think that another problemof the sixties, one that we'll

always face, is that the Congress and. the Administration frequently"

ignored or perverted the results of evaluations. In the Job Corps,

whidh is contained,in OYP, the war stories are legion about the

"good old days"--how teams would retire into the back room to prove

that the program worked and have it by the next day's testimony.

That's not the posture we can get away with any more. Nor would we

want to. Then, salesmenship was all-important. Accountability

Ia4ged by several years.

Other times, significant results were simply drloopped. For

instance, the Department of Labor conducted some admirable studies

on how to work with offenders--some of the best work that has ever

been done in this area. But we never got those results either into

CETA, or in what's been done in LEAA-OJJDP or anywhere else. The

fessoris did not translate'info policy.

At the end of the sixties, the attention simply turned away

from youth. There has been some research doie on other subjects

that is quite good, but the youth area was not programatically

popular and therefbre the research community followed the dollars

and public interests.

40

. The result was#that when Congress began to consider youth legislation in 1976-when it decided that the youth employment problem

was again a.priority--answers were not available to some really

fundamental i4sues. I do not mean refined answers; I mean even

"ballpark" estimates. For instance, althougp the overwhelming

number of persons 22 and under in CETA are in work experience components, no one can tell whether We're getting 10 cents of productive

output on.eaCh dollar of wages and sq.aries, or 50 cents or 100 cents.

If a mifor benefit of work experience-is social output and we cannot

provj.de any answers about the value of" output, then it's very hard

for,Congress'to make a deciSion On whether work experience programs

ar& wise or unwise investments.

,It is also questionable whether service approaches are worthwhile.

In the Youth Employment and Demonstration Project Act, there are really

two schools of thought expressed. Under Youth Employment and Training

Programs, every job we create has,to be "enriched,"--it has to 'be a

'career employment experience, with transition services,, efforts tb

overcome sex sXereotyping,'counseling, placement, occupational exposure,

and the like. Yet, anotperyart of YEDPA, Youth Community Coffservation

And Improvement Projects, -says you can offer no transition services,

that the aim is to have kids working hard, sweating, hui?dingi tangible

products. Thus, in Congress, there were two views about what worked

and what didn't work. Services do not offer a product other than

their impact on,youth, whereas/1n work experience the youthwho work

provi4de some product to society in terms of their output. In choosing

between work and service approaches, you have _got-tip know how much

the output is worth and how much the services impact upon future

employability. We don't know answers to either of those questions,

--5 1 tm

#4. L.)
Anothei'question is yhether, or the degree to which, resources

should be targetted. YEDPA was4not*a targeted, law. This Administration

has gone far beyond the legislatton through the regulations. A di

that consciously because that's tie policy that this Administrati

believes in, yet that belief is founded on equity, not on efficiency.

The law was very unclear. Some of it said_you shouldeerve the persons

who are from families with 85 percent or less of the' lower living

standard. Other parts of it said services or work should be available..

to youth whateverftheir family income. Other programs were to the

pooY only. The targeting didn't reflect just a lack of good legislation, it reflected different views about what the mix of enrollees

should be. In fact, the Department of Labor was directed to spend up

to 10 percewt of resources under Youth Employment and Training Programs

on experimelits to demonstrate whether disadvantaged youth benefited

by being in programs with non-disadvantaged youth: So there are those

who believe you should target and those that believe you'shoulcl mix

enrollments. We don't know thelenswer-to which is the most effective of

those strategies.

Finalky, we don't know whether youth should be in school or out

of school. Should you put your money in prevention' before dropping

out, should you put it in alternative education systems outs-dp

traditional schoaols, or should you put your money in non education

treatments for dropouts. All three approaches are mandated in the

legislation. There was no answer about the best intervention Point.

There was a basic question about how many youth really wanted

to work. Interpretations ranged across the map based,on the same

statistics. This is a critically important question. If youth

don't want tb work, then what do the unemployment numbers mean? If

you offer jobs, how many will lake them? How 'Pruch are personal

impediments, the :lack of desire or commitment to blame and how mph aWe

extewnal factors such as the simple lack of jobs? % We also haven't any answers about the substitutiontissue. This

is one that we're struggling with in public-service employment. We

have made some inroads, but there still remains a great deal of uncertainty about what added dollars buy.in terms of added Services.

HOw much do you add to employment when you create joips? Do the

workers displace others? If we are judging our effectiveness, then

these net efforts mist be determiped. We haVe absoltitelk no answer.

Another fundamental4question is whether an intervention cad, be

effective that lasts more than six or-nine months which seems,to be,,. 7

the longest of any of our interventions. In therjob Corps, which

can offer youth up to two years of ehrollmelV,. the average length of

stay is a*und six months. Is there any way to developia structured

series ofinterventions of longer duration that produce a 6antum leap,

in employability? Can we go against the grain of labor markets and

against the grain of the developmental process as well?

These are really fundamental questions and I AO not think we

have the answers--or even some best guesses-s-yet. 11.14ave mentioned
some of the reasons we do not have the answers and why we were charged

by Congress to learn more.. But what are we going to do specifically

to answer these questions? This is what we'll discuss in detail over

the next two days. At this junction, I only want to point out some

of the significant differences' relative to what was done in the 1960's.

,First and foremost, the delivery and research institutions are o * much more sophiSticated and much more developed. We can put a-massive

program like Entitlement in place with a careful research design almost

overnight because CETA prime sponsors are so sophisticated and because

there are research intermediaries. Entitlement was a massive implementation effort. In some cities, over 6,000 youth were employed over

a several month period. 'That's a massive phase-up--tripling the 'number

of youth mployed in the programs in these particular areas. iIt could

be done because,the delivery and research instilitions already existed.

Another important difference relative to the 1960's is that we

have a Congress and an Administration that's ready to carry out structured investigations. Congress has mandated an analytical posture

and s1pports the attempt. As a result of,this,mandate, it has been

possible to develop and implement a structured conceptual approach.

It May not be the best conceptual approach, but at least we tried to

set out a plan at the outset to coordinate disparate ;forts, so that

at the end of the_road they could be drawn back to et r. As we mover

to the second year of YEDPA, we are trying to tighten that plan, to

integrate, it even better, to fill in holes and to make it work. Because

SOYP has control over most yodth programs in a single location, it is

able to link Job Corps and SPEDY as well as YEDPA efforts in order to

look at youth programs as a whole.

Ir. I think that each element of this plan is much more structured

at the front-end than the average project in the 19(?-0e. ,I am

talking now in terms of demonstration programs that are multi-million

dollar initiatives. The scale has increased. Where before we would

do something and we'd end up with hundred enrollees= -50 controls and

50 experimentals--from which we would try to draw some conclusions,

now we're talking in terms o,f 500 or 5,000. That increase in enrollees

gives an enormous advantage and powerto the present analysis over the

analysis done in the 1960's.

I think there's also a much greater willingness to invest longitudinally than we have ever had in the past. We're investing very

substantial resources in the Continuous Longitudinal Manpower Sur\''ey

and in a new National Longitudinal Survey Panel which well hear more

abo.ut later. We're putting, an enormous amount of.money in the Entitlement

program withthe'hop 'of dOing a longitudinal tracking of the participaAs.

The research inves ment is larger to t!-ose in the fast.

i Another diffe nce is to look 'much more at alternative ervice

approaches, that is, to assign ,youth randomly'to cliff rent ervice

modalities and-then look at the outcomes. This doe not rely oh

absolute impact Measures and does not necessarily eed to determine

whether a program is ,a good:Ovestment4 rather it/tells whether one

intervention is more cost-effective than anothe

7

4U
I thinkwe have also been given the authority andresources to

cover all bases; One examine we'll talk about today is the school-towork transition demonstration. Here what you have are entry and exit

tests, followrup measures, control groups at-each site, and multiple

deliverers with multiple sites. You also have impact evaluations that

are based on very large data samples frIbm all. the. as well as

knowledge development efforts at each project, process evaluations,

and impact evaluatio4s. We can compare deliverers, different

sub-approaches, effectiveness within different client groups, impacts

in different economic circumstances, as well as estimated overall

effectivenesS. All bases are°covered even if sevetal.b not pan out.

,Will allithis work? Will we gain anythingrom it? We are already

seeing flaws in the designs. We are already seeing opportunities for

improvements. A lot of other questions come up daily thatwe would

like to have addressed. We have already seen a lot of shoddlawork--

research designs that break down in practice, as they always seem to

do. I think that in our discussions we must factor in these realities'

that out of every ten experiments you try, you may actually succeed

in following through on the experimF01 design in five br three, or

one. I don't kvoti whata good battigg average is, but it hasn't been

high in the pasf and we have to expect that and realize that we're

going to have a significant slippage rate.

Yet, we have also seen an enormous amount of highly technical,

high4, sophisticated work. A number of deliverers have shown

unexpected sophistication and a'number of 'approaches have yielded creative

structures. &iven that background, what we hope to get within these

next two olt.ys is a genuine opportunity for us to take stock Of what's

going on. There are thirty demonsti-ations in place, already involving

,SQ4 300 local, projects. They have been` describedabd_structured in,

thd KnMPledge Development Plan. Somehow, the results have to be

distilled. mhere are enormous opportunities to improve designs and,

to better_coordinate. If we're doing a set of pre-measures and postmeasures in ohe place; it-would be useful to have the same sets of . questions Dn the National Longiitudinal Surveyor the CLMS. We could

compare th results of entry-exit with the ov rall evaluatql,!tms that

have sampl sizes somuch larger and also have a longitudinal focus.

..,..,...-7 Where like prOjects are' being mounted in different areas, wip want

. /to have consistency of research design. That's a vety technicS1 aspect,

but it's something that we have to get to. What we'need today is to

have groups that are doing like things start talking-to each other;

starting a dialogue which will hopefully lead to integration oh a

continuing basis.

We have a chance to adjust research designS. When' we talk abbut

them today, when we.ask questions, these things are not set in concrete.

There can be adjustments over time.

I think we dlso have a chance to see how the pieces 'fit together,

that is, to see the overall conceptual framework. The only way we

are going to be able to pull the. results together in the future is by

sitting hery and talking abou4what. Jverybody else is doing. By doing

11.
I

this we can get a sense of the interrelationships in'ail this activity.

Perhaps the most critical*issug'is how we're goitrg to translate

all the information into knowledge and policy--how.we're going to

impact on. that and wh5n. This is:somethivg we must start thinking

about because policy cannot wait. The questions are going to be asked

and we have to provide whatever answers we can when they are needed.

I think another purpose for the oonferenceandone that's important,_

for ipe personally is to recognize the work that everybody has done in

mounting the program. We are'talking about obligating $45 million -

in structured demonstration, research and evaluation activities--

what must be the largest scale effort of this ,type in:history. We

are talking about massive amounts of work by yery few people.

Particulaly we're talking about Joe1Seiler and his staff in the

,Office of Community Youth Employment Programs.

I remember about a year and a half ago, at one morning,

when Jo& and I were writing this plan for demonstratiOn research and

:evaluation activities. He came running into my my ,office with a

great idea: " "Let's call.it a knowledge development plan." So, we.

labelled it knowledge development. Now Joe is affectionately known

as "Mi.. Knowledge Development." I-expect .ama hope we will find room

for improving the work 0 date,,but I think you will agree. that never

,has, so much been done by so- few.

4

e

y--

9

a

6

c
,

SESSION I: YOUTH INCENTIVE ENTITLEMENT PILOT PROJECTS (YIEPP)

PRESENTATIONS:`

MR. EVANS: apr first pane'l is perhaps the most integrated panel that

you'll hear because it deals with essentially a single program--Youth

Incentive Entitlement Pilot Projects.

Without further comments here, let me introduce Judy Gueron of

the MDRC, who iS/in charge Of research for YIEPP.

MS. GUERON: The key program feature to remember in thinking about

the Entitlement research design--the feature which distinguishes

Entitlement from all other employment and training legislation-- is

the job guarantee. Any.youth that meets the eligibility criteria

for Entitlement is guaranteed a part-time job during the school

year and a full-time job during the summer. The specific eligibility

criteria makeDntitlemept somewhat of an unusual job guarantee,

different, for example, than what's being discussed under the welfare

reform demonstrations. Under Entitlement the guarantee is conditioned

not only on income, but on age, residency, schodi\performanee_

and attendance. Specifically, Entitlement is a guaranteed job for

16 to.19 year-olds who-resiae in the communities where the demonstra:

tion is taking place, who live in families with income either below

poverty line or on welfare, youths who do not pave:high school diplomas

and who.agree to return to or continue in high schopl or a program

leading to aaequiva/ency, and who meet miN.mum academic and performance

requirements.

X The demonstration is baking place in only a limited-numbeir of

sites. There are seven Tier I programs. These are the larger programs

with anticipated enro lment from 3,500 to 10,000. They in e

entire cities of Cinci nati and Denver, and 18 counties .of Mississippi.

These are large areas. Then, tMre are ten smaidler Tie-71LN paths

where variations of E tit,lement Are being explored.

At the local level, CETA prime sponsors hNei operati4nal

tOponsibility for Entitlement, although some of the prime sponsors

have subcontracted direct operational management to othexagencies,

in the Locality. . At the nationAl'levei,* MDRC is responsible for planning,-development of guidelines, research, and 6ffering continuing guidance to

the prime sponsors in implementing the Entitlement-concept.

the demonstration ban enrollmept last'February. It was slow

in:the early months. InThpril there were about 15,000 youth enrQaled,

29j/p00 in June,. and, at. the latest count, 33,000. About 30,000 ofOeseWere'in the seven Tier I sites.'

In authorizing Entitlement, Congress 1sted a number of specific

questions that they wanted addressed. These are listed in detail in

,- the paper`-:" .The.issues can be grouped conveniently into those concerning program:implementation,,impact, and cost,

The purpose of the implementation analysis is to understand and

explain the programmatic development ofEntitlement sites and to draw

some lessons about the. operational, and administrative feasibility of

- 10 -
I

0 this approach r.--What dOes it ake to translate a legislative job

guarantee into an operation prograM? What's the ability of the

CETA prime sponsors to manage sucilva'job guarantee? What if the

importance of the different instil!utional arrangements, such as

those. between prime sponsors and school systems, and how is the

private sector,responding to the wage subsidy prov ion in Entitlement

which permits full reimbursement of wages inth iring of youth?

The impact analysis is really directed at testing whether the

program succeeds in its short and long-run objectivg. The major

short-run objective is to get dropouts to return o school and to

keep potential diopout's in school. In the long r run, it is hoped

that the increased education and the work expe fence provided by

Entitlement will lead to improved labor marke experience: Another

part,of the impact analysis addresses the qu tion if particlipation:,

how many yoAth will- want jobs if they are av liable?'

, "-- Finally, the cost analysis'is directe at proVa ing total cost

information- which, when combined wih wha we atta' on 'the partici-,

pation rate measures, will yield some e timates f what it would

cost to extend Entitlement nationwide.

This research effort is being'ca ried out by a number of organizations under MDRC's overall respon ibility. Researchers from ABT'

Associates, with Ernie Stromsdorfer s principal investigator and

Robert Jarrett as the project manay r, are responsible for the

impact analysis. Researchers from DRC under Joe Ball and BM

Diaz haVebeen working -on an 'mplementation analysis and the cost

Analysis is being designed.

1 MR. JARRETT: The impact analysis is focused on three p-4ncipal

, areas: The first is an examination of participation'in the programs.

The second focus is on short-term effects on youth eduCational

attainment, labor fofce status and earnings. The third is.'on the

ldnger term impact on youth--the post - program education, labor force

status, and earnings.

There are three principal sources of data: First, there is

a program information system set up by MDRC to collect routine data

on program treatments, 'on hours of work, wages, earnings, types of

jobs, etc. Second, there are testa from school records, concerning

attendance, tardiness, performance, and types of school programs

being pursued. Third, **and most important, there is a longitudinal

survey of program eligibles being conducted, at eight sites, fourof tl'e seven large Tier I sites (Cincinnati, Baltimore, Denver and

rural Mississippi) and comparison sites not participating in the

Entitlement. This survey sample covers all eligibio youths, not,

merely program participants. It means that we-not have a

treatment' group of program participants, at the pilot sites and

a comparison group at e parison sites, but we have a third group

of non-participant n the pilot sites. This design cleaily allows

us to sore ly get at outreach questions dealing with partn. It also gives us much stronger techniques for controlling

- 11 --

2:'
/

potential response and Site btases. The survey will yield a rich

data base for assessment of Entitlement, but for other research as

well. It includes approximately 8,000 youths and their parents. In

addition to the baseline, three subsequent waves are planned in 19'79

100, and'1981 so there will be a three year longitudinal data base.

The data set is fairly extensive, including socio-demographic

characteristics, detailed work; schb61 and training program

histories, information on attitudes toward work, education and

themselves, income sinformatior, both on the youths and their

families, work histories of parents and a limited set of fallily

interaction data.

To return to the principal researchnuestions, the participation

survey allows us to.address two specific questions posed by CongresS

in YEDPA, regarding levels of enrollment.and the number of youths

prbvided employment in relation to those who could have been

provided employement under the program. In one respect, this is a

very straightforward analysis. We will have information on the size

of tii74-total el- : popluations in our study sites and information

on the propo ion of these who participate in the program. It's a

relatively simple matter to deal with participation rates by different

characteristics of youths, different backgrounds of youths and to

talk about the participant population in contrast with the .nonparticipants. But it is much more complex-because participation

is not simply a binary variable7-joining,or not joining. It's a

continuous variable and some youths are going to,enter the program

and drop out,quickly, while others may participate intensively through

,high school. The extent of participation matters greatly. It will

affect impact and cost estimates. An 4applied participation model

must be developed that deals with participation as a continuous

. variable. Another complexity is that we're not looking at one

program going on during the school year. During the summer, youths

get full, -time employment and they're, normally not in school. During

the school year, they are in school and they're getting part -tire

employment. The. treatment is different. The range of opportunities

and options facing the youths as they make decisions about what

they're going to do, are different. Thus it'is necessary to keep,

track of the difference between summer periods and in-chool 4

periods in the analysis. Certainly in the participation. research

we.would expect participation rates to differ for summer and in-school

periods.

going to be doing a preliminary participation study,

'reviewing participatioiCduring the startup phase of the program,

during the last spring and summer.*A major report will be'produced

after the second wave of survey data has been completed and Abeived.

This survey is now scheduled for-the spring of 1979. We can't clo

a final reprort, really, until after the third wave of survey data is

received. That report would probably be available at the beginning

of 1981 and we'll have youths eligibleto join the program up to

close to that point. Our second wave of survey data will not capture

the last six to nine months of that potential dynamisni of entering

and, leaving the program. 0\

- 12 - 2
4. The short-term 'impact analysis was originally perceived as

two distinct studies, one on ip-school.impact's and the other on

employment impacts in the short-run. It quickly became evident tR,

us that these-really are two'principal dimensions of one larger

consideration- -the overall time allocation of eligible' youth.

Essentially youths have a'finite amouneof time'they can spend on

various activities and if they're.confrohted withan Entitlement

program which offers a job, .they have to make some choices about

how they're going to use their time'and abqut what's important

to them in their time. They have to divide between work, school,

school-re.)#cted study, extra curricular and leisure ,time,actiVites.

One would expect that there would"be a considerable nutber pf tradeoffs

made, with worki,ng.youth cutting back on studies or,leisure or

time in school. We are very inteerested in this dynamic'and the

behavioral rqpponses.of youth in budgeting their time in response to'

the program. We are develdping essentially._a, title allocation model . which will-generate the estimates of intermediate program impacts. B

he scheduled reports on in-school impacts essentially

parall lsithe participation reports with a, major report at the beginning of 1980 and a final report-coming at the beginning of,1981

when we've had a chance to observe the program running its full course.

Ve'will have a post-program labor supply and

employmdnt model and a model" to estimate the demand fob'- secondary

education. InAhese estimations, essentially the intermediate or

short-'termoUtpwts from the previous study become final program

inputs in the po'St-program models. Our thinking has also been / undergoing some evol'ution with regard to post-program impacts.

Previots* we tended to think post-program impacts as post-secondary

impacts with the.focuS on graduation from high school, continmation

of post secondary education and post - secondary eteirience in1the

labor market, as principal impact dependent varia es. We have,come

to realize, that we may haVe a population of youth in this program

who arein the program for a to ..period of time, who leave the

progra, but who are still in high school or in some fprma

I. secondary school. This can occur because the program is not

only school-conditioned, it's also age-conditioned. You leave the

program Whervyou graduate from high school, you also sgeave it when

you turn'20. years old, which shouldn't seem like much of a problem,

but, when 7,(ti consider that the program's primary target groups

are dropout or potential dropouts who, are frequently Older _than

their grade peers, it is likely that they.niaS, become ineligible

for the program while still in good standing in school and yet not

haying graduated. We don't know the extent that this Will'

"'occur. If it is a substantial group, we will have to broaden our

thinking of postmprogram imp3cts,4tb include essentially in-school

impact measures as long-term impact measures.

The post7program dy will be based on follow-uo waves,-=the

.third and fourth wave of longitudinal surVey. The study will, be

completed shortly after the completion of wave, our and will thus

be available very early in 1982.

104

r 43'-

1
44

There ire other possible effects of 6a ganging program. For

example, when Entitlement was getting st'rted, there was considerable

discussion about changing the eligibility criteria. A number of program operators wanted the,income standard raised from the poverty

threshoDO to 70% percent of the BLS Lower Living Standard. In some

cases, this was because they were simply having initial difficulties

enrolling youths; in many others it was because the Entitlement

poverty critera were different froth those used under CETA. If the

/

c angers had been made, they would have seriously jeopardived the

esearch because the youth samples had been drawn, and most of them

ad been interviewed. With the change, the sample would have been

o longer representative of the eligible population for the

rogram ,and all aspects of the research would have been jeopardized.

esearch considerations were.a primary reason for not responding

to the request that. the income criteria be changed. In the future,

however, other operational issues and pressures may arise which

affect research designs. Research may not always win out.

MR. DIAZ: The implementation analysis is interested in understanding and explaining the programmatic development o;othe Entitlement

demonstrati'bn at the sites in order to draw lessons concerning

operational feasibility, both generally and under varying

site circumstances. We are approaching this by first documenting what

occurs and then tryir to explain these results in perms of the

capacities and the interest of the operating agencies, their

accustomed ways of doing business, and their prior and current

operational relationships. We are also examining the effects of

local politiCal and social factors at the sites.

The implementation research begins with program description,

program content and operations, how many youths were enrolled, what

are the character tics of enrollees, what kind of work they do,

which agencies arinvolvdd and so on. Next, we'll be looking at

operational factors' that we think4Will have some effect on the

decision of youth to participate or not participate in the programs.

These, programmatic factors will obviously supplement the impact 11

analysis. Third, we're going to be looking at school-prime(

sponsor linkages under Entitlement;documenting, analyzing the roles

of these two primarylagents in the Entitlement offer.regthool

systems and prime sponsors have generally establi.shed a number ot

working relationships over the years and Entit1ement will provide

these two institutions with the opportunity and

necessity°to coordinate work experience and school services toie

greater extent than has usually been the case with in-school work

programs. We wantto see .how this relationship operates and why.

Fourth, we are going to,be examining the adapablility of the CETA

system to implementing a large-scale job guarantee concurrent

with ongoing and possibly competing program responsibilities.

Fifth, we will be looking at the Tier II innovations. Most of the

research is concentrated on T4er I. where the majority of the money

is being spent and youth will be Terticipating. However, we are

a - 14 -

V

s

ti
going to carry out a systematic documentation of the program., innovations.

such as entitling 19-to 25-year olds and working with unwed

'mothers.

,J. .. These five areas are being covered by the general implementation

research. They will be supplemented by special implementAtion studies

in three areas. The first is private sector work experience. We

are going to be doing a special study examining -the role of the

.,/ the private sector in the demonstration. Under Entitlement prime

sponsors. can and have provided up to 100 percent wage subsidy to

the private sector to provide work slots. We want to examine these

arrangements to determine their potential for assessing.aobs in 7

the private sector. Our second specia study will be examining

Entitlement in rural areas. One of t4 Tier I sites is in rural

Mississippi. Among the Tier II sites, Steuben County, 'New York,

Montefey California and' Alachua County, Florida, have large rural

areas within(them. We want .tf examine iuch things as job-creation, a

potential factor affecting youth, and other issues relating to

Entitlement in rural areas. Finally, we are also going to be looking

--at the quality of work under Entitlement, MDRC will 'survey a

sample of work sites to develop a profile of the quality of the

work experience of the different programs. The importance of the

entitlement work sites lies in their ability to provide an

exemplary work experience that develops and reinforces the podltive

'N\aspects of work for youth. Therefore, we tentatively indentified

five factors on which a sample of at least. 50 at each

Tier I site will be rated. These factors have been derived and adopted

from the literature on job quality. They are, (1) the content

of the work provided; (2) the organization and management of the work

site, (3) the level and nature of supervision (4) the youth perception of the value of the assignment, and (5) the value of the

work to the gponsoring agency or firm.

Tie implementation analysis must address fundamental questions

about the meaning of the Entitlement concept in operational terms. For

instance, the conditioned job guarahtee necessitates tight

monitoring of eligibility. An open-ended Entitlement where anyone

eligible can theoretically demand a job'combined with projections

of participation that are bound to be off because of the upredictability of youth response to a new program, means that both

funding agencies and prime sponsors have'to be concerned about .-!, keeping the Entitlement str9ctly limited to eligibles: This

requires the creation of nets systems for monitoring eligibility. It

means stringent enforcement of requirements for proof of eligibilityin the form of supporting documents. Inevitably, such

monitoring and,proof requirements may also act as a discouragement

to participation for inaivkduals. Parents, for example, for a sense

of privacy, may be unwillingto reveal income information, or they

may find income statements too complicated.

Another operational issue results from the school-c ditioned

aspect of the job guarantee. There is a need to establish school )

performance and attendance starAards., It means little if udents

,010. return or stay in school simply in a pro forma'way without'

- 15 -
progress toward a degree as intended by the YIEPP. Early efforts'

reveal that school attendance and grade standards governing

school participation for the general school population generally

on't exist. This is due to the general reluctance of schools to

uspend or xpel students except for some overt act, such as

hitting a teacher or committing some other crime. Therefore;

most prime sponsors have had to negotiate with schools for the creation

of grade and attendance standards specifically for Entitlement eligibles.

Not only was this Process time consuming but the standards are

frequently vague or ignored.

Thirdly, Entitlement as a job guarantee implies an operational

ability to quickly provide the guarantee, a job to those who come

forward. A large-scale time limited work program would also carry

some urgency to its implementation. This seems particularly acute

in Entitlement, however, since the program has visibility, For the

most part, prime sponsors have attempted to recruit eligibles as

rapidly as possible to reach protected levels of participatipn

and with some Nexceptions high enrollments were quickly reached

once the program got underway,but especially at the Tier I sites,

there were delays in processing from enrollment to placement.

This happened usually because the large numbers enrolled were

too much forthe processing system. Speed seems to be given pridnity

over preparedness. Our analySis must separate such start-up

problems from those likely to affect continuing programs, as well

as the long term effects of too rapid a phase-up in some sites.

Finally, there s a question of how the job guarantee

is perceived by its potential clients. There are incentives at

work in the Entitlement program against advertising the program

as a gu'aranteed job. One was a fear of being overwhelmed by a sudden

demand from youths. for jobs. At sites where less than a

complete jurisdiction was entitled, there was concern about the

reactions of non-entitled economically disadvantage areas making

their dismay and frustration known. Generally, therefore, primes

did not stress a right to a job-in,,spreading the word.

MR. BALL: First hand observation tends to separate rhetdric

from reality. The idea of'an Entitlement as spelled out in the

legislation and in the various kind of application procedures is

somewhat different than the reality to date for prime sponsors

which has been primarily'getting kids and a lot of slots in

a big hurry. What did get advertised was that this is a big job

Progrhm and if you're poor, you can get a job. Prime sponsors

went about business as usual in some cases. Experience with the always

rapid buildup of SPED and with very rapid phase-up of the

PSE program in 1977 was us6d and the primes generally resorted to

old routines to get the program started. With theSe mechanisms

they tended to be able to do it reasonably well in ,terms of getting

enought slots for the kids. It was helped along by the reqpirement

in the competition that final applications submitted to the

Department in December of 1977 had to have a project sheet for every

work sponsor specifying how many kids, what kind of work,

4

- 16 - 'f
what kind of supervisory arrangements and so forth, so there hOd

to be paper commitments to cover every projected enrollee.

In general, the program implementation was not impeded

by a shortage of slots. The result, however, was a heavy .reliance

on public and non-profit agency sponsors, most typically those

which had run summer or in-school programs. Potential for privatesector involvement was not fully edloited.. In-school programs,

co-op programs that the schools had run, of course, invoAved private

industry on a small scalp for years and years, but this was the

first big opportunity for a- major private sector participation with

youth pork experience. The prime sponsors d4ri't have close working

relationships with the. private sector. Th refore, they typic411y went

to intermediaries like Chambers of es, the National Alliance

of Business, and in some cases organizations that had been created

around problems of youth unemployment in the past, such as, in

Cincinnati, the Citizen Committee for Youth. In the big' Tier I

programs, Denver was the one that made the biggest emphasi.S on the

private sector buildup. By the end of June, they didn't have 50

percent, but they had almost 30'percent of their kids in the private

sector. ,, Some of the smaller Tier II programs, which had the luxury of

being small also pushed for a priVate sector role. In%Philadelphia,

70 percent of the kids at the end of June were working in the private

sector and a 4rge percent of their were working in manufacturing,

which is a little bit exceptional for the demonstration as a whole

where there's a lot more small retailefirm involvement. Hillsborough . County, New Hampshire, which is a_healthy economy, with light

industrial base, decided to take advantage of the demand situation

there to try to create jobs above the minimum wage with some skill

development potential above entry level. The jobs that have been

1,created there have been primakily in the private sector and several

of the kids are working above the minimum wage. The prime sponsors

that turned to the private sector; without exception, offered the

full 100 percent wage subsidy. Prime sponsors argued that the

business community hadn't been involved with CETA and particularly

kids and that unless the offer were sweetened to 100 percent,

they would not had been able to get tenough private firms to

participate. In the process of negotidting grants with the Department,

the primes were 'required to start. thinking about and develop plans_

for the reduction of, that subsidierlevel over time to see if they

-couldn't begin to move toward Cyst sharing arrangement ukith private

firms that would be somewhat more in.li

I

e with the 50,percent OJT

X

rule of thumb. The prime sp.onsorslhave een rather reluctant to

/ change the deal in adds-Ore/ during ....e oeriod of original

commitment and we will see whether, new arrangements can be negotiated

when grants-are extended. ......

a

f)

- 17 -

r

(
4 One price\that may have been paid for the rapid start up is the

limited involvement of unions. The legislation authorizes and

encourages primes to work with unions, to develop appreniticeship

positions for kids. They haven't done it. In general, prime

sponsors regarded this as a secondary and future m2Aon. This is not

to say, however, that the prime sponsors entirely avoided the unions.

The legislation says that if kids are put into jobs which may

be a,potential displacement,orimight undercut prevailing wages, than

the /6sitions had to be cl red with local organized labor. The'

same thing occurred where jobs were restructured. Further

cooperation of unionspwa one criteria in the grant applications.

Strong efforts were mad to clear with unions. At those sites where

the unions, especially public unions were still restive over

problems created.by the 1977 PSE build up, formal arrangements were

made whereby they reviewed every job description' and participated

in restructuring. In Cincinnati a representative from the union

local monitors work sites continually, writes up his own

monitoring reports, and indicates whether or not they are'in 1/117

way infringing on union interest'. So, the unions have been

involved in a kind of "damage avoidance" way.

-To date, then, the program has not developed a lot of options.

The kids are,paid the minimum wage. There are 22 kids out of 30,000

at this point who are being-paid above the minimum. There will be

some efforts made in some of the programs to try to induce the

private firms-to pay more, but it's basically a minimum wage

program. Ip addition, there's very little skill-specific, formal

classroom !raining. It's a pretty barebone work experience

program.

Yet it is early and we have just finished looking at a program

in its startup phase. This has been very tough in the Tier I areas

where the massive workload forced a lot of program quality

questions°to the back burner. Attempts to get the schools more

actively involved in tying their efforts with the work side of the

,situtation, were not pressed. In part, that was a function of the

desigkof the funding structure to begin,with in which the money came

throug e prime sponsors and it was not much coming through,' education -

channels. Mostly, it wa bet se oi.limited time.

% ,The,Department has recognized the need and the desirability of

encouraging the schools to do more in enriching the treatment and

there probably will be tome money available'to develop some Pilot.

efforts to expand education.

To date however, the schools have not been too active.

They were asked to cooperate certifying,kids being in school

and reporting on .ttendance. They have been asked to be flexible

if possible in scheduling so that participants could work after .

school. Although last spring they were not very flexible, there is

0

- 18 -
some hope that this fall, with a little more lead time, hey,will

be a li tle more adaptive. SO faf observers are not very encouraged

that the schools are going to make a big effort to bend their ways

to edapt to this program.

Ov rall then, the program has focused on creating jobs to

guar ee employment. The measure of the performance will come

in t is Aar as prime sponsors have more time, to involve the

private sector and unions, to work with schools, and to seek

enrichments and linkages.

DISCUSSION:

MR. STROMSDORFER: The Most distinguishing characteristic of

YIEPP and it's analyses is that you an observe the entire process

of the program operation and impact. There is sequence of events

that begins when you start setting monies down in areas and'there

is'a logical structure of actions that fcalow..Thei'e is

a parallel structure of analysis. The key policy questionsthat

Congress asks have the same time sequence. I've never seen this type

of correlation written so well into the law and implemented in an

operating program. Each new step is related to behavior that went

before and answers a successively' more difficult policy question.

I would also like to point out that given this close correlation

between the sequence of events, the policy questions as they develop

as well as the analysis, that the data that are generated are eminently

practical.

MS. HIGGINS: Over the next year the Administration is

going to undertake a complete policy review concerning youth employAemt with focus, on what-we're learning from these YEDPA

activities. The aim' is to get answers both for budgetary

purposes and for future' legislation about what kind of ptogramS

make sense.

The Entitlement program involves most of the issues that need

to be addressed.. On the marco.slevel, we first need to know how

many people are in the universe pf need. The Bureau of Labor

Statistics, OMB, the Department of Labor and others may argue over

the same set of numbers but there is a chance to find out who's

outs ther4, how many there aie,'and how many want jobs. That's

critically important.

The other dimension of importance is the essential bondage

betweep work and education. Everybody talks about the importatkce of

linking the employement and hiring and education, sys,tems, but

nobody has really determined whether it's feasible and whether it

makes any difference in outcomes for youth. So, I think we'll

find out some of the answers under Entitlement. The feasiblity issues

are crucial*. Can the prime sOnsors and the schools deliver this

kind of program? Can.the jobs be created and4will they be

meaningful? Will they be .linked to education?

Other questions will, also be addressed with crucial implications.

How can eligibility be determined?How can alternate education systems

be structured? In brief, Entitlement is a critically important

experiment which.will have widespread policy. implications.

p

"44.,
OP' .MR. LERMAN. I know coming frim the Welfare Reform Task

Force, where we're thinking about some of the same questions that

entitlements do raise special issues. It's good to see that some of

the questions will be raised, for us when we go through our welfare

reform pilot projects. Some of them will even4pe answered. I'd also

like to emphasize the targeting implications. If you look at the

unemployment rates for disadvantaged youtN.and you look at the gap

between their employment and that of advantaged youth, the gap is

greater for in-school than out-of-school youth. I recall some

numbers of the following orderblack disadvantaged students had

an employment/p4pulation ratio of.about six percent as compared to

about 36-38 percent for advantaged students.-So, Entitlement

is addressing perhaps that group in society with the highest job

barriers and a problem which is critical.

4 The second general point I'd,like to make is that the Congressional

mandate is highly specific, relative to.the mandate for other kinds

of studies and other kinds of programs. The result is that a lot of

the research design has to be directed just at the Congressional

mandated issues and notat what we might set out here and think are

important issues.

At the same time, it seems to me that some other important

issues are left aside and let me just note a couple of them,

because they haven't received too much emphasis. One is the eligibility

conditions for such an entitlement. Lt's one thing to have a

separate demonstration program, but in going to a nationwide

entitlement, broader equity issues are raised. When you have a pure

incomeEest at a certain point in time for entry into a job

' entitlement program, it tends to create the kind of notch that those

of us who have looked at welfare programs have noticed and decried,

that is, the notion that a family which has an income a few dollars

above the standard are fully out of the program,.while those just

below are'fully in the program. When you consider going to

national level, an issue like this becomes very important.

Another issue is how you best structure awage subsidy.

It's the kind of issue that you'd like to know more about. NOw,-with

respect to the three areas--impact, cost, and implementation--let

me begin with'the third one, implementation. I think it's

important that we study the processes and the barriers when we start

to phase in a program, but I would like to see more structured

;questions about what specific aspects of implementation should

be assessed. The notion of delivery capacity is just a little

too general fo me.-1'd like a question having to do with what kinds

of waiting lists we observe or_ for how long a, period. t would like

the dimension of private sector,involvemeret to to be studied and

laid out in detail. To say we should have some private sector involVement

where poll stafted out with a hundred percent subsidy, does not

ku

answe most of. the important questions. It would be surprising if

you co ldn't get people to employ youth with a hundred percent

subsidy. The question might instead be what happens when you

have a 50 percent subSi, or a 75 percept subsidy,as opposed to a

100 percent subsidy. Also;

dV

t ere ar. k questions about the extent of

private sector involvement b yound just employment.

-20- 3j.
eeMoving to the cost side,there has been little discussion

about the problems of measuring. costs. You know, when you first-think

about it, you say, "Okay, we're going to see ,what the costs re."

It's a lot more tricky than that, although I'm not sure the "ins

and outs," I would haye liked to have seen'a little6re discussion

today on that and whether or not there's going to be- questions

like the cost per site,and the factors differentiating low-cost

and high-cost sites. Secondly, .I would like to See.a reference to

social costs as opposed4to budget costs. Here, there seems to be

a very naturaltway of drawing the time allocation model into the

analysis, whereby we can see what we,are diverting youths-from, ` '.they are drawn into the programs- Do the- hours of

participation come from pure leisure, unOroductive or counterproductive activities or from school studies and other- things that

are unthinkable?

Moving on to the impact assessment, let me'.j:ust say that one

of the p oblems that I have with the analysis is that there is

an impli it emphasis on the notion of a job versus unemployment as

k

compared to a job versus other jobs. Only at the very end was the

mention of minimum wages pointed out. These are minimum wage.jobs,

but the problem goes beyond that because when you think of the wage,

you think of earnings divided by hours worked. Now, we may have

a good idea of what the earning are, but we've alsosgot to know

what the real hours worked are. This concern is being voiced . very heavily by a member of the Council .of Economic Advisors staff

with respect to the welfare reform proposals. To See whether these .

wages are really minimum wages, we have to compa.e. these jobs to

other jobs to check the hours worked and the effort exerted. I think you also have to ask about relative job content.,,

Another aspect of the wage and job nature question is that even

though the eligibility is based on welfare participation or

poverty, nothing is mentioned about the welfare marginal tax rates

and whether or not families whose youths are in these progra'ms

have that income counted for welfare purposes and maybe food

stamp purposes, thus producing low net gain. This might be a

reason why we observe differences in partidipation froth one group

to another. I think that's a very important question to look at.

Turning to the participation question, we've got to look at

the degree of participation, rather than just lboking at the participants

relative to eligibles. We must consider the flows into and out,of the

program, as well as the equilibriums. What about r epeaters?Is

it easy to flow out.and then flow back in: That's .the kind

of question that you want to ask. I've been some interesting. numbers

lately that seem to'indicate that actual turnover between jobs

is close to 1,00 percent a quarter for black teenagers and that a even for white teenagers, it's extremely hign. So, if we observe

very low turnover in these jobs, it may mean that these jobs are

better than the other kinds of work you would have gotten'or that .

stability is a very, very important charadteristic of jobs. We

oug,ht to ask, is this a way to rtiluee youth'unemployment, that is,

to try to improve the stability'rof employment.

- 21 -
A second problem in loo ingat particiPationflealsiwith the question

of eligibility. When you take a sample ..in a site of'a poverty , population, you have to recognize that people are moving in and . out ofkovertlr. The number of par_ ticipants ovet_time cannot

just be compareoL with the number eligible at a point in time. 'Theflow .out may noerequal the flow In in a period when the economy

is improving, so that U may over -state the number of eligibles,

simply because ,you'e taken the survey in oneeyear and you're 16.

, extrapolatin t anothevear. - The impact assessment is)-also based on the notion that the job

program has no impact on the eligible non-participants, that is, by

adding a huge number of part-time jobs to the supply of those jobs

and concentrating on low-income ybuth, you, don't affeCt those lawincome youth that don't go into the'program. I would expect

just the opposite.

.

, I would like to see more on how many are drawn from unemployed

and from those not in the labor force when we attract people into

these jobs. This is important for generalizing from available

aggregate census data.if we had asked questions similar to thoge

in the sgniuT, how many are we drawing from people who would

otherwise-hay been out of the labor force? What is the nature

of the attrac ion of thE4e jobs to the people who are normally

out of the la or force?

I also would like .to see something on do we reach appropriate

. . , targets and what are appropriate targells for employment population

ratios for these youths. Where Entitlement exists, are the poor

youth in a, much better emPloyment, situtation than the relatikr,ely

4 advantaged youth or are they.,,stpl in about the same position or are

they still/ in an even lOwer position ?, I think that's a,question

that many people are concerned about when., t'hey're asking what

Would be n appropriate target in terms of increasing the jobs for

poor youtH. °

i 4 Finally, we'must ,a--k aboukpe-infIation impacts °of a job = ., guarantee program. Many people argue.that it will be minimal: I ° believe that in, those market segments where there is"considerable,

excess supply, what"happenswhen you add to the number of jobs is

simply an increase ill employment and not much increase in labor

cost. When you're talking abour'expanding'to,an Entitltment program A

for the nation 'as a Oole, I think this question would be .a very

important one. . . , q

ra 22
MR. BALL: Mr. Lerman raised questions about hours of work

and time allocation. In the manpower ant1Cth, manpower' research

industries, trig numbers seem toitake on a kind of intrinsic value

and, in that vein, 1-irough the end of June, kidsin Baltimore

have been paid fo4936,887 hours of work;-givetor take a 'few. ,ThiS

'ft doesn't speak to whether or not they're working, of course. We

have to deal with 'what the prime sponsor reported as', pay time. ' We're planning, however, with the'monitors we have .in4"Tier

to monitor well over ten percent of work sites with'an instrigent

which is still in open developmental process; but Whioh we are

... fielding now. We expect to start 'getting :to about.7)),wor - sites a month, where we are going to address fhe q uestiqh of the . quality of the work experience, whether or not the kids are-.working

and whether or not there are apparently too many kids'Ecietive work

that's entailed. We're not going to attempt to do a quantitagi,

assessment, but we are going to monitor these work sites fairly°,.*

closely. 3 4. ,.... MR. FARKAS: On the self-selection issue, we're ,going to tty a

variety of things to pin down the extent oCpossible self-selection

. bias. There are relatively standard econometric techniques which . will be applied to estimate bias and make-corrections. If we had

: instr mental variables, we would,have used them but it isn't

cleat at the outset which would be the critical factors. We will.do

a pr equation and we will try to be clear about what the omitted

.variables might be` as well as the possible direction and magnitude

of bias. That's about as good as anyone can do.

MR. STROMSDORFER: I think we're going to do a little bit better, 1 but ti-Nre are inherent problems. What's lacking is knowledge about

the inflyence of d and side elements on,supply of,labor and behavior.

t5

We can control se what hat by,getting information on demand character- istics of-the la or market and we ?lave the benefit of having . --4.---.. both a control labor market and an experimental labormarket,, where

ycu have participants and non-participants who are eligible.

The issue is how well can we model the differential supply and

demand effects. That's hard to do. We don't have the necessary

statistipseand.detail that we would like to have.

MR. PACKER:I just want to emphasize the need-'for structuring

.*e specific questions at the outset of the reasearch. I hope that

somebody'has sat down and laid out the hypotheses that are going tdbe'rejected Or accepted. We believe _the program will ,mean that

disadilantaged youth who meet the eligibility requirements will be

more likely to graduate from high school because of the jolg.

-'guarantee than if it were not present. Can you tell me nowt what

.difference in graduation you expect between the controlled sites and

the test sites--that-is what difference would allow you to reject

the null hypOthesis that it doesn't make any differen&e whasoever.

I hope somebody's,gone through and can tell me what are tie ten

most important hypothesesthat you are testing and what th ,*

standards are for acceptance or rejection.: It is ot enough to

say that we are going to get a,data base or are g ing to know lotS

of'things about the program. What are the ten most important hypothese

that you're going to, put most of yourintellectual filpancial

resources? For instance, how many youngsters ar.e,:going to participate

in the target population? That question has to translate into a

hypothesis. If we're very Sophisticatiga, we might. be able to

have a quantitative estimate. If the unemployment ratio between

-% 23

' 0
Ar, advantaged and disadvantaged youth is now one to six, then the

program will move it up to two to six, with a confidence limit of

x and "so forth.

We must be sure when we're spending the public's money, that

we have a scientifically solid approach and the ability to accept or

reject specific hypotheses which are'recognized as the ones that'are

implicit in what policymakers are doing and thinking about in such

programs. Be sure of the policy questions. If I find out that

when you 'double that employment population ratio from one to six

to two to six, or when you improve the graduation record from one in

four 4o two in four, and you did it because the best kids out of the

eligible population participated- -that is, self-selection bias.

____Howeveit-,m-not--sure that the Congress will find that that's

sufficient reason to not have the program. If you could say,,in

. a sense, that if you had this Entitlement program, you'd eliminate

the difference between the employnen and graduation experiene

minorities, disadvantaged youth and the rest of thk population,

I don't think anxbody would give a damn about self-selection

bi&s. If at the end of the Entitlement program, you can't tell me

whethdr you've rejected that hypothesis that it makes a difference

inthose two things that I talked about, I think there will be a sense

that we've not done the research in the manner that was truly

intended.

MS. GUERON: Initial design of the research was based on a

number hypotheses, including the two you just stated. Will the

program decrease -the drop-out rate of studentsthat are already

in school and will it have any4impact on their future employment,

"-after they graduate from school, as well as their future

education? We do not want to end up with hypotheses that have been

set out but never answered. In fact, the legislation passed by

Congress was itself quite specific abott the questions it wanted

answered and the hypotheses it wanted tested.

MR: TAGGART:The thrust of the issue is not with sample sizes

and confidence levels alone, but in the structuring of hypotheses

that .say if the impact is of a given scale and dimension the hypotheses can be rejected or-accepted. I think tlfiS'question of how

you formulate these hypotheses, as well as of how you follow through

to test them speaks to the fundamental difference between

statistically significant and policy significant results.

MS. GUERON: What we had not done, I think Mr. Packer is asking

to have done," is to formulate it in terms of policy significant

impact expectations. If, we find that the-results of a given magnitude

ate documented, what then is.the policy action that this

magnitud% of impact justifies? We should not just come afterward

and say: "Well,_yes there was some impact on this;" and then make a

polijudgement. Mild we look at those issues doeforehand and

decide What results would be large enough to reject or accept

conclusions?

MR..PACKER: don't want tojush that too far. Many of thssucs

are statistical. When will you b ableto tell uS whether the number

Yof kids who come back to school, this month, is statistically

different, than one might have anticipated without the program? Are

more kids enrolled in their senior year who might have dropped

out at the end of their junior year? Will you know in a couple

of months or when? .

24 -

3fr,*4
MS. GUERON: No, all that we know, until we conduct a second

survey, is-what we can gather from an information system that

tells us what's gone on at program sites:. We will know how many

dropouts returned and how many in-schObl kids are re- enrolled,

but we-don't know the net effects,on those out of the program.The

second survey, which will be conducted in the late spring or early

fall, will allow a comparison between what's happened in the Entitle- . -ment site and what's happened in the control site.

MR. PARKER: Will the results be available by the time legislation must be drafted next year? January 1980 is the scheduled

date for submitting administrative recommendations. When the

budget goes up a year from this.January, if we're going to ask for

any money to continue youth progiams, the legislation has got to

accompany the budget. So it makes a difference whether the survey

is in-the spring or the fall. If you think the research is going to

affect the next youth bill, then we've got to be writing that

legislation a year from now to accompany the budget that goes up and

requests money. So it's not a matter of ind4fference whether we have

a sprr4 of fall survey.

MS. GUERON: We ha& assumed very recent y that we were conducting

the survey next spring. As it turns out e baseline survey

was delayed, and we haven't yet gott the baseline survey results.

MR. .PARKER: Why do I heed a baseline survey? Why can't I

ask the school system tomorrow? I meane it's October, kids have reenrolled in school. I don't need to know their lifeline. I just want

to know if kids who are eligible seem to be enrolled more in the

test sites than in the control sites.

MS. GUERON: You can do it tomorrow, if you want to answer the

question with that type of mTthodology.

MR. PARKER: The legislation must be ready in January 1980 and

I don't think we're going to be able to tell them we spent Xhundreds of millions of dollars, but we have problems in the field,

and you',11 have to write the legislation on the basis of the information you had before.

MR. STROMSDORFER: You've been asking two questions,

A though;

You've been 'asking, 'That's the grdss effect, 'and what's the net

effect?" The first-question is easier to answer than the second.

You don't get net without a control group or until you have the

second survey results. K MR. PARKER: I heard Mr. Taggart say we had not frozen everything yet in the program design questions and I think perhapS

that's something to be debated. Maybe we could do some other

things. It might pay to do a few little samples. Right now, I

mean, I think one can do that for a very small piece of the money

with a limited survey instrument in'order to get results for

January 1980. I' guess the one thing that I would like to see,

soon after this conference, is the policy hypotheses that would

be accepted or rejected by next December 1.

- 25 -

Q0'Jo
$

MR. HAUSMAN: Ts it your impresgion at this point.that the

program is getting at the targeted groups -- the people who are

difficult both to get back to school and into employment?

MR. BALL: It's my sense that early participation in March,

April, May .and June of 1978 consisted mostly of in-school kids,

because thdy were the easiest to reach in the phase up which

started in mid-semester. What's going to be happening September,,

October, we'll shortly know.

MR. LERMAN: Is there really a job guarantee? Are Jerewaiting lines? -Are there shortages of jobs?

MR. BALL: There weren't shortages of jobs up through the

end of June., but in the summertime, there may have been. W(e.....,know,

based on our statistic's, that there have been waiting lists, for the

jobs, but this had more to do in the phase-up with implementation

problems thanijob shortages. Kids had to wait two to four weeks

at a lot of the sites. Some of that may have had a discouraging

effect.

MR. LERMAN: In most of the sites, were there more jobs than

there were people interested in them?

MR. BALL: There were more job commitments that looked like

they could bet. operationalized effectively than there were kids

bangingon doo's through June. But:there's a political twist to

this question of whether there is a job guarantee. The prime sponsors

where entitlement covered their whole political jurisdiction didn't

have much trouble putting the word out broadly. We've got some

indication of how many newspaper ads, how much 'time on television

what the total outreach efforts were and the like. The cities, like

Boston and Baltimore and Detloit, where only a portion of the

towns were entit d had greater difficulties: Baltimore right

away had probl with city councilmen from one of the white

ethnic neighb oods not4sin the entitlement area. 17fiy

held council aring in Outrage over the fact that.poor kids from

their neighborhood were not getting jobs. This constrginli, Baltimore

-from utting out the word on, a broad basis, because they were

afrai ,of the repercussions. In Boston there was the samO problem...

in tween a Roxbury-Dorchester combination of geographic areas,

given the p st problems in Boston. So whether or not there could be

a jobguaran was a func(iom of the politics more than the

41.0- scarcity of jobs.

MR. PRESSMAN: I'm not - sure whether there was even within the

Entitlement area, a job guarantee for kids who had already dropped

out. Were they offered jobs? -Did they know about it?

MR. BALL: They were only entitled in the technical sense

during the early parts of implementation of.the program. .That was

because we had the in-school eligible population to reach and

build up as quickly as posglble. Naturally there was a tendency

- 2-6 -

C.-
,

..e. *lit

to focus, on thi group. Over the next couple of years,' and . even this mester, we'll see whether or not the out-of-school

kids come back to, enter 4e program. - MS: NUERON: 'Less than ten percent of the'kids enrolled in the

demonstration now are dropouts and the 'rest are kids who are in school.

That does not speak to what these ratios will be down the line.

They probably will change because we know that the initial efforts

were concentrated on in-school youth and that the effort to

outreach and to develop altelmative educational opportunities

for dropouts will;be much greater than ty were lasp

MR. TOIKKA: This is a'very important demonstration

becauie-a-represents a commitment to a serious research component

of z) demonstration program. Speaking as- a person who did a fair -''

amount of work on the research design, it is important to note

that it was terribly complicated and there were a lot of compromises

necessitated by finaWcial considerations. The statisticd3 precision

af.many of the results that'we'11 be getting are very, very sensitive'to the-research and design of the total sample size.'

Answers often depend on timing/of research output. For,instanQe,

if you are trying to determqpe the impact on:graduation then you

will have to slice the sample fairly thin to get a reliable

statistical sample and have to wait a long time. I'd like some

comment on how those ,compromises were resolved, particularly with

respect to the priorities among hypotheses.

MS. GUERON: The questions which had to he address were

those in t gislation. An initial decision was made 101at we , iiheS1

simply can't a ford to survey poor and non-poor youth in ach of

the communities and that we,have to get at impacts on ineligibles

by other less refined means. What we really had to strive fof was,

adequate sample of eligible poor youth in Entitlement and

can rol communities and we couldn't extend it beyond that. We had to

ass re that for in-program and transition effects We Would have

a large enough sample to tackle those questions. We have a great

deal of confidence about answering return to school and school

retention questions, less confidence about ultimate impact issues.

One of the things that we feel has helped is the extension of the ' program itself. We will be leisoking at extending the research design

also. . :k.e %

(

MR. TAGGART: The question of whether an Entitlement is real

or realized is a difficult, one. For instance, We alreadx,know that

the alterfiltive'education systems within Entitlement sites were

not set up in time to attract significant number of youth to date.,

We alread know there's not enough money in the budget for -"

alternative education, to tip extent that that's an attraction that

pulls out-of-school youth and makes Entitlements a viable option.

We want to test whether or not if'we add alternative education

options, we will increase that rate of return tO school. So, in this

coming year, with extra funding, we will change the program to

add educational resources., There will be high and lOw-impact modeis

to see whether the composition of the school systemand the

alternative edupation system makes a difference. Tn other words,

we will adjust the program mid-stream to *neat known problems and

to test new / variables : ' Effects on early enrollees will therefore

0. - 27 -

0"

u
be different than those on later enrollees.

We know that the uptake rate on'Entitlement such as food

'stamps, changes with time. We may reach 30 percent of the eligibles

in the first 'ear,QQ percent at the end of the second year and

a 100 percent at the end of the third. Is there any way to 4uess'

the trend in uptake rate? If we Assess the early results with wha't

accuracy can we project thete into the future? It may be that we

don't want to provide any answer until a year and a half, because . we do not have faith in the steady state potential even though we

had accurate statistical measures of impact to date. Perhaps

it would be better not to rush the results.

MS..GUEION: When the demonstration was first initiated, it was

only for 18 months - -a pretty short time to test a new concept. As

. the demonstration is extended all of the components may be

extended over time, so that for example,in.determining how may kids

drop out of school, we can be inking at the sophomores in

school during the first,year of the program and the sophomores

in school in the second y ar of the program and subsequently. I

would imagine that the ogram impacts will change.

14*

The extent to-w is the start-up period will differ in results

from the later period don't think we can have a handle on until

we've seen the eCtual data on the program.

MR.-TAGGART: But is there any way to guess ahead from trend

or to give ballpark estimates that somehow discount the early results."

We know that early results of the first round of entrants are not

reflective of ;Oat we expect. Can we already say what we expect anti . then discount eNy' results or project them up or change them or

alter them or adjust them in some way in a presentation in ,order to

provide a basis for more reasonable pQlicymaking? Will it be

damaging if we give out information early, which may not be

reflective of the reality? Many of our demonstrations in the past

havebee ruined by premature judgments aboutthe

results: For instance, this clearly happened in the case of the

income aintenance experiMents. It may be better to wait until .the

. right ti e when there is sorre sense of the necessary disunity on

the kesu ts.

MR. STROMSDORFER: It is not a good idea to let out results

premature y, since numbers are never neutral111-they're alway' political.

My bias as an academic and a scientist is to get those results

out to-the people as fast as possible.*Just that always has to be

conditioned by the reality that numbbrsare not objective. They're

always subjective and we've got to be prepared to justify with as

muchobjectivity'as we can the n s we develop.
40,

SESSION II: YOUTH COMMUNITY CON6ERVATION AND IMPROVEMENT

(r PROJECTS (YCCIP): DEMONSTRATION AND RESERACH

INTRODpCTIO'N:

MR. LEVIN: One of the obvious dangers is trying to reinvent

the wheel. Instead, I want to suggest that some decent wheels ex-

.ist ?tow. We have <to find out where these are and how ,they operate

and then try to replicate them.

The-second issue is that of inter-agency cooperation or the

lack of it, at the local level. And when we get into the issue

of inter agency conflict and cooperation on all these levels, we

want to ask the question of how much do these strategies resemble

a strategy of feeding the sparrows to feed the horses.

A third issue is thd operation of inter- agency programs.

PRESENTATIONS:

,MR. KELLEY: The VICI project is small, compared-to what

we've heard about the Entitlement project. We'll have an active

number of youth. participants in the area T5I 480 at any given time.

Throughout the length of the project, we will probably serve about

1,200 to 1,300 youngsters., Although it may perhaps be small in

numbers, however, it's my hope that we do have a lotto gain in

terms of the knowledge deVelopment potential, in three areas- -

replication, evaluation and impact. One of the first things that

we did was to have a literature search done, concerning replication.

There has been very little documented about the process. The

anthropologists have a lot of great stopries to tell us. For instance, there is the famous chainsaw story about the primitive oli*

tribe which used axes to fell trees while the older gentlemen of

lbf

the tribe used to sharpen those axes every night. A very committed Peace worker got the idea to bring a chainsaw to

increase production. on. Theiresult was they cut a lot more wood,

but a whole sector of their society, namely the elderly, became

dysfunctional and there were more problems brought in by the introduction and replication of a model of s etching that works in a

very different,context. So one of the t gs we are s arching for

is how t>he replication process can be'achie e withou dysfunction.

To do this, we must begin to look at programs as havi g certain

essential or key characterAstics and other characteristics that

,might be incidental to the locality. It was up to us to tease out

the difference, and we did 'that after a review of program7.

r

4 29 -

4 2,
MR. DELONE: Both of the p ograms being._discbssed on this ( panel are funded under the You Community Coriservation and

Improvement-Projects or YCCIP. This part of YEDPA has $115 Million.

'It is for 16 to 19 year olds and it is for workexperience and training in producing tangible physical benefits to eqmmunities. The

great majority of this money goes out by formula grants, through

the prime sponsor system. Some $'4 million is.held in reserve

for discretionary grants.

1 /

The Corporation for Public/Private Ventures was asked to

//c/IO one of the, discretionary demonstration projects. We were

to identify an exemplary program or exemplary program features and

replicate them in a variety of cities...bOth to tent the process of

replication and to look at a delivery, system which differs from the

%o

normal pri

11441 sponsor system and from the HUD reliance on communitydevelopment rporations.

g, 'To decide what was an exemplary project, we drew on a

rather extensive survey of youth employment programs that the Ford

Foundation had funded and supported in 1977, which included

literature searches, field survey and a lot of interviewing. We

had information on a lot of programs that fit the general YCCIP

_theme. None of those programs, however, were models. Either they

lacked solid data to document that it was indepd a good program

although people felt it was, or the target population was a

little bit different from the tar!et population called for in

YCCIP, or there was no way once you looked at it, that'you could

really call it exemplary. . 0 . ,.. THE VICI PROJECT: What we ended up doing, was look at some

features and aspects of programs that reasonably experienced

program operators felt made sense. What I think became a central

concern to us and a central featdre of the VICI project, is the

actors,at the local level who get involved. In a program of this

sort to make it really effective, you need a wide range of people

involved in order to mount and implement a program smoothly and

to provide a training experience for young people that does lead

them somewhere, and that does, have some benefit for them. So the

question of linkages became an.essential part of the program design

that we attempted to develop and cull from other programs that we

had surveyed.

And what the VICI program attempts to do is in each, of the

eight sites_, is develop a program that has the following essential

- 30 -
4

features: One, it's a commultity improvement program aimed at

producing these tangible physical benefits, sometimes emergency

home repalcisometimes rehabilitation and sometimes weaqeriza-r

tion. Two, t has A strong involvement of the construction

trade unions in that town and specifically to have journeymen

instructors Working in these programs at a s ratio of one to six, which was in fact what the-unions felt srongly°

emselves was the necessary ratio to provide Itood quality

pervision. Three, it is a must :,that adequate materials and

supplies monies will be available in each site by getting a Contribution from a community developmeft agency or city, government.

And four, guidelines about youth' li ibility implicit' in theelegislation are met. Those were some of main features that we ---

attempted to put in place through a process of invited bids from

15 cities, very careful screening; of those proposals, -lots

''of field visits, a lot of work with them to try to build them

up and finally approval by DOL, and award of contracts

through the prime sponsors by the Department of Labor.

MR. KELLEY: Eight sines have ultimately. been selected and

were able to meet the-'very stringent and imposed-Oaidelines.

al

On of the keys throughout tle project was thatCP13V could . actua y draft thegUidelines with DOL, and review them, and make

sure that they were adhered to, and could provide technical assistance in getting the projects off the ground.

We're also responsible for the researchf We could stress the

research and demonstration facet _of the project, from the very

begining.and that has had some long-term benefits. (For instance,

in introducing research designs).

We will-be evaluating the replication portion mote through

process or documentary'evaluation. than in any quantitative way.

We have subcontracted with two process documenrers who will go

to each site including the sites.which were unable to meet the

guidelines, and to pin down at each site, what's happening and what

makes this lcogram. work and:whatcould be helpful to the f4turc

initiation' o the VICI ptogrdffi in new sites

The model-that we're; using is basically a case study model,

the process documenters will be especially attempting to document

the local context, the economic conditions of the city,he

political structulle, job opportunities and the like.They

will look at the type of work that goes'on, the type of administrative

structure, and the local cast of chdracters involved.

.46
Our process documenters are elsoigoing to look at the role of

CPPV itself and its interaction with local actors. They will also

interview DOL national, and regional folks who are not on the sites

but are intimately connected with the operations.

A second area of research is work valuation. Our youngsters

are going to be putt4ng together and building and remodeling and

rehabilitating. There going to be a torf of very tangible products. Howe do we tag that with a dollar 'number? We drew upon the

very generous help of a lot of ,our co-contractors. We are

to develop and refine ,a work valuation methodology which Tight

eventually be incorporated into a formula-funded community improvement type of effort. We wanted something that could be done practically, accurately, -and inexpensively.

As part of their day-to-day' work tasks, there is someone in

each of our sites who goes out to the work project and does an

estimate of bow much it's going' to cost, for the VICI project. We

are asking these evaluators to ,estimate not only how,g)uch it will

cost in terms of them getting the material and in terms of the

amount of labor including our journeymen and the participants, but

how much a private contractor would charge, for the same-work. We

are going to check those estimates by professional estimates.

We will take this as acceptable evidence that thiS is the valUe of

the output, if not, there may be certain systematic correction

factorS that can be built in. How do you capture salvage costs?

How exactly do you capture overhead from one city to the next?

We're excited by the challenge.

The experiences of VICI participants will also be assessed

relative to a comparable group of young people with similar backgrounds who have participated in other manpower programs.4 At the

leaqt sophisticated level, we'll compare our VICI progiraWts wit the

aggregate of'other YCCIP programs. We will not be abld to mak any

very hard and fast inferential conclusions to that.

Tile next thing that we hope to do is in every site, to

actually. compare the VICI youngsters with 'those in YCCIP projects

and the HUD demonstrations that are going on. (They exist in half

our sites). We will use the existing aggregated data, the exist,-

ing intake forms and the existing termination forms for this type

of comparison. We will-Use disaggregatbd individual data for

these if we can get it and compare again the ilmpactat termina--

tion of. VICT with 'these programs- that resemble VICI in'termsof type of work performed and participant characteristics.

- 32 -

5

I

f 7 /1
I

Finally, we do hope to come clo er to a half decent research

design. What we are saying to sites s that you get your 60 partiCipants, screen them, use your own local screening processes,

get'them,up and moving. After that,' we hope to build a waiting list.

In addition to screening the'60, get a few more for 'backups for

early dropouts. We then hope to, build a waiting list which will be

organized by a lottery system. Folks that are at the front end of

the waiting list will get into VICI or have the opportunity to

get into our program at some time. ,

t- )

For those youngsters who don't do well in the lottery, they r- will be,,,ystematically referred to other manpower programs so that

in this way, we hope to end up with two equal groups, comparable

study groups, both the VICI and other program.participants.

.MR.'tCHECTER: The`HUD/YCCIP is a bit different from most of the

others discussed. One, it's being operated by another federal agency

which acts as a prime sponsor, while at-the local level it is being

operated by community development corporations and other kinds of

neighborhood development organizations. That is, it marks a

, 1978 throwback to the late sixties with direct federal funding of

local non-profit operations.

A further difference is that with most of those organizations

youth employment programs are not their primary reason for, being.

They're in the'business of developing their own neighborhoodS.

In the economic sense, or abroad community development sense,

and housing rehabilitation sense, they have another agenda which

for their purposes is primary. We thA that makes a difference in the

kinds of work and the kinds of attitudes at the worksite between this

program and more traditional youth employment progfams.

/

f4, There were three principal objectives for the demonstration. One

,was to demonstrate how well community developtent corporations can

put together resources available from other sources, such as

weatherization labor., material's and so on. That distinguisheslit

probably from some of the prime sponsor fornuila funding operatilons.

The second objective had to do with the nature and quality.of

,the work being done. The notion once again being that'we're an

organization that's primarily Apterested in something like rebuilding it's'community. It's going to have that is a goal which

will affect the nature of the work done and the impact its

4 business.

. . The third aspect, of the whole demonstration was to compare the'

i ' way, this project impacts op organization, participants and the

communities themselves, compared with host- formula-funded YCCIP

projects in these same localities. ,

This.is an $8 million project. There are ten sponsors spread

all over the country in both large and medium urban areas as well

as two rural areas. There's an average slot level at each site of

about 90, so there are 'about 500 positions.overall. We

expect to serve somewhere around 1,800 tg 2,000 participants over

the course of a-year. , - 33 -

u.
The program began operation Febrtary 1st, and now we are

in negotiations about how long each of these will be extended,

but it looks like the average length of the program in each local

ity will be somewhere around 20 months.

MR. FELDMAN: Our evaluation consists of three sets of site

visits, plus analysis of participant data at five of the ten sites

Those five sites were putf.together in consultation with HUD and I

think the idea was to look at the ones which were the most promisin

Subsequently, it has developed that these projects are not necess7

arily the best.

g.

"No The project design and methods of analysis were based on

the three site visits to each of the projects and an analysis of the

application forms and the termination forms from each of the ten

cities. If possible, we were going to use the same kind Of data

analysis'from up to four comparison sites in each of the locationg.

And to anticipate the end of the comparison probl m it is probably

not going to be possible to find very comparable groups, against

which we can compare the HUD-YCCIP. N

One of the major reasons is that there is a very sub;tantial

difference in the scale of the projects. The HUD projects are

substantially urger than the comparison projects and therefore

the potential for the community impact and he potential fbr the

impact on the organization is very different.

We are to be supplied with adequate data for each of the ten

locations on all. the YCCIP's and we can make some kfind.of comparison.

Thbsp comparable groups are not going to be there api.,01 so-if -the

question is, how' do they compare; in some ways things are stacked

in favor of the HUD YCCIP's because of the options they had due to

the larger scale of the effort.

On the positive side, we have been able to devote extensive

time at each of the five of the ten site locations. We have beenable to interview people in terms of the basic objectives that were,

described:

First, the impact on the community development corporation,

second;'the impact on the neighborhood and community, third, the

.impact on the participants. We are looking at impact on community

development corporation in terms of the structure of the organization

and the organizational changes which occurred that can.be attributed

to the receipt of the IIUD -YCCIP funds. The grants between half

a-Million dollars and $800,000 ort the average are substantial

for many of the relatively small community development corporations.

.9

- 34 - t,
4

In addition, some of the community development corporations,

were not iry-the biasinessrof youth employment, so it produced

a substantial change in their objectives and ,in, the nature of the

organization. They had to tool up to dp something that they had

not done before. Some were, in the education business, some had

other functions,, but the receipt of this meant that they had to pull 8

together teams in rehabilitation, in weatherization, in solar energy,

and they had not always had 'those experiences before.

In terms

.,.of impact on the community, we are interviewing

community leaders, political leaders, leaders of other community

agencies and wq are exploring the possiblity of random digit

dialing within the actual target communities tcosee what perceptions

,of random samples of people with telephones are in those areas.

In terms of impact on the participants, in addition to looking

at the nature of the characteristics of the participants, the

nature of their employment while in the program, the kind of

terminations that they had, we are also going to do a, limited

number of interviews witparticipantg for their perception of

the program and on what it meant intheir lives. We are looking

4, at how they perceive it now they are in the program and what they,

anticipate in the future. We will be comparing that, we hope, with

the sample people and the comparison of XCCIP's. The hypothesis

was that this approach would produce aneffort to put together more

meaningful work and it would have an impact on the community.

It was not supposed to be just straight employment and in the

best of the cases and indeed most_of the cases that we have

visited it is not makIrdork. As with the CPPV projects; an effort.

was made in many of to places to hire journeymen supervisors,

in some cases from the unions, Ipd to provide a relatively, low ratio

of at least less than one journeyman to .qen participants.

Let me gum up in terms of what neral findings have been

up to this point and I would stress that these are interim findings.

First, that in the five cities we have looked at, most o the

projects seem to be working well in the sense that they h ve- been

able to tool-up to do things that ma,41, of theN have not done

before. We were relatively surprised at how rapidly organizations

which'had no experience implemented the programs. Second, in spie,

of the fact that this vas pot supposed to be a social support ',7

effect, services were pr ided in many of the places. 'It has

happened because the CDC' mere prepared with other programs to

s provide social support.

The work seems to be going relatively: 4e11 and in most cases .. j--

it's meaningful work.,Thee seems otobe a trade-off between the

impact on the institutionland the impact on the community. The

greatest institutional impact of the HUDgrahts occurs in the oases ' ( Where the organizatio'n had little experience or small resources.

10

- 35 -
4 Those that have had experience doing this before were able to

take money and plug it into ongOing programs of rehabilitation

or housing relocations.

Among the most important,impacts on the .perticipants.seemsl''

to be the receipt of.money by the participants and the Work

experience. But in addition many of the jobs-demand the kind of

work performance Comparable to-unsubsidized employment. Cgrpentry

skills, tile setting, plumbing and,the like have provided

significant skill training.

The job supervisor seems particularly influential in working

with the youth. The journeyman seem to have developed very good

elationshipswitir-the- youth. Theyseem to serve economic role

models because of the high income - -in the area of $30,000 a year

and An, ethnic status which is usally the same as for participants;

Obviously the programs where the participants are engaged

in complex work are providing greater opportunities for skill

acquisition but at this point it seems significant that whete

participants are engaged'in complex .tasks and where the craftsmen

participant ratio is not high, the outcome seems fairly positive.

Although it's still' too-early for a definitive comparison with

formilla-funded YCCIP's, I think we can say in general the HUDYCCIP's are larger than the comparisop groups and that the scale

of the project makes it easier to have substantial community

impact 'that for -the prime sponsor YCCIP's.

,s DISCUSSION:

T _a__

MR.,SHT(ANK: Why do objective research on youth programs?

Large sums of money have been allocated for ma quantitative

evaluation effort yet no one is asking what e pitfalls olie such

research might be, or whether it is even appropriate to sIhat we

are trying to study.

The object of the research is a network of youth pr grams.

The first problem them is that some programs are ,noNco arable,

others may not be replicable. The desired outcomes are oggy at befit

and of ter totally out_of the control of the program operators.

For instance, in employment training how many,program\graduates

are placed upon completion? The factor of how jobs are distributed

in the labor market is beyond the control of the training ° *program. The outcome of acements may in fact have-nothing

to do witoh what the job raining was about. The objective social

-science xesearch model May turn out to be more of a burden than a

beacon for,policymakers.

Let Me see if I can tell,you why. I will also argue that the

most important part of the program experience of participants

may not be at all quantifiable and therefore not subject to traditional

outcome measures.

- 36 -

S

re
There exists a terrible tension between doing objedtive

evaluation and trying td make a program spcceed. A basic variable

._required,for a successful program is that the staffiAn its role as

teacher, Itrainer, counsellor or receptionist, be'strOngly,

nor better yet evangelically committed towhat they are doing. That s

the elecricity,the juice if ybu please, that will supply the ergy

tO,make the program light up. That mightbe the most important program

variable and if it is absent nothing at:a1-1----Iriy happen. That will

put researchers in a tough spot. Are thy,part of the program commitment or are they just objective.observers?' If their objective

observations indicate things-are not going as well, as the staff

believes, ,then the staff may just lose it's juice. If the data

suggests things are going well, this implies that if the program

is g-aplicated it 'will do just as well everytime. But if what is

'really making the program- a success is the zeal and comMitment

of the staff, then merely reproducing the operational structure can

by no.means assure success. How do you replicate the juice?

As I listened to comments on how the various youth programs were

developing, I had a feeling that what-was being sought was some

quick success.. Seniority and tenure permit me to risk the

prediction that there will.not be any measureably dramatic'

successes,in.any of these projects At-best the results will

show'50% success to 50% fbilurefthe halt-full, half-empty glass.

'Phe4se--klrAs of figures will not excite3. our sponsors on Capitol

Hill. Soma serious'folid on-the Hill m4y be happy to discuss such

results-but it does not _Rake good\press-cofiy. If it is_

instant hula-hoopguOceS7p- that ispdanted axd you can simply declare

"as a re-suit Of fhe etia'tlementsgrogrant. all of the kids who

participated are babX;in schbol: we are 100 percent-successful"-,

the profonents,will love xpil.-They will then assert that every kid

who is about to drop out ot schbolwill choose to stay in and all

th9se who dropped,out.will 4o back;Ito school if we just plug in

the entitlement program. 0 . ti

I am obviously exaggerating'to.makV 4g4nt. Looking for

quick fixes has little to do with knowledge and alot to do with

public relations. If you waptto create'-0'_strong, positive image,

I would urge you to exhange the research for a public relations ' department. I am not suggesting that P.. 4an evil thing. This

9ountry runs on public relations not on objective research; but,

would suggest to evaluators that public relations is not your

problem--you were hired to develop knowlpdse. That was a phrase

Joe Seiler thought of and he is to be cr'editgd for the good idea.

Substantivelearning from a program as,i develops might produce

bound information and perhaps, eventua y, some important

insights, but it will not produce sexy ectcular results. If

that's what is sought after, do .public tionb not research.
' For some time at the Ford Foundation we have been looking at

exemplary project--programs that work. In some of the paper that I.

read for this meetiOg the question "does it work ?" "appeared quite

often. That is a tough question. The existential fact of what does

or doesn't work, depends on what you mean by "it works". In the

1 exemplary program search I was struck by the number of community

developmAt programs that do work and quite effectively. MDRC's

program of supported work is an example of a program that works. The

data show that while the people are in the program they are doing,

very well. The next question the objective researcher may ask is

1 what will happen when they leave the program. This suggests what

is needed is a longitudinal study.. Well ot. course we could

,study these people right after they leave the program, a year later,

five years later, on into old age. That is to make sure the intervention really "woiked".But in.th'e peroccupation with post-program

data we nay lose sight of the quality of the experience of the

participants while they were in the program.

I am not opposed to longitudinal studies yet I am not aware

of any great insights that have come for example, from th4 Ohio

State materials. It may be interesting, and I am sure we have

learned some things, but it did not give us the E=MC2 of employment,

and it never will. The problems of unemployment are intr-icately

woven into too many othef social and economic problems to be

solved by simple mathematical formulas or 'correlations. Longitudinal

study researchers will agrue "these people did well in the supported

work program but what about when they leave?" That question

scares the hell out of program operators because'they know they have

no control over that. My response would be, "well, that depends."

q depends on the level of unemployment, where they live, their

rae.or color, .how much discriminat n exists in that labor market,

and a long list 6f variables over h employment programs have

little or no control.

Not only are longitudinal questions the toughest ones to . answer, it is also difficult to prove that whatever post-program

effects we find can be directly attributed, positively or .., negatively to the pOpgram. This suggests to me that in,o-Eder to

understand what is "working", we need to know the experience

of a program in a given time frame, and accept that'as valid

information on the basis of which to make ou judgements.

Sometimes I hear researchers say, "I have a hunch." I would

define that as a felt sense or the existential experience of what

is, happening. Often I find hunches more interesting than some

conclusions based on longitudinal data. Numbers may be accurate

but not insigitful. A hunch may gike us'some interesting insights

in to what is actually occurring that connot be expressed in

quantitative terms.

Another corner we pint ourselves into comes from the tension

betiveen advocacy and so called "objective" research. I do not agree

with those who have suggested that'there was no research in the

sixties. There was -plenty of research. Often the problem was that

many of the people who were researching were also advocating the

programs. There were social sc'entists who believed in Social

change. Oft times when they developed hard data that did not fit

- 38 -
4.

their rifghekpectations or what they are advocating, they simply

made sure that some of the data died in the printouts.

Mobilization for Youth, a first large scale social

experiment was a good example of what I desc,ribed. Millions of

dollars worth of research was never publighed. My hunch is that

'IA, did not come out nearly as good as expected. It probably

showed that half the kids benefit from the progam, while the

other half did not. Perhaps only 30 percent of the participants

benefit.The funders would not have considered that a success. To

the planners and researchers who were yearning for the thing to

work, it was a disappointment. MobiTization for Youth was going to

be the proof that juvenile delinquency could be solved by creating

opport'unities.\ When the, data could not demonstrate that theprogram had "solved" the problem it mysteriously faded away and was

never published. Was it conjecture that the sponsors would disown

MFY if they had seen the research outcomes? Would they have considered

it a failure because only a few.kids got jobs? One kid even made,

it to Harvard Medical School. We had his picture all gveY the place;

he was our success story. My experience tells me that if a\_

program can succeed even for only thirty percent of the kid, then

it has not failed.

Based on this kind of experience I am concened about the

proposal to do a cost benefit analysis of the youth employment

rehabilitation projects. Save the taxpayer's money! I will give

you the outcome of such an analysis. It is cheaper to contract the

work out. What a cost benefit analysis fa4s to comprehend isf'

that not only are old buildings being rehabilitated by kids who

never saw a hammer, a nail a piece of plywood, and could not

read a ruler are learning how to do construction work. It is an

education for them, so of course it costs more to rehab with that

crew. If I want to play the numbers game ?could cut the training

cost for each person and reduce the total cost to only a few

thousand dollars. The trouble with thtt course of action is that

while manipulating the numbers, you sacrifice the substance of the

program to make it look good for e funding source. That is

not to say that program content, "is not researchable, but we do

'not know how to measure that ,.ntent and it's human value. Too

often we substitute quantification which may in the end tell us

more about number manipulation than about the tvorth of a A program.

We can only.Measure what we know how to measure. This is an

important principle but it is often overlooked. What happens - to the young unemployed or school dropouts in these programs may

not registe on your questionnaires. Their own experience may prove

to be.th ost important source of your data even if it does not

fit the survey. For them it mi4ht be a life-saving experience that

somehow did not register with the, researcher. How db/you gather

that. kind of data? You start by learning to listen. Don't lose,

what happens to people while they arp in the program. It sounds

funny to talk "experimentals" and "controls." These are the participants in the program.

-
S.

I mentioned the community development corporations (CDC's)

as an example of programs that work, For almost a decade the

Ford Foundation has been supporting a number of CDC's. Our

overall estimate is that they 4.re successful. A key factor that we

were testing with these CDC's grew out(of some things we had

learned during the'years of the anti-poverty programs. We found

thatto deal with complex social problems through short-term'

funding was a serious mistake. Operating agencies cannot focus on

programs if their major concern is where their next grant is . Coming from. The Foundation decided to do an experiment with lbngterm funding and the effects0Ve been most positive. It has

conviced me that 'social programs heed apinimum of three years

of funding if we are to expect any serious results. The notion that

we can produce instant results in six months or a year in luperous

in the face of ,pur years of experience that tells us otherwise.

It is simply unreal. I appreciate the political pressures from

"pending legislation" and on the folks who have to'run up the Hill

to tell the "good news' of ,what happened, but knowledge development

is the goal, it will take time, and the results will probably

not be spectacular enough to excite anyone on Capital Hill.

A final comment on quantitative research and social psychological

problems draws on the experience of one of our 20th century giants

of learning, Piaget. He did his research on his own 'children. This

is a remarkable Teat when you consider the impact of this man on

the world of learning. His universe was observing, his own kids and

how they learned, but he was able to synthesize a lot of

observations and develop an astonishing insight into how intelligence

clewelops. We are in desperate need of some synthesis of successful

youth employment program experiences. The numbers may give us

guideposts but they will not tell us the what and how of programmatic

success. We can not be satisfied with the numbers alone. Go

beyond them ...go to the existential side and see what you .oan'find.

,Perhaps it will be insight.

There was another disservice that'we anti-poverty warriors

dievourselved,in the sixties. Sometimes I think we were our

own worst enemy. We had not read McLuhan arid we did not know that the

medium was the message. We insisted that we were social scientists

not hucksters so we.had to tell the truth. It turned out that

the truth as we told.it was misinterpreted. We acknowledgedthat

some things did not work and our political naivete even said whole

programs were ill-conceived, believing this would result in

an objective social science discussion of "the problem." Instead

program funds cave stopped and generalizations sprung up like

. "the War on Poverty Was a big bust." This was taken to mean that the

social problems we were attempting to solve were intractably:. What

followed was first benign Iwglort and now the- politics of Proposition

13. The creditability gap, did-not come only from the right-wing presg%

It came from liberal democrats who siad "nothing works". When I

hear people say "the Job Corps, Neighborhood Youth Corp, Head

- 40 -

, 5

14.
Start, MDTA have all been failures", I am furious.

What is my evaluation of thesdl programs? I would. say some of the

Job. Corps camps are good and some are not. Some Neighborhood Youth ' Corps programs are bad, other's are good: like most institutions in

society, some are great,. some mediocre, fair or poor. Some of

the graduates on follow up studies come out very good, others not as

fogood-, other look poor ,or have dropped out altogether.

Most institutions divide up abdut thiS*way, and I am sure w will

find the same outcomes with the youth employment Programs.

Therefore, look carefully at the data but look beyond it. Be aware

of the political /implications that will be ,drawn from .the

research findings, but most importantly look for the positive

impact of the program on the parti4pants.

MR. BARNOW: The concept=of the VICI project makes sense.

Once we have idghtified programs that do seem to work then it's

important to know how to replicate them and I think there's going to

be a,lot of difficulty,in the project in going through the replication

party. In some senses, replicating, these training programs and

work pxperience programs may be somewhat easier than replicating

preschool or high*school or grammar s hool programs with a particular

curricula and the like. -Yet,the-lite ature on educational

programs suggest that it is quite ficult to try and figure out

whit are the keys,to replicating. hat are the key things that

you are trying, to replicate? I just hope that it turns out that

we did indeed isolate The key'factors under VICI.

An important issue here, for our experience and existential

philpsophy, is whether or not failure to replicate means a failure

of the program. It's quite possible that some of the features

of the original program-were just not suitable-for elsewhere. It's a'

serious question as to whether or not it should be replicated in

full undeiNp-all conditions.-Alsid, I hope they will determine'

whether or, not failure to replicate certain. aspects is a

positive-or negative factor and I hdpe that will be loQked at.

The VICI project will. also look at_work valuation. It

remipda,me of the man described by Oscar Wilde who knew the price

of everything is the value of nothing, because what they're

doing is-determining the prices things would cost in the marketplace,

but that's not necessarily the value. I think it's a very difficult

issue to try to get into the valuation of public sector, comm

property resources.

The third issue is the posy- program analysis and here is one

area where 1 am quite concerned. It seems to me that both the

,VICI project and,-the HUD-YCCIP projects are going to look at

termination data, rather than using any post - program interviewing.

I'ts very dangerous to only look at what happends to people at

ter nation status. For instance if you look at,the positive termination rate, under public service employment,it's typically

around 30, 35 percent. If you look at the rate of employMent six

months later, it's around 60 percent.sTermination success depends

partly on pldconwnt efforts and so it's dangerous to just, . pick the one point in time, and especially the immediate dhte of

term.inat,ion.

- 41 -

--\

7 4-.1
i

I think the question of whether or not we want to look at the

po t-tprogram data and how seriously we look at it, depends am'

what our goals a're in the program. It's not clear entirely

whether we should be concentrating primarily on in-program benefits

or post-program benefits. I have the belief that Congress

is really interested in the post-program benefits and that seems

to bewirSETs in the law.

, Another issue that we want'to go into in detail, is whether

of not the people in various programs are comparable. How dO

peopleg4t into the HUD-YCCIP,, versus a plain old ordinary YCCIP?

Are they similar people? - In the HUD-CDC project, it's important to try to figure out

whgt the goals-,are and set priorities, so we can figure out what

we want to look at. In this case, I was most inter sted in the way

that HUD put it together rather than DOr.1 I mess atis just my

own bias. HUD goals were to look at'the impact of e program

both on institutions and ths community, while

DOL had an alternative' goal of comparing it to olcher delivery

approaches. Right now a lot of our money goes out to the prime

sponsors and we don't give them technical assistance and we have

to learn ways to work better so we can he,lp them. This is true notjust in the new programs but it's true of the larger CETA efforts

in Title I where some prime sponsors feel, that we haven't helped

them enough in ways.of exploring the programs.

One concern I have is this comparison of the HUD programs

with the other YCCIP programs especially because of the fact that

the HUD programs are bigger. I don't think we should compare HUD

programs with 500 people and then go say, the big program has

more value. dhaewe want to do is determine whether there are

economies of scale.

MR. PACKER:

'Why don't we just take this money and lay it out in formulas

and hire some kids for three bucks an hour-instead of paying for high

powered research?-The answer must be that those researchers are going

__IDcontribute an answer to Congress and the policymakers that will

allow them to allocate their resources better.

My judgement based on my experience is.that Congress is

dying for some good information. They'live on PR because that's

al.lanybody gives them. Solid research work, even lousy research

work has an impact. The ideas stick there, and you'd be amazed

about how little information Congress will use because it has to

have some sort of information to make a judgment.

Hopefully, we'll get some answers and we wil ecicle that

the money we spent on research is worth more t roviding a

- 42-
couple more jobs for these kids because that's what the alternative

is.

MR. FILER: One thing that I think we have got to b prepared

for is that the research can have negative as well as positive

findings. People had better be prepared for the fact that you may

prove that your things really are as ineffective as we thought.

They might be and that's a perfectly valid outcome from the

research that's been funded. The purpose of the money that we are

all being given is to find out what is working; not how can we

best present something that looks as if it is working.

MR. PACKER: Most of the research which has been done to date

could never have been justified in science. It's not been goodenough. It is net good enough in my judgement to continue to

finance research at the.expense of jobs for kids, unless it does

better than it's one in the past: We hava had lotg'of misadventures with r earth, partly because the researchers didn't

ask questions that were going to answer, and didn't

follow through.

MR. KELLEY: But our schedule of inputs into policy must be

realistic.. It must take into account what we know alieut the

lags of the data coming in. A lot of those data that you want are

not to be ready on time.

MR. PACKER: You could do a study starting now and finish

it in three or four, months to know it more kids have come back to

school in the places where there are entitlement projects. That

Could be done by just getting our there and going to those schools

and asking teachers in the ninth grade, whether more kids are

coming back than usual from the eighth grade. That's why I'm

here, you must have answers on time. Legislation is coming

that is going to lay out billions of dollars to be spent in '81

and '82. My impression of the people on the Hill is that they really

do want to spend it 41 the right sort of wa and they deserve to

know 'what you can find out before they lay ou a couple of

billions bucks. ' MR. SHRANK; I think that a lot of research could- have been

A lot sharper than it was. We will say that next year about this year,

but I would take yolrexkMple,'as a perfect example of what not to

do. L don't know what the attitude of those ninth grade teachers

are, but these kids weke bringing back, maybe those re precisely

the kidd the ninth grade teachers really didn't wan in 'school,

and pushed them out/because they were difficult le ners. So for

me to go in there and just say to ninth grade teat er's, how

many kids came back and soon, may give m e. the wro g kind of

information. So I don't want to do that. That to e is not the way

to solve' the, problem. You set an example of what of to do.

I also don't know what the school,records would s ow, but I would be

very leary to go to Congress and -say the program worked or not on

that basis.

MR. PACKER: 'I certainly don't/Want people to think that the

DOL money is being laid out there in a fashion that if the answers

- 43 -

t.1
s

are not in accord with our thinking we want to suppress the wrong

angwers, because We dOn't. We really do want knowledge development. It may be presumptious in a sense to put those words on it,

but I think it's not a bad description. We have tried to develop

knowledge and I think philosophically, I am a. strong enough believer

in the democratic process to say as analysts we just find out as

much as we can. I believe policy rs, the true policymakers, the

politically elected policymakers!ll who the people have given

so /ereignty, will do the right thing. The intent of the research

money iS..to find out as best we can and to give them the information.

MR. KELLEY: We are all-in favor of doing research that

is relevant, of a high technical quality, as quickly and as cheaply

as,possible. I wanted to go back through to the question that's

been raised about the value of output work. No one would

hypot7esize that kids will do this work cheaper than a private

contractor. That's not what we're arguing. We're arguing that

there is some value to their output and how much is a critical factOr

in estimating overall benefits and costs. We're not trying to

answer it in-a real economists' sense of what the value is. We are

looking at the alternate supply cost and there's a reason for that.

One good reason is that it's written into legislation that the work

that is done in these programs must be work that would not be done

by a private contractor. Presumably then there is work for which

there is no market demand and one r son for that is very often the

work is being done on the homes of v y poor people, who are not

going to have the money to fix up.th it porch or"rehab" their house

or put on a new roof.

If on the other hanp, you argue that ere is a,public

policy goal, that there is a social goal, o provide that kind of

decent living accommodation for citizens in inner-cities, then

the question is how much would it cost you to meet that social

goal otherwise and supply side costs then start to become a very

relevant estimate in my mind.

MR. DELONE: We're devoting a lot of energy to following up

at least 80, participants if we can make the contact three months

after they finish the VICI program. We want to follow up with

the same number of non-VICI folks. We're doing a three-month

follow -up and longer it proves.feasible and resources are

provided.

.

I 43
SESSION III: YOUTH EMPLOYMENTAND TRAINING PROGRAMS- -

. DEMONSTRATIONS AND RESEARCH

PRESENTATIONS:

)4

R. JONES:' We will be discussing five relatively unrelated sets'ofprojects next. In this session, I think we should look at citiesprtilization as well as comparability amoung the data.

MR. PRESSMAN: Pm from Youthwork which is a nonprofit intermediary

established to assist,in the research and demonstration aspects of

the exemplary in-school grants-program. One difference between our

project and the others that. will be discussed is the scale and operational complexity. We ultimately received over 500 proposals for

exemplary in-school projects and had to select between 45 and 50 from

that. This was the "let 1000 flowers bloom approach." Ten additional

projects that will )De funded noncompetitively will enable us to fill

/ in the Anowledge'aevelopment gaps that exist in the competitively funded.

projetts.

One of the things we will emphasize in our knowledge development

thro.ughout the demonstration iS following the perceptions of young

people over time. We also, as an in-school program, are very interested

in that linkage between the local educational agencies and the prime

sponsors. The programs that we have helped to fund are supposed tobe exemplary projects, demonstrations, innovtktions, if you will, and

so, in additipn to providing some work, they are supposed to do better

at teaching liasic skills, to improve counseling and guidance systems

and to.do a number of other things that affect the overall participant

outcomes, rather than simply providing a job.

We have a three-tiered approach to our assessment. There is a

management information system that eyery grantee has to implement in

order to get their money. We also have an intermediary local knowledge

development pace that requires the local prime sponsor to do knowledge

development about itself. We provide the technical assistance and

some of the suggestions about the designs and research questions.

Finally, we have independent research being done on related subjects.

I think that we ,have to recognize from the outset some limitations

of our research. One is that we're dependent on local knowledge development. A second is that we've assumed that there were a lot, exemplary

projects out there to be studied. One of the first thiAgs we have to

find out is whether, in fact, the selected projects are exemplary._

We talked a lot about innovations and about the bpportunity to study

innovations and their impact on institutions, but I'm not sure whether

we'll have all that many innovations to look at. So I think it would

behoove us, to be very realistic about some of the limitations of our

research and knowledge development.

MR. LEZNICK: I'm from the National Institute of Education. We have

one of the 17 interagency agreements that were referred to this morning.

The Department of Labor has charged us with the responsibility of

studying the Career Intern Program which is an alternative high school

program developed by the Opportunities Industrialization Centers of

America, under the direction of Leon Sullivan. It's'a program designed

-45-
for 16 to 21-year-old dropouts'andpotential dropouts.

& DOL is interested in learning whether or not a program,that has

proven effective on a small scale can continue to be effective when

the approach is implementea on a larger scale. Our research is intended

to answer four basic questions. What happens to the program itself in

the process of implementation in additional sites? We're carefully

studying the implementation process at each of the four sites. (1) What

accounts for the changes or adaptations and for fidelity to the design?

(2} Does CIP continue to ,have impact in helping youth when it's implemented in sites other than the original prototype site? (3) What happens

to the young people in the CIP that could account for its effectiveness?

The early evaluation identified what seemed to be the essential elements.

We had. some hunches. This gives us a chance to find out if those hunches

are true. It is hoped that theanswer to this question will help the

Department of Labor and other federal agencies design other programs.,

Of course, the 3Agt qUestion we hope to answer is, "How does this

approach compare in electiveness, feasibility, impact, and other

factors important for policy with other approaches serving youth?"

MR. TALMADGE: I'm not.sure we are going to answer all those questions

although wee will certainly address them. Let me start with the first

one we refer to as the fidelity of replication. I don't think any of

these projects are replications in the true sense,of that word. They

are attempts to implement a project that was successful on one site,

but they are appropriately modified to fit into new situations. I think that when we identify deviations between the replication of

sites and the original model, then we need to assess each one of those

in terms of whethbr or not it's positive or negative adaptation.

In looking at student outcomes, we will have a randomly assigned'

'control group determined by "lottery, In fact, we go one step beyond

that. We.are going to have a matching of students prior to assignment

to treatment or control groups, s6 that we will have a matched pairs

design that will enable us, td handle problems arising from differential

attr'tion. Should, for example, the best students g;aduate out of the

am before we get their post-program scores, and/the worst students

in e treatment group disappear for one reason or another before we

get their post-program scores, that would leave us clearly in a

situation where treatment or control groups, though comparable initially,

would not be comparable at the time we did the evaluation..

We have had some probleMs in implementing this design. There! were

delays in,the program and there'have been serious"difficulties in getting as many kids in as called for in the experimental design. We

may end up not having any of the nice thing's we're supposed to have.

We may have a choice of serving some kids or assigning everybody to 1 the coritroV group.- It won't be quite that bad, but we could end up in

a situation where it didn't really Make sense to assign enough kids

to the control group to give us a meaningful comparison because that

..,would significantly cut down on our bility to.assess the replication

. "effort itself. We don't know exactly at we're going to do in this

Drt)- 46 -
case, but we will resort to the usual tactics of a search for comparison

groups, perhaps,'coming from the feeder schools from which the CIP

interns come,iperhaps from other sites that lave similar problems, but

do not have a program. We really haven't worked that out yet because

vle're still hoping that it will be possible to implement the design as , originally planned.

In the area of trying to relate characteritics of the program to

specific outcomes, we will be relying heavily on ethnographic types of

analyses,,,in-depth case studies of students, interviews with teachers,,

students, parents and community members. We, have ethnographers who

will essentially live on site, observe as participants in classroom and

non-classroom situations, and try to tease but some of these hidden

relationships for us.

Id the comparison of costs with ..efctivenets, again we will have

problems. The CIP program is soMewhatique_in '.11at'lt is basically

a high school program - -an alternative high school program--that

provides high school degrees, not GEDs. It's not clear that there is

another program quite like this to which valid comparisons could be

made. However, we will be making what comparisons we can. We will

be looking primarily at those outcomes associated with participants

getting additional education or getting meaningful employment. We

intend to foltow this up for as long a period Of time as our contract

and the initial start-up of the cohorts will allow.

We Afe also assessing achievement gains in reading and math.

We're looking at career awareness, self-concept, and sense of control

over their lives.

MR. DELONE: The CCPV Private Sector Initiatives Demonstration could,

in fact, be called both a demonstration and an experiment. T4e-N

demonstration is an effort.to test out and, in some cases, to/develop

and test out, a variety of programming appYoaches. We're not setting,

them up as research experiments, but rather as programs that we're

trying to make viable and then research very carefully. There's a

distinction between that and a pure experiment. The test of direct

incentive subsidies to employers is more of a sure experiment.

On the demonstration side, there arena variety of cross-cutting

questions that we want to try to answer for the complete range of

approaches.. We want to try to find out whether disadvantaged, outof-school youth can become more competitive, and whether opportunities

can be permanently increased for them, given a supply of jobs that

our programs are not going to materiallyltaffect. Does'work in the

private sector in fact produce better attitudinal adjustment, better . skill development, better human capital accumulation, than does

work experience programs in the publib sector?

Central to all that we're doing is the question'of what the

private sector will respond to and get involved with. That's a somewhat

unusual research emphasis, but it becomes a particularly critical question with the emergence of Title VII, and the creation of private

industry councils, which, in manly cases are organizations beginning

r - 47 -
a ptcocess, wondering what the hell they're go,ing to do wheri they're

finished. There is a dearth of good program knowledge and experience

that'.s well-documented that can be exported and transplanted if it

-works.

Because of our concern with the private sector, we are caught

in some rather exquisite dilemmas. One thing the private sector

doesn't like, for instanq, is'peop],e from the government coming in

and collecting a lot of data about what they're doing, which is exactly

what we need to have'for research purposes. We must dea; delicately

with that kind pf tension. In all of this, we'll be trying to look

at firms and what happens to them in terms of their'employment records,

who they hire as they becche involved with our programs, whether bey

change their intake and hiring procedures and their supervisory methods

and work str ctures'to accommodate youth. We will be following youth,

colleihg tensive data on them at entry., following them through the

program and, afterward to see if the effects of a particular training

prograt or a subsidized work experience show up.

We'll be looking very carefully at, the process of program development through I4rocess evaluation, trying to find out what really does

go on. If we do have a program that show". some 'good results, what

Made it, a good program? I think that's a key element of thiS research.

On the wage subsidy-direct incentives experiment, we are very

much at the beginning of what is an extremely comptItleted technical

planning process and I will not try to. get ihto the complicated research

issues that are involved there, except to say that I think we will be

particularly interested in testing varying subsidy levels to determine

whether the take -up rate is significantly affected andforwhich kinds

of firms. The assumption to the test is that the OJT subsidy,-either

in amount or duration, is often not a very strong incentive for firms

to hire the youth who are our concern in these programs. - We are in a very early st e all phases of this demonstration.

Among our programming initiatives, several are off the ground, but

most are still in the planning stage, particularly the direct

incentives experiment. Were trying to involve a lot of people outside of our organization to assist us in the techrfical elements of

planning this demonstration and the associated research. We still

have a lot of flexibility to change directions and we,would really

welcbme input from people who are interested.

MR. MUCHNICK: The Youth Community Service Project in Syracuse provides

stipends to 16 to 21-year-old youth who are both out of school and out

of work, to participate for a year in meaningful community service

projects. ACTION conceived of this model of volunteer, communitybased youth service for three reasons: First, we thought it was an

alternative to the'usual way of deliveripg necessary community services.

Second, we thought it was a,pew way of thinking about approaching young

peoples' problems of the transition from school to work. Third, we

thought of it as a major alternative to the customary ways of

thinking aboutorganizing.a national yolth service.

61

- 48 -
It is, in essence, a test of one model of a volunteer community14°'

based decentralized youth service. We think it's different from some

of the usual approaches to youth programming, primarily because it's

founded upon public values of community service and citizenshipe not

simply the private gains of jobs and career training that.the young

participants malcderive. Its notion is that as necessary as jobs are

for young people--clearly they are very necessary--jobs by themselves

are not sufficient to overcome the estrangement that young people feel

from adult members of their community and vice versa.

So our notion is that the value of YCS to the participants is

surely their personal gains, but also the service that they render to

their communities--if you will, their acts of citizenship, the

expressions of civic loyalty. In return, we expect the non-youth members

of the community to complete the civic bond, to give expressions of

support and recognition, expressions and commitments of timband energy

that really make the program not just a youth program, but a program

of total commu ty involvement.

One of the ma or questions, then, was whether or not it is feasible

for this kind of to al community involvement, whether the ideas can be

learned, understood, and implemented at the local level. For that reason,

to test that feasibility, ACTION has placed primary responsibility for

the administration and implementatioh of the.program on the Youth

Community Service Corporation in Syracuse, New York.

Between the 15th of March and September, we have gone through a

period of testing the basic systems, a period of making the transition

from ACTION to local grantee operations, and a period of educating the

local grantees at all levels--the local board of directors, the local

staff, the local sponsoring organizations of youth "projects - -in terms

of Aat the philosophy, ideoloay, and research objectives of the Youth

.Community Service Project are.

Youth Community Service initially meant, all things to all people

and, therefore, there were never any conflicts at the beginriing'e

It's only when we started to get down to the nitty-gritty of approving

a particular project that people began to understand what was different

and what was not different.- So the vague notions of "do-gooding,"

which everyone embraced eight months agO, begin to really wash out.

We haVe about 150 volunteers in service now,. We have another

165 in the' process of beingmatched to services and another 200 or

so who have applied and been accepted for their orientation in October.

We have developed something on.the order of 400 service opportunities.

Seyeral hundred more are in the process of development at the moment.

They cover a variety.of needs areas. A major emphasis has been on the

quality, we think, of the projects, and some of them are particularly

significant. For example, we will.be,producing Syracuse's only bilingual newspaper and, in fact, the only Spanish-language newspaper.

In,terms of research issues, I think there are' basically two

major dimensions. The first one is the feasibility of this model of

a voluntary community-based youth service. What is really the capacity

of a local community to understand And implement this concept of total

community involvement in support of its young people? Secondly, what

- 49
is the capacity of sponsoring organizations,, neighborhood groups,

public agencies, established_ voluntary organizations, and youth

themselves to develop and supervise meanin ul projects that engage

youth in a.variety of really demanding task ? The major thrust is

the implementation analysis that is being un ertaken by the Research.

Corporation of the City University of New York. That analysis will

take anlin-depth look at the whole process of implementing and educating the _local community in terms of witat YCS is all about and involves.

There will be ethnographic examinations of 15 to 20 projects for the

"hunches" about what works and why. These will also consider issues

of replication--whether or not the project could be done somewhere

else. The Youth Community Service concept involves such conceptual

changes for program administrators at a local ltvel that it cannot be

done on a simple sort Of revenue-sharinr, no strings attached, basis.

The question, therefore,'is what kinds of easy strings could be used

to get this kind of notion across, but not encumber the program

seriously.

The other dimension is the effects of the various program elements

on the' various types of youth who participate, as well as the effects

on the community at large. In-terms of impgcts, we want to find put

if youth adopt the philosophy or ideology of community_servicb.

Then, we want t find out more about individual choice in their

selection of theb-'13rojects. We don't match people with slots. They

select the projects they want to serve in.. We will need to examine

day-to-day activities in the projects, what the youth have, in fact,

engaged in, and whether it's meaningful and non-traditional in the

kterrs we use. We will look at the type of sponsorship, if it's a

neighborhood group as opposed to a city agency; the type and quality

of the supervision; the possibilities of attracting volunteers from

different income levels, since there are no income eligibility limits

on the program. ° Finally, there is the question of participation ip service learning

efforts. Will volunteers who efgage in service learning have a better

experience than those who do not? Will it be'a critical part of the

4 program?

The Urban Institute is in the progees of developing a series of

research designs by which to valuate some of thes fectS and exper-

.iences. 'Because of the proce s of educating and d oping -;ystems

in the first six months, we d d not proceed headlong into the

design of research before we really knew what the relevant factors would

be. We wanted to get some of those systems basical'y established, so

that the research designers would know what, in fact, they should

evaluate. The major thrust will be" discovery; not'specifitally

hypothesis testing. There will be some element bf comparison with other

programs. We will be gathering the same descriptive data on our YCS

participants as. is being gathered in Other DOL youth programs.

MR.. SEILER: The Services Mix Alteinative Demonstration Project addresses

the question about the most effective mix of work and services--whether

youth will benefit more by standing and sweating or sitting down and

being served. They will be in Broward County, Florida, Oakland,
. California, and Los Angeles, California. These are just starting.

The project is designed to assess the relative effectiveness of

three basic alternative approaches that prime sponsors frequently use

to assist youth. The first service approach is the labor intensive

mode of participation in work experience. ,The second service approach

is called a service support mode. There is no work experience. All1

paid hours are in classroom type activities, skill training, educating,

orientation, group counseling, that sort of tjzing. The third service

approach is a mixture--a suppofted employment mode. Half of the paid

houis are for work experience; the other half are class-type activity.

In each of the three projects there are 300 youngsters, 100 in

each of the service modes. Each project hes a random assignMent

process for participants and is matching them in.pairs of three.

Thesematches are established by sex, race and academic proficiency.

The three pairs are then randomly assigned to each service mode. We're

also controlling for the amount of dollars taken home by each youth so

that we don't have the distinction between allowances and wages. In

addition, each participant regardless of service mode receives job

development and placement services from the same, program unit and

hopefully -le level of effoft for each will be the same.

The duration is 4 monthS. The initial two months are for start- . up. The last two months are for the analysis and write-up. In the

middle, of the 20 months, 12 are for program write-up operation and 8

'months are for follow-up. The follow-up includes 30-day, 96-day, and

240-day follow-ups. The duration was dictated by the fact that,two

years was the maximum amount of time that the program could b§ funded.

. There are two major types of information in analyzing effectiveness.

First, we have a series of behaioral change tests, which are made at

program entry and exit. These'include measure5of job knowledge, work

attitude, job-seeking skills, self-esteem, and acaddmic proficiency.

The second major set of_inforMation is the outcomes on employment,

earnings, and education in the 3 follow-up periods.

We have some problems in measuring costs where services are

received from other sources, but we will try to tie these down as

much as possible. The behavioral c ange measures that we use a

I

re

as good as are available, but we ca not get two people to agree on

how you deal with measuring work attitudes and other things.

DISCUSSANTS:

4

MR. PALMtR: It clear froM the presentationsthat we're talking

about a tremendous variety of projects and objectives. There are

five categories, but within a couple of those categories, there's a

multiplicity of different kinds of demonstrations. I am struck not

just by the variety of activity but also.by the different assumptions

that seem to underlie a lot of the programs. For example, the Career

Intern PrcAgram is really a ternative to traditional schooling--an

attempt to enable peopl to complete the degreeprogram. On the other

hand, a Jot of the Ex plary In-School Programs are an a tempt to

redirect the school ystem itself in order to serve more uccessfully

the youth who. are i vo,lved. On the other hand, the Commun ty Youth Service,

- 51 -
Project is taking youth who are out of school and giving them as much

as a year's experience, in some cases, with the expectation that they

will certainly further their education. Nevertheless, it is entirely

out of that educational framework. So there are very different assumptions underlying these different approaches as to what may be needed

or what our assumptions are about the failures i the system.

So, to some extent, this variety of activities and assumptions

reflects both strengths and weaknesses in terms of where we are right

now with youth employment policy. It's indicative of ho little

we really know. Also, it may be that we're trying to do too much

under these particular pieces of legislation and demonstration right

now, and Vlat the resources that are available for knowledge development

are being Stretched too thin. -There's been a lot of people who've

mentioned the speed of implementation, the sparsity of resources and

so on. Still, it is an extremely exciting and unique activity, I'

think, with tremendous promise to be very productive.

The stren ths are the fact that there are a multitude of exemplary

projects that seem to exist out there now that have considerable promise'

and they',re all being looked at, or at least most of them are,being

looked at, in one way or another in much more systematic form than,has

-previously been the case. The new projects that are being started

seem to me to be indicative of a tremendous amount of imagination being

brought to bear on this problem and also a willingness to examine interventions that are based upon very different views of what the world is

.like out there and what might make sense. That is, there's an incredible

open-mindedness about the whole approach.

- I' Al struck by the extent of use of"intermediaries--bot intermediaries t at are nongovernmental as well as other agencies thro gil

interagency agreements. The intermediaries are really playing a dual

. role. One is.that they are providing the research capacity to examine

what's happening out there, but they are also, 'in Many cases, playing

a very .strong role in terms of getting the operation going. 'In the

interagency case, there are very different mechanisms,'a whole variety

of them being used, with more or less participation of federal agencies

and their counterparts at state and ldhallevels. I hope that'a fair

amount of effort is put on .4he evaluation of these approaches. We

must try to understand and learn getter about the role of theseintermediary organizations and other federal agencies, in terms of both

structuring and delivery and the research aspects of the whole effort.

I can imagine that in many cases, what might happen is not simply

jumping from a demonstration to a national program, but beginning to

build on the kinds of structures that are out there now, using intermediaries a$ technical assistance vehicles aid other kinds of things,

to work with progfam operators at different local levels.

The capacity, in many cases, simply won't be there, to go from

whdt can be successfully done in one area to replicate it on a natio 41,

level, but over time, using the kind of reservoir of talent and exp tise

that's beginning ,to grow up outside of the federal government it may

be possible. A major development in the last 10 or 15 years, is the

extent to which'there's.a capacity growing outside of the federal

government--some accumulation of thd knowlddge and some continuity.

52 -

6
I hope that this effort really-promotes that even more an that there's

some focus made to stand back from that set of intermedia ies and ask

what role can they play in the future? .What have they done well-in

this process and what haven't they,dbne well? What gaps are there

for other kinds of intermediariesrperhaps, that are lessresearchoriented and more operationally-oriented,_ in terms of the kinds of

assistance that they can provide?

Several of the program approaches that are being tried out are

predicated on the assumption that that particular program may the

only one in the environment, if it were done on a national level, say,

or if it were a major program. For example,' the entitlement programs

are really an attempt to deal with the universe of need among at least

low-income youth. The Youth Community Service Program is, in some

sense, the same thing. Can we create enough slots or service opportunity

to meet the demand of everybody who would want a job? In reality, of

course, .we're not likely to end up with a world0ahere one of these

programs would be the only thing that would be available. We're

probably/going to want a whole mix of them. In fact, the notion,

in some cases, is that the youth population out there is kind of

monolithic and. either the entitlement program will work or it won't

work for disadvantaged youth or either the CYS will work or won't work.

But I think we have to recogniie that we're going to need a kind of

a "different' strokes for different folks" approach. Some combination

of a number of these different programs in the environment at the

same time is very likely to be desirable.

I don't know how you deal with thatwhen you start evaluating

these individual programs, but it's important in interpreting the

results, I think, to keep in mind that you shouldn't look at them

as though they're going to be the only program in the environment.

In some sense, this may mean thSt they will appear less successful as

individual programs than the entire group of programs would apppear

or would be in any given environment.

A goal of the CYS and the entitlement programs is to measure.

what the demand will by for these opportunities. In fact, it's being

talked about even more.abstractly. What is the demand for jobs on

the part of youth? ,Well, I guess one of the implications.oE what I'm

saying is that, these programs are not really going to be able to get

at that. What I think you're goirig to be able to measure to some

extent is the demand 4n a particular locale at a particular point in

time for this partigular kind of service opportunity or job. Npw,

that's a much more limited outcome than we'd like to know, but yet

it's still going to be very helpful for program planning purposes and

will begin to tell us whether the kinds of results that are coming,

.out of the aggregate models using CPS data make any sense at all. I don't think they're coing to enable us to get precise estimates of

what.the universe of need or participation will be, but clearly it

will be helpful.

As a third point, I think there's probably going to be an important need for the ability to construct control groups for a lot of

these programs out of data bases that are totally independent of what's

- 53 -
being done'in the context of the program. Such constructed control

groups can be used for many different programs to enable some

comparability across programs. I would think that the longitudinal

survey clearly offerg'this opportunity and I hope some thought will

be given to how that can be structured in such a way as to enable

different samples to be drawn that can serve as control groups for

comparison purpose to participants in various programs.

Fourth, it i clear in number of thediscussions that it's

-very,difficult to pay much ttention to variations in the demand

side of the labor market-an in the economic conditioris that are

going to exist in the individual sites. "The outcomes you may get

' under tight labor markets in one city or one point in time for a

given program may be very different from what you get under diffeent,.

labor market conditions in other areas or at other points in the

business cycle. Rather than expecting _that, in the context of any of

the individual demonstrations, much can be done about this, what's

needed is some funding of systematic research to try-to expibre

.better--maybe at a more theoretical level--how to get a hanale on

these questions:. "What effects do different variations in labor

market conditions have on the outcomes of programs like this?" and

"How can we better measure what we &rne-n by tightness or looseness of

labor markets for the populations we re concerned about?" Understanding

of the demand side and of the broader economic context may then be

applied to the individual demonstration efforts.

Lhav6-1a,sense that there's an unwarranted degree of expectations

and optimism regarding the precision and value of what the r research

results that are going to-come out of a method like this really can

be. Therp are a number of reasons for this. It's important to-reeag7: _ nize thae,the reSearch' methods we have available to us in the 'social - science disciplines are simply not very powerful'in terms of trying to

understand the kind8 of outcomes, the kinds of effects, the kinds of

interventions, that are being tried''out here.

I've had considerable ipmiliarity in the past with the income

maintenance experiments. There you had as simple a( intervention as

you can imagine, in terms of what was being tried-[direct" cash payments,

uncler very controlled conditions, where you were looking at variations

in two things: the amounts of payment that was given related to income

and the marginal tax rate or the rate at which benefits were reduced.

Even so, after a,.set of five or six different experiments under very

.controlled conditions there was considerable controversy over the exr

tent to which we really do understand what the responses of people

are to,variations in those two different variables. On the basis of

information obtained, however, the uncertainties have been reduced.,

We can now say there hasn't been a massive withdrawal from.the labgr.

Market as a result of it and that's an important result. I think

there will be eounterpaits to those Rinds of results coming out of

these experiments. But it's only after 10 yeafs and several experl nts

that you really have the'sense that precise estimates of labor sup y

are good enough for use in your cost estimates. It took wel..are

reform that'long and we're at least 10 years behind that kind of effort

in the youth employment fields, whidh in most cases, is not going to

lend itself,to that kind of precise treatment in any event.

- 54 -
Secondly, the speed with which we can get answers has been

addressed quite a bit and I think it's important to recognize that

it does take years and years and an accumulation of research and

understanding. Programs must have a chance fo evolve and settle

down, in some sense, to get over the early mistakes of initial implementation. , even where answers can be forthcoming, in ma cases,

it is going to take 3, 4, 5, 10 years, and I think Congress mist be

told. that. Frankly, they're going to have to understand that you can't

simply pass a bill and expect that 2 years later, you're going lo get

Canswers to such critical quOttions as they're asking,in this legislation.

Y. Thirdly, a range of different efforts are going on here, but only

a very few can be talked about as experiments where you structure . t em in advance, systematically varying certain kinds of treatments,,

:3----measuring the results, and.then try to understand what their effects

are. Unde the best of conditions, when you have experiments, it's

difficult to get precise research estimates. .ender less controlled

conditions, particularly where there's the tension between operation'l

concerns and research concerns, it becomes very difficult to get the

kinds of answers to the questions we're asking and that simply has

to' be appreciated.

A fourth problem is the replicability;problem that's been raised.

I don't-think I have-to go into that in'any detail, except to point

out it's not only a question of replicating it because of the uniqueness

of.a particular site or the uniqueneSs of a particular program, but

,there's'also a question about the capacity of the system: We're starting

out in many cases with the "cream of the crop," fn terms of the prime

sponsors and in terms of independent program operators, and so forthi

and trying,ideas out with them. That doesn't mean, if it works thert',

that you can, in a couple years, just have a national pram that, will

work as well.

As I suggested earlier, the way we get from here to there may

have to be a very slow cumulative process with the kind of expanon .

of the network of expertise and bringing up the speed of people at

local levels, where they do not have that expertise now.-

Also, on the funding side, on the intermediary side, andon the

technical assistance side, the capacity is very limited. It's already

stretched too thin, perhaps, indterms of -t of demonstrations,

let alone what it would mean if you tried e national progrSms

in many of these areas within next couple of years. So, we have

to build up 'on that side. I'lhiMOsome attention needs to/te given

to that in terms of the interpretation of the results and a.16t of

thought must be given to how we get from here to where we want to go

with those programs that do appear to show the promise bikf tieing valu-

&Ae on a national scale.

Last, I think there's also a question of our ability to define

conceptually; in terms that can be operationaly.justified, the'

specific treatments for interventions that we're concerned about.

9
Many of theM just don't lend themselves to the kind of clear definition

tI- itenable us to test them out and talk about how changed in them will

effect changes in outcome. 'The kinds of interventions we're talking

about here are extremely complex inmost cases. Taking.the example

of the income maintenance experiments, there you did have two precise

parameters that ypu could control in an operational-Asense and then try,

to look at their effects. But Mere, the interventions are really the

programs as a whole ana there's no way to separate out Thdividual

components of them in most cases.

Some of the discussion seems to suggeA that were going to be

able to vary components or individual elements of these programs'structures and see what difference that make's. I'm dubious thatAthat's

going to be possible in most cases. I just don'tfthink that'the

metflodology or the resources that are being brought to bear are going

to permit that to happen: You're just'not going to get effective

answers to those questions in the strict research sense. You're going

to get some important answers out of the more qualitative analyses

that are being done and from the hunches that have been made. It.will

be unrealistic to expect our statistical techniques, a's they're applied

here, o be able to answer with certainty oz within prescribed bounds

of cert inty, wht those variations in particular treatments are.

Obr be-st hope is to get some overall sense of the effect of these

interventions at the general impact leVel, and that is an extremely/

important first step. to take.

et 1 One exception to this may be the wage subsidy experiment, which

I think is the closest thing to an experiment'that does lend itself

to the kind of treatment not possible in a lot of the other programs.

I hope that will be given a lot of thought. It seems to be, since

it's being very slowly developed over time. If the targeted employment

tax credit passes Congress, as it appears it might, it creates problems

in what ought to be done about this subsidy. It might be-useful for

the Labor Department to test that legislation, in the sense of asking

for some ability to dp,some experimentation that ties it in with what's

being done here, so that a year or two from now, we won't just have

to live with the results of the targeted tax credit as structured in

the law, but can also say what variations in that particular program

might yield in differential results.

This leads me-to the following conclusions. The importance'of

this entire research effort, .at lea§t'in the short-run, ought to be'

focused on first-order questions. We ought to make sure we get answers

t

to.such basic questions as: Was feasible simply to mount and execute

the ptogram under the design'cond'tiohs that we were tuing to accomplish? Who is Being served? Are we reaching the target population?

Is it working, in some sense, at that level? That's'the best inform,-

ation you can hope to get from this within the 'next year or year and

a half. We're not going to get much further than that. I hope that

(,

enough attention is given t fhose questions and is not focussed on

some of the ones in which th' answers are necessarily going to be

equivocal. We must not miss getting this. basic information out of the

1 D - , V

< - 56 -

e
experiments. I'm optimistic that the more sophisticated systems; being

put in place to provide the data to answer the tougher questions we

won't be able to answer very well, will yield the kind of data we

need to answer the easier questions. The kind of rigor that's being

brought to bear,.because of the quality of research people that have

been brought into this, is going to insure that there will be a

systematic look at a lot of .these fundamental questions. There are

'the kinds,of questions in Which there's a great deal of interest and

from which a great deal of knowledge can be derived to inform Congress

and to provide the grist for the PR work that needs to be done.

The same priorities suggest themselves to me by the schedule

of timing for the legislation. If I work backwards to inform FY81

decisions, the administration really has to decide what it wants to

do by this time next year.. That really means that we've got about

six months--maybe nine months--to generate information out of this

set of activities, to inform that decision-making process, at least

within the administration. You have a little bit longer to develop

new information for the Hill.

But I'd like to reiterate the notion that we shouldn't be pushing

for more sophisticated answers to the tougher. questions in that time

period. That would be a mistake in terms of the confidence one can

brave in the answers, and more fundamental questions about just what

seems to be working and what can we do successfully, even though we

don't yet understand the full impart.

MR. NICHOLS: We're looking now to find out whether mistream correction

shouj.d be made in this knowledge development plan and whether the kinds

of suggestions I might make should te taken as self-criticism as much

as of anythixig that's in that original knowledge development plan, since

I do have some overall responsibility for research design.

0 I want to emphasize the need for consistency across these various

L.,. programs. John Palmer just said rightly that this is a grab bag of

projects we're.discussing on this particular panel. There are many

different projects demonstrating many different things and it's going

to be very hard-to draw generalizations from them.

We want to strive to bring about'some kind of consistency, so that

we can not only make comparisons withiu...each of these projects, but so

we'll.also'be able to make pretty good comparisons of one approach

against another approach. It should/be our objective to try to compare

across and within these demonstrations, but the way this appears now to

be setup, that will be much more difficult then I would like to bee it.

I'1 have some suggestions later on as to how we might change that.

First, let/me try to convey to you the spirit with which these

results are received in the Labor Department. It is not our objective

'to have every. program get a.positive evaluation. We do not want to

have all of you tell us that everything works, because the purpose of

this is to find out what works best or what doesn't wdrk. We'll be

very annoyed if all these evaluations come in saying that everything's

great. It.will be very helpful to us if we find out that some thing's

-
9.

don't work at all. We tr d them and they were ideas that seemed

promising, but we had to discard them.

On the research side, it's probably always true that the majority

of r search never pays off. You just don't know which proportion ahead

of t me. We have no way of knowing which of these things are going

to w rk and so I think it's appropriate that we are trying lots of

thi gs. That doesn't mean we want them all to be successes. They are

in no sense all prototypes of what we want to do. We would be happy'

to see some failures, a whole mixed bag of results, which will alaowl

us to rank these in relative terms. I know I speak for my boss

Assistant Secretary Packer and for his boss, Ray Marshall, who is

very interested in letting the chilSs'fall where they will, in an JP

intellectual sense.

A feature common to most of the demonstrations,being discussed

in this panel is that they are not experiments with random assignment

groups And the like. They lack the pure.classical experimental approach,

in that all the results are hedged ahead of time. Some of the researches

sound a great deal like advocates rather than scientists. It may well

be that advocates run better programs and that may be a model for program

runners that we do want to tap, bi.A.it's probably not a good model for

getting research results on something you might think you could replicate on a large scale.

If I wanted to compare Tesults across projects, what would I need?

I guess I would just look at the textboOk Way you do research. First,

you should state the goals of the program. The overall goals of the

youth programs are pretty much the same. We want to reduce youth un-,

employment. We want income gains to come out of this and A variety

of social things like reduced crime rates. Skill development, school

completion, and the like are almost inputs to these other larger goals

that I mentioned, that is they presumably result in better employment,

more income, and the achievement of the social goals.

Since all c.f these programs pursue different goals, the program

should list ahead of time what the goal of the program is, ,how the

model pursues that goal and that should give you a very clear idea

of what you can test, what you can learn from the program. The

hypotheses should be stated clearly. They tend to be fuzzier tflat I

would like to see them .-If they are stated clearly, cleanly, and simply,

then we have a much higher probability of coming up with something at

the end., a much higher probability of being able to compare results - across projects, than if what is being shown is whether a very specific

kind of demonstration can fly. We need to learn about the gdneral

results more than the very specific ones. Sometimes we learn general

results from running specific demonstrations, but they should be

designed in Such a way as to yield some insight on the more general

results.

One place to start, then, would be with the data collection. It

would, be helpful to have the concepts that are used here defined in

the same way for all the projects. It would be helpful to be able

to use the-data that these people:Collect for other purposes after

.7. 58-

71
this particular project was done. It would be helpful to go in and

find out why seemingly contradictory results came out of these projects.

Therefore, it would be nice to know that these people had agreed on

lc

' definitions of what was skill training, what was work, what was whatever

the conceptS are that have to be defined. There needs to be prior

agreement on concept definition before the data are collected. There

also needs to be prix agreement on categories of data collection,

so that results ca e made comparable and one huge data set can be

put together encompassing different aspects. Ideally, we would like.

a very expansive data collection effort to be made for all of these.

* I realize that's not practical and so that all we can ask for, in some

cases, where the data will not be exactly consistent, is that there be

documentation of the data--extraordinary documentation. That is, how

was the data collected? How were various things defined? What is the

difference between your definition and definitions being used on the

other projects? I think it would be nice if these contractors would

share their definitions with each other and would sharpen up ahead

of time any differences in concepts that they did have, so that we

would know, after the fact, if we wanted to use all the numbers together

4nd what relation they bore to each other. I would think an excellent

part of the final project report *buld be a very precise documentation

of data, so that it could be replicated and linked to other project.

. As for research methods, 'I perfer to start with the classical

experimental model and then,ask, "How does one pitoject differ from

another? What did we do differently from the classical experimental

approach--which is the way we're surest of our results An that we

know exactly the probability with which we can accept.or.reject

hypotheses? How does your project differ frdm'a classical experiment ?"

Many of the discussants have already hedged their. results carefully,

saying that we cannot expect too much. .Yet it would be helpful to

have those hedges stated in a very preci64 way and to link them to

what the classical experiment would have been. °%"What intellectual

compromises have been made and why?" The answers tb,that would allow - us much more easily to compare the kinds of results khat we get from

each project.

I've heard a great deal of comment about our.being very modest

in our expectations of all this. Well, for $100 million, we can't

be tbo modest and we do have to expect some results. If our science

is such that it can't yield precise results to subtle questions, then

we're going to have to ask simple questions.

It's much °more useful to get good, solid answers to simple questions than it is to get fuzzy, u eliable "hunches" to hard questions.

Many researchers have said today t at we shouldn't place too much

burden on research, asking the im 'ssible and coming up with a bag of

smoke. Maybe we really should concentrate on the simple things that

we can answer, rather than orr about asking questions we know we

can't answer.

Let_me.say that I am pressed with the variation within these,

projects. There are a good many things being tried and I think all.

the bases have been touch d and most of what should have been tested

is be4g tested. I think hat's exactly what was needed. Its just

- 59 -

S
L-:

that this incredible variety could yield a bewildering set of contradictory results, unless some consistency is imposed by us at the top.

GENEhAL DISCUSSION:

4e

MR. JONES: I think we may be overly concerned about the lack of

classical experimental design. One criticism of the classical

experimental design is that it controls for type one and type two

errors but there may be a type three error--which is asking the wrong

'question when you might have asked the right question. We may be

trying-to find increasingly better ways of doing things that probably

should not be done at all!

MR. TALMADGE:A I think a wrong assumption is being made by various

members of this group--that it's easy,to do good, research, or at

least it's easy to do good research answering simple questions. Going

back to the comment about ,the classical experimental design, I think

probably everybody in this room has heard of the Hawthorne experiments.

Those were designed to address a very simple question--does changing

the lighting in work area affect productivity? Of course, they got

very strange results. Every time they lowered the amount of light in

a work area, productivity went up and it reached an all-time high

when people were functioning at a level of full moonlight. I don't

think that very many people know that this particular set of experiments

involved a control group. It turned out that every time change in

performance was observed in the control group. Everytime the

treatment group's lighting changed, the performance changed in the

control group, whose lighting was not changed. If we had implemented

the classical experimental design and simply compared the two groups,

there would never have been a Hawthorne effect.

It's clear that we can do statistical analyses and reach conIlusions, but a great deal of thought nee ft to go into considering

possible alternative hypotheses for assembling diverse kinds'of data

to build a case. o MR. BRIGGS,: I feel we're missing what is.perhaps the greatest contribution of these programs in the discussion over classical research

design. It seems to me that the major overriding focus behind all

this is changing institutions.' Programs are effective if they can

change institutions in desired directions. The apprenticeship system

has changed a great deal because of apprenticeship outreach. Vocational

education changed a great deal due to competition from CETA. What's,

really missing here is concerrkfor institlitioNal change in the most

important institution of all affecting humans--the educational system

itself. Many of our schools have not done a very good job, particularly

for the youth served by these new program. Ih some sense; this

4slation has come about-at exactly the right time. The schools are

in trouble right now. Enrollments are declining. They're running

tout of people. They're receptive to new ideas for the first time and

ready for us to move in.

On the, other hand, as I've moved in and studied at the grassroots

of this program, just going in and talking to the school people and

seeing what they do, I find ithat there is a tremendous amotint of things

- 60
being done in schclols that the literature doesn't even mention.. The

new resources have allowed some of these schools to do things that

they could not do before because budget cuts were taking away their

staff, their pepple, and their initiative. It's encouraged them to go

out and do some entirely new things.that they've never tried to dobefo5e. I think that finally, after 15 years, we're addressing the

most important institution of all in this whole power establishment.

If we don't get the schools straightened out somehow we're going to

have manpower problems'forever.

There's been a good deal of talk about uniformity or getting

uniform measures of program outputs or program objectives. However,

I think there may be a confusion of means and ends in a very basic

sense. I think, when you consider the whole range of employment

training programs, the results of the research and the demonstrations

are used by a number of different actors and, depending on where those

actors sit, they have different agendas. I would go as far as to say

that I think that a typical legislator would probably think more in

terms of whether services are actually delivered than in terms of

subtle assessment of impacts.

The distinction between ends and means. is very critical. One

program that is a very good example of what I'm talking about is

Entitlement, If you look at the legislation, the ends are really

defined as keeping kids in school or getting them back in school.

There's an implicit assumption that this is desirable and there's

no mandate to test that assumption. , I think this sort of phenomenon

occurs in other areas as well. I think there's a real problem because

the people who use the research may not have in mind the same broad

objectives as the people who are doing the research.

MR. POULARD: Perhaps there are some things we don't knqw that we. stiould

learn and, upon learning them, might apply them seriouslNand honestly to

address the problem of unemployed youth, But I wish the solutions

to the problems that have been raised were that.simple. I wish I

-could believe that it i$ the absence of knowledge which explains those

problems.

I find it difficult to believe that .the high incidence of unemployment amongst youth is explained primarily in terms of what we don't know.

I have worked for a number of years with populations for whom these

programs were created. As a practitioner, I can say there isn't that

much mystery, as might be suggested, with further discoveries in

uncovered truth., ,T wish that were the problem. It would be easy if

the accumulation of a few more facts would provide the remedy. But

the residual of so many basic societal patterns and attitudes, political

stances and the like, seem to be so obviously at the heart of the matter.

Any serious effort to , understand unemployment would also have to address

the reordering of priorities that are not gracefully acknowledged and

addressed. I do respect the fact that when one goes to the Hill with

empirical data that is hard to-deny, regarding the cost .effectiveness

of one program or another, perhaps he does get armament to sustain that

prbgram. But it's more armament in terms of rearming a B-B gun as

-,61 - '7
opposed to pulling out the cannons to deal with the enemy or the

problem. I don't know that, by coming up with cognitive reasoning

and new empirical data, we will address the attitudes, kactices,

orientations, and commitments, or lack of commitments, that seem to

be the x-factors that explain the historical problem and the everincreasing problem of unemployment among youth.

I wish that in looking for answers designed to help remedy the

problem of unemploytent among youth, that.data found might be

presented with passion or that research engaged in might be done out

of commitment, so that there is not merely numbers to surface,. but that

there is support for a cause. I wish there could be "advocate research."

I don't think that bastardizes research at all. It tempers it.

It suggests perhaps a likelihood that the efforts of the researcher

might better foster the objectives of the Department of Labor--and

wIlI don't imagine any researcher would have engaged in a.contract

u less those objectives appeared, even to the pure scientist, as being,

orthwhile, so that a case might be made for, for want of a better.term,

advocate research, without denying the integrity'That the scientist

wants to hold onto.

I would hope"that it is not so much numbers and abstract charts

and printouts, but rather commitment to the resolution of a problem

that is factored in this as much as anything else in the process.

Practitioners like myself don't have much to contribute to the formulationion of a question that would provide the kind of answer which is

better or more easily collated in the computer, but practitioners who

are wording in areas addressed by the Department of Labor do want to,

say to the scientists, "We hope that you find ways to utilize whatever

data that you do surface, to advocate as best you can with that data

the causes that the Department and the legislation and the Department's

regulations do uphold."

It is too risky, too hazardous to just assumethat it is appropriate

let alone judicious, to take the pire researchers' approach. If the'

attitude behind the process is one tpat is devoid of passion and' com-

'mitment, that is not a virtue.

v 62
i

SESSION IV: BASIC RESEARCH

PRESENTATIONS:

MR. LERMAN: There's much to learn about what's happening and what

has happened in the youth labor market. We've started to identify

some of the important questions that we think are critical. They're

broad questions, but we think we're moving in the direction of finding

answers to some of them, and a year from now we'll be in an even

better position.

The first question that I think is critically important is to

ask why the racial differential inayouth unemployment worsened in

thRlast decade or so and elPecially in the employment population

ratios. We see and have seen successes in equal employment efforts

and even successes in narrowing wage differences among black and white

workers. Given the fact that the situation was bad enough in the

beginning, it is hard to understand how it's gotten considerably worse.

The second question concerns the causes of such a generally

high youth unemployMent rate in our naelon. In European countries

the relative-and absolute levels are much lower. Is turnover really

the problem that people say it is, or are there other problems that

cause us to observe, even in tight labor markets a high youth unemployment rate?

The third question concerns the seriousness as a social concern

of youth'unemployment. How is the unemployment spread amoung youth?

If it were fairly evenly spread, presumably it would be less of a

problem, How does it affect the current income o"families and the

young Terson's experience later on in his career?

Some answers to'these questions would cause us to be less concerned

about youth rates and some more. Without going into any research findings, I'd like to point out a few interesting facts that demonstrate

why further study is needed. For instance, it is frequently observed

that youth unemployment is very short term in nature. This conclusion

rests on data from Employment Earings, where it shows that 40 percent

of that youth unemployed are unemployed less than five weeks, and

another 20 percent between five and 15, and only a small percentage

on a long termbasis. That's true at a point in time, but if you

look at how much unemployment youths experience over the course of

a year, you 'find that most youth unemployment is borne by people

who experience moSe than 15 weeks' unemployment°. As a matter of fact,

something on the order of,75 percent of all the weeks of unemployment

amoung youths were borne by those who had at least 15 weeks or more

of unemployI'ent during the course of the year.

A second f ]cequent statement is that when unemploytent rates are

relatively low, the people that experience a lot of the unemployment

are youth. While the incidence of unemployment ks high amoung youths,

most of the long term unemployment during low uneinployment periods is

still borne by adults\. It turns out that dip group that made up the

highest percentage of long term unemployment is prime age adult white

-males.

a CI
If you look at the unemployment numbers that come out of.CPS,

which are the basis for the 40 percent estimates of,blaCk teenage

unemployment--you find that these are not comparable with,numbers

generated from other sources. I looked at about five'or six sources,

from various surveys, all with relatively similar concepts yielding

very, very different results.'

MR. WISE: We have been looking at several different sources and

types of information to first try to identify the dimensions and the

characteristics of youth unemployment and then to see if we could find

factors that seem to be contributing to or simply associated with

the unemployment. We've been looking at,cross section data, some

time series data and some .ongitudinal data. This means that we're

looking at Current Population Surveys, Census data the National

Longitudinal Surveys and another longitudinal study put out by the

National Center for Elucationai Research.

The first-finding is that we get very different results on basic

things like the unemployment rate from different sources, namely,

Current Population Surveys versus longitudinal data. For example, if

you compare results based on the National Longitudinal Survey with

Current Population Survey data, it turns out that there is a month,

October of 1972, where one can make a rather good comparison.

You find, for example, that the unemployment rate for white

males based on CPS data is 11 percent._ That's high school graduates,

by the Way. The comparable figure from the Nation -al Longitudinal

Studies is 5.4 percent. For blacks and CPS gives 23 percent versus -

13 percent from the National Longitudinal Study.

We don't know the reasons for this. It's not simply a matter

of definition. Maybe it has something to do with the fact that one

is a household interview where the questions are asked of the head

of the household and where the other one is a questionnaire ans ered

presumably by the youth themselves. However, the difference eemed

to us to be much greater than could be accounted for by the differences

between the.questions that were asked.

It turns out, not suprisingly, that there are substantial variations across areas. The variations tend to be associated with

demographic differences across areas; that is, the proportions of

the labor force who are youth tend to be related to aggregate, demand,

which is no surprise, rate of growth in the local economy, and the

movement of industry from'central cities to suburbs.

AnotheAtfinding--I should,say a non-finding--is that there is no

evidence that high school training--industrial-vocational training- -

has anything to do with earnings or employment after participants

leave high school. It does turn out, however, that persons who worked

;through high school seem to have much better experiences when they

leave; that not only inierms of wages but also in terms.of how

much they work.

I I - 64 -
It turns out also that unemployment, as one might expect, is

linked to other social problems like broken homes, families on welfare,

violence and crime. For example, we conducted a preliminary survey°

of youth in the Boston area. We had a student go to basketball courts

and interview people who were playing basketball. It turns out that

the first/20 people interviewed indicated that they had committed a

nonviolent crime in the past week,

These suggest the kinds of things we're examining, and I would

like to indicate some of the possible implications. It appears that

aggregate demand policies can reduce the youth unemployment rate,

but they are not going to so ve the problems of the youth that are the

'hardest hit, such as bla c.sigh school dropouts. It also seems to us

that policies need tgebe directed toward this group; that is, policies

.directed toward solving the youth unemployment problem n.general are

not, going to do the trick for the.people who are thp most severely hit.

And then one has to think about bringing these youth into the labor

force. Most of them are simply not in the labor force.

What about research priorities? Well, it seems to us that we

need information on youth that are not in the labor force. Current

L..

information is simply no v/ y informative about people who are completely out of the labor forc . We need to know how they use their time,

how they support themselves, what they are doing in general, how they

get jobs when they get jobs, because eventually most people move into

the labor market in one way or another. In that is implied that we

need to have research that is foci4sed on this rather narrow group,

although it also needs to have a broad perspective.

.... And then one final thing. It seems to us that there's more-. that

must bepdone to relate problems of youth unemployment to personnel

policies of employers and other demand side practices.'

MR. TOIKItA: I'i going to summarize two studies for you. The first

one is a rather small effort which is nearing completion, which

investigates the potential impact of employment and training programs

on youth unemployment. The second'one is directed at answering, the

question of what is the social and economic significance of youth

unemployment. In doing the first study my objective was really to

look at the process in a decentralized system such as CETA by which - a federal grant to a CETA prime sponsor is translated dnto labor

market impacts. In thinking through--this process I was able to iderit-

. ify 11 independent factors that come into play. Their magnitude

surprised me. Some of them are self-evident; but What I attempted

to do was to put all, of the factors together in an analytic framework

which would allow us to say what the impatt on ybuth unemployment would

be of introducing a manp9wer program and leaving it in force. If there

is no behavior change, if the people leave the program just as they

came into it, then the program h s a small statistical impact on the

labor market-, :depending criticall on the targeting of the program and

the job placement rates. I'm in he process now of trying to calibrate

the model to simulate the impact of the YET programs. The program

can be effective in getting youth nto jobs, but if all that happens

is that they're in dead-end jobs efore they came in the program and

they're in dead-end jobs after tley leave the program, there's not

i

r 65 -
going to pe any substantial long term reduction in the unemployment \\\

rate. It's fairly self evident, but I think the strength of this

analysis is that it-allows the integration of research findings.

Topics amenable to empirical research include fiscal substitution

and crowdif4'out of jobs in the private Sector and the increase in

labor forte participation rates which may occur because local labor

market conditions are made better by the program. When it becomes

clearer what the empirical-Xagnitudes of some of these things are,

it provides a'method of seeing' what labor market impacts will be.

The second study, is larger and it's asking a very basic question:

"How serious a social and economic problem is youth unemployment?"

I think we'd all certainly be in favor of reducing unemployment, but

tlie tough part comes when you realize what the cost of this might be.

For instance, suppose we faced the uneasy choice of reducing youth

unemployment at the expense of increasing unemployment of other groups

in the labor force. Even though our focus is ybuth, 'es our feeling

that the unemployment rate is an inadequate measure i many ways. It

.ignores discouraged workers--people who've left the 1 bor force because

they 'can't find jobs. It ignores involuntary part-ti e workers. However,

One could make the reverse argument, particularly with the youth pop -

ulation,'that unemployment may be less serious than for adults for a

number of fairly'obvious reasons. For example, youth are,less4',

frequently breadwinners for their families. Their non-market opportunities such as school or leisure activities are typically grea.4er than

for-adult workers.'-Ahd so we approach this by trying to sort 'Out he

issues. Which groups are worse off, when you take broader look t

it? You just don't look at unemployment, but must distinguish be een

full-time and part-time employment, the reasons for unemployment, and

much more. 1 The technique of analysis that we're using is to take a March

Current Population Survey and link it with an April survey so that

we're able to observe changes in labor.force status that occur from

month to month and really aiNect the analysis at three basin questions:

First, how does the labor matket experience and behavior of youth

relate to family income' - Obviously the problem is less serious if the

unemployed are from affluent families than if they're from poor families.

Also it's not clear what the relationship between the employment status

and earnings of the head of the household is On the labor Market

experiences and behavior of youth. There's some reason to feel that

young people come into the labor force because of hardship, because

the head of the household is unemployed.

The second question is how much of unemployment iS truly voluntary.

in the sense that ydung workers ,shop around for and frequently leave

jobs? How much is the result of employer decisions or layoffs?

The third queStion is that of the longer term impliXtions.

There's a very sharp correlation of age with both labor force partici-

. pation rates and unemployment' rates. Participation rises with age,

and unemployment falls. If we could be absolutely sure that that were

true for everyone, the problero would really not.be that significant...

ft

- 66 - ")r,

. k,
it would take care of itself. While teenage unemployment rates are

high and unemployment rates for the older groups are smaller, the

labor force participation rates of these older people are larger,

so there are more peQple in the labor force. If you actually

translate that into the numbers of unemployed, you find that there

really are substantial numbers of unemployed, let's say, in the 20

to 24 age group as compared with the 16 to 19 age group. It's entirely

possible that there is a set of young workers that are continuously

unemployed, and if that's so, itLs certainly a much more serious

problem than if unemployment were randomly distributed in the-Vopulation.

MR. ZIMMERMAN: The purposes of our stu'y are, first of all, to assess

various strategies for estimating the value of the output of partiaipants

in youth programs and, secondl, to provide at least preliminary information on some of the estimates of the-value of that output.

We are going to 12 prime sponsors, and we also are going to _one

Department of Interior YACC site and one Department of Agribuliture

r YACC site. We have essentially completed the field work and nbw are

in the beginning stages of data analysis.

We've found that the prime sponsors, like probably a lot of

people'in this room, appear to have very mixed feelings about whether

-or not -this whole endeavor makes sense. Some think it's not very

important at all, especially those who really believe, especially for

youth programs, that we should be concentrating on the longer run

benefits--whether they-be a return to, or continuation of, education;

or whqther they be earning gains. But some think that it's very

A\important, partly because they believe there is a high correlation

between the value of the output that's produced and the efficiency

with which that output is produced and longer run measures of success

of the program, and partly because they believe that it's extremely

important to them politj_cally at the local level to produce real outputs.

I think ours is essentially a data collection study, getting down

to the project level to pick up information about the value of the

output produced. I think I can't go very much further without addressing

the question of, what is valued.

In the most basic sense, the value of the output is usually assessed

in terms of the supply price. That's the price that an alternative

supplier would have charged to do the work.

But the Next qUestion adds further complications. What is the

demand for the output? Is it a service that is useful in the sense

that there is a measurable demand for it? How does one try to assess

the usefulness of the work that's being done?

We want td find out whether or not one can select projects with

--relative ease, whether or not one can get estimates of alternate

supply prices and 'what the problems are, whether one can get at least

information about the demand for the product or services that are being

delivered and then whether one can go further and talk about a quantitative estimate of that. We also are getting into project costs in

1

- 67

4

Ft 0 ^ o;.

1'
0

order to estimate the net value of output.

There are some theoretical issues that I think we will .beable * to clarify a bit. We want to know whether this work is being done.le

efficiently. Yet one problem that we've had is the treatment of

inefficiencies in the regular work force. For instance, in one case

where kids were doing maintenance work in schools, there was never a

question about whether or not they were doing the work efficiently or

whether or not they were putting in the hours. The problem was that

they were doing work that had been done two or three years ago by the

regular work force and that the regular work force was not doing

additional work over and above this. It was not a problem of fiscal

substitution, because the regular work force was still being paid.

It was a matter of having a little bit, more time for coffee breaks and

that kind Of thing. Is that situation an indictment of the youth

program? It. is hard to say that, because we_also found cases where

people had been put into situations that really did free'up other

people to do more skilled work.

I think another problem that has-to be considered is the treatment

of training time. One can dismiss it with a head in the sand strategy

by only going after pure work projects, but that's not easy to do,

because in many cases.there are mixed,projects.

Demand issues are an even more intractable problem. The fundamental

issue, for me, is that youi.e dealing with what essentially is a,aeunterfactual situation. If you want to find out whether there's a demand

for the work, you have to ask people-what would have been done in thd

absence of the project. Some people have a hard time even thinking

in these terms. In several cases in our study we found a situation

of people saying that if the youth project hadn't been there, they

would have used Title VI workers, who are doing something else. There

are a lot of programs out there, and the whole,issue of the way publtc

services are being provided and the sources of payment is important.,

We've found in some cass/where it is indicated that in the absence

of the project, the work\4ould not have been done, that if project

funding ceased, the work would be continued from other sources. The

project itself induced a change in thinking about provisions of some

services. Uftfulness, thus, depends quite radically on the Orspective

and the respondent.

The last issue is whether we can streamline work valuation studies,

perhaps by doing them without Project-by-project analyses or, instead,

with some standardized format.

We have been unable to develop major short-cuts date. There

are possibilities for some kind of valuation mechanism in a reporting

system. There are two examples of that right now: 0 is used by the

Department of Interior and the other is used by th epartment of

Agriculture. It appears that they might have a m del that could bring

in some useful results. The problem is that it's 'nput based

measurement system. They have various wage rates that they,use by

type of work, based 6E\thewas-es of regular employees oing the work,

and they simply multiply the wane rate by the number of hours that

the kids were doing the work.

- 68 - 81
so. npDISCUSSANTS:

MR. ROSE : .,I'd like to first discuss the measurement of'work outpyt

of youth rograms becauseJL think it represents a classic *ample of

.anattemp at ex-post facto analysis. We rushed to get programs'

underway; and thpn a year later we called in a Couple of researchers

and said, "What.baid we do out there? Find out for us aj tell us what

the impact was.',' This is a similar situation to when we first started

looking at the results under the Manpower Development Training Act.

We discovered that you can't u4 operating statistics. You're just

kidding yourself. What you find is that nobody's measuring the same

thing, and they're all using different definitions. You_ought to

set up certain sites that you consider your basic data sites, and ft

is from those sites that you'll really get some hard information as

to what you're really accomplishing.- To go in afterwards and try to

pull out information from operating people who are trying to survive

on a day to day basis, will'not be productive. Xhe system is not

designed to-provide-statistical information-of ally validity because

they're asking this of operators who cannot answer the questions for

ush a

The theoretipal work in the'causes of couth unemployment evoked

-a certain'deja vu. I'm skeptical.that the work w41 help to drald

eflConcluSions that will help in the formulations of policies directed

toward the employment and 'unemployment problems of youth. I don't

quarrel with the,data sources, nor with the methodology, but Most of

the questions you're pursuing we already know the answers for: that

is, we know what the statistical results will be.' The qUestion really

is whether we have enough guts or enoug sense-to translate these into

policy. issues!

For instance, you talk about broken homes. What are we going to

do about broken homes? You're going to prove.again that 'a lot of

these kids come from broken homeS, that lot of these kids are°

illegitimate, that they're from welfare families, that they're associated with crime and theyre Associatgd with drug abuse. "What then?

Our difficulty is making the jump between the information that's

available and the policy. The question is where do you move from

those facts to program and policy and Who has the courage to make that

move.

The*e's very little discussion about institutions and what ley

do to kids. It seems to me that we ought to,think about that rat er

"'than just simply manipulating some secondary data. We must realize

the fact that we have institutions that do a lousy job of what they're

supposedo. do, and unless you bring that in, you're goiig to be

leaving out some important parts of what you're trying to get at.

Simply saying that the institution is no good and then walking

away from it or throwing mor&-Money into it, is.not a sollition.- We

ought to foe a little" bit moreoriginal*about creating new institutions

that pick Up where these.institutions fail: We need to do something

more,about picking up youngsters,,for example, before they become dropouts and having institutions that face the youngster as h'moves out

' -

F

o - 69 -
"'

df 'the school system because the school system, is either warehOUsing.

the people that we're concerned about or pushing. them out.

MR. ADAMS: Having recentlycompleted a. book on youth unemployment, .. I submit that the literaure'teIls us a lotmore about the qUestions

. that are being asked than, has been ackriowledged. In the theoretical .: studieS of auses'and cures,".the objective is to determine the characteristicS of the youth employment probigs in the United States

and the cduses.of the postwa .increase,including demographic changes,.

the problems of the central city:the youth school to work transition

problem, et cetera. The'se explanations have been explO'red in countless

other places, and'I think the knowledge_to.be yielded by further work

is ,limited. The iost important elemelit would be any help ideterminim'g

hoW jobless youth,fingnce theiF consumption and' spend their time while

outside the labor market. Wesimply kribw)very little about the non-'

, 'market activities of.Youth and the' alternative opportunities offered'

to these j,ndividuls and the incentive structure that exists as it

affects labor market aqtiyities.-, . 4

. . ,

. .- . I'm also encouraged bythe plans to massage the data from the

high school class of 1972 bec4use, ;rankly, too little has been_done

sfaith these d&ta. It's.almost been a mystery as to what 'the data have

to say. Certainlythe 'potential is there, and the early findings.

.illustrate thepotential importa4ceof this analysis. My. hope is in

their further analysis they will shed light, on at least furur problems:

One; blackneSs asa source or ckse of thd problem of teenage unemployment; two, the sole of educatio and racial returns to,educatidn; three,

the residUal effects of teenage joblessness; and four, the nonlabor , market pathologies of teenage joblessness in terms of the incentive

structures'. v c.--tz'

. 'e. . . :Turning to the study f the Potential' act of employment and \ e - training programs on youth employme , I have doubts about. its , potential because it focus on empl nt and'unemployment, the . dichotOmous division o'f;lab market activity. In so doing iteflects

g.;qview that these-congepEs asure something important. I'm not Sure

they do. f'tfank the:point/was made that for some Who are unemployed,

r. the hardship may hot be significant. At the same time some df' those

who 'have- jobs, may, in fac,t, have very .significant siVrt and long

terM employment prOblems.. .gonsequently, lookinig at the dichotomous . nature of 4e labor market in terms of employment and unemployment.

' Stet gaily, dc4sn!t produce very ,interesting result,i.

d . 1-, , ', ....

. . :

, In the study,onhe,soCial and econdmic significance pf teenage-

'7uhemploymen , the objective is to measure the extent of teenage

unemploymen , the extent to.whidh tee4ge unemployment impoSes hardship

on ateenag.r's family, 'theextent to which teenage unempLoyment arises

from volunta y-supply dAcisions and thd extent to which. teenage

1 employment p eSaqeS of su sequent.unemployment. 'The Study focuses, on

Shortrt,run labor market de sions, matching March -April CPS data

yit1 more than can .possibly,loe delivered usinT:the CPS... The' determinants pf six 1 bbrmarket_statusxconditions,are being 'explored by the study.

- . But hey are not, nor cgh there.be'any controls for school enrollment

. 6 ...-

,vr' r. .N ' v , I

,

fr..

;

4

- 70

P.4 4
status. That's an extremely. important variable to consider in terms

of the employment or social and.economic -significance of teenage.

unemployment. Most of the work which is to be done with transitional

probability models could also probably be done with tabular analysis.

To me, it's a scalpel being used to dissect an elephant. yihally the

one year work experience that the CPS will provide, in my ,judgment,

A will .provide very, little useful insight into the residual effects 'of

teenage unemployment. We peed to go much beyond that. In measuring

4 the work output of youth programs, the objectives are to provide

preliminary information on the value of output of youth work projects

and to assegb strategies for valuing that output. I do not understand

why. we are going to look for data, after the fact, rather than planning

-ahead forthe data, that is, needed in evaluatioh% ,Quite frankly what

we ought to be looking at is not-the immediate good or service pr&luced,

blit what it dOes'for the individual, what the long term impacts of that

program are. That value of the goods and services, while of interest,

is not the most.important question.

GENERAL DISCUSSION:

MR. TAGGART:. In designing our knowledge development agenda, we have

to make a choice between dollats for research, for evaluation and for

demonstration. What is the best mix? Earlier, we were talking aboUt

action programs delivering benefits to youth. Theoretical work does

net have such an offsetting benefit.

How much theoretical work is.tit worthwhile doing? We must then

subject theoretical work to he same set of policy questions to which

we subjected the demonstration activities; that is, in what time frame

will significant policy results be produced, what hypotheses are there

and on what grounds will they be accepted or rejected? Then, how much

of-basic research Or empirical analysis should we invest in, when should

we invest in Tt, when should we expectresults, and what track should it

be on? These are.the same questions that were asked about the demonstration effortS.

. For instance, it seemed to us that work value should be estimated.

We rehized the difficulties, but "substitution" is no easier to

measure, or 'guess at, than "work valuation." The questions are,

"Are we researching the right issues or the wrong. issues?",and "Are

we investing enough7 do we need to invest more?" The modeling simulations &seirvery cheap process compared to the other things that we

do.' To design these demonstration projects costs more than it does4

to run "everything that's been done up here. What is the relative cost

effectiveness?

MR. WNOW: I'm not sure what to say abdut how much money one should

put in, but maybe there is something-to be said about the 'sequence.

I think if one were to loopack over sa years, at least in social

welfare research, and ask what had an impact, there have been two

kinds of things, One is assessment of .problems. A second is demon-.

strations of possible solutions. You need both types of work.

' MR". TOIKKA: This panel is discussing basic research with the intention

n.

71

1. f 5;

.s

.
4

9

)

of that research having an impact on policy. We must establish at least

two things;, one, that the research illuminates questions that have

not previously been adequately examined; and, two, that that knowledge

would Make a difference in terms of prograTiz.tic choices. I think the

employMent impacts of government programs have not been adequately

explored and are a fundamental issue with obvious policy implications.

The same is true of the study of the social and economic consequences

of teenage employment.

In many cases, researchers do not, for a variety or reasons, make

the necessary Link between their research and the policy options. I think this is_not entirely the fault of the research community. It

grows, i.,4k* opinion, out of a lack of communication, the fact that the

individdals involved have different agendas, different priorities.

It also grows out of the fact that there has been an incredible volume

of research generated in redent years, and difficult even to

summarize that, let alOne to make judgments'as to what(piects of

research are better'than others. And, in many cases, I think that we

rely on the judgment of other researchers to make those assessments.

In some ways this is good. In other ways, however, it is unfortunate,

because I think it leads to a narrowness of focus. One way to expand

focus is to facilitate'more communication between the research community

and the people making policy and.administrative decisions.

V

4%

tl p

, 72 -

. ; \

1:1 O
SESSION V: DATA COLLECTION AND EVAILUATION

PRESENTATIONS: . MR. BRANDWEIN; We've set up the national samle survey tracking

. systemto aid in evaluation of the overall CETA decentralized programs.

Very brietly, we use the census to collect data on a sample of

enrollees, selected,f'rom 147 sponsor samples at the end.of each

quarter, We develop a sample which is interviewed to get more detailed, more reliable data on the characteristics and preprogram

experience of the enrollees, than are possible froM sponsor reports.

We ,then follow up the same period1cally to determine post program

.experience. .Wp a4e developing a comparisoncgroup, using the CPS as

the basis, and will use social security earnings data to compare the

record of this group with that of the CETA participants to estimate

the'net effect of the program.

That-'s all very easy to say. There are substantial conceptual

and technical prbblems with this type of comparison group effort,.

and I'm not optimistic about early er definitive results, so.we will

be trying to develop some alternative ways of estimating net impact

or the difference a program mAkes.

In any event, we've now extended the Continuous Longitudinal

Manpower Survey (CLMS) system with no changes to cover the fiscal

y ar :78 enrollees of:the two YEDPA programs for which funds are

alilocated to all sponsors, YCCIP and YETP. .

, The immediate purposes of this are quite realistic, -quite

/)

achievable, subject to some only modestlogistical'and timing limitations. The first purpose is to document who,in fact is being

served by the tywo YEDPA programs. that information will also enable

us to 0 a general comparison of the characteristics of the yciuth

enrollees in other CETA programs, with thOse in these two special

YEDPA programs.

4 --;j

The secokld purpose is to aet thjearly views of the par'ticipant

on the program. °- That will be useful for -public relations purposes,

if nothing else. . . And the third pgrpose is try to determine from fo ow-up

, ., . , tracking how long enrollees stay i`n the program, whether t'14ere are

significant differences betWeen those w stay in longer and those i

Who leave early; whether they return tq or remain in school,, and

what the 'initial post program Gabor market experiences are in

terms o employment and earnings. 4

. , 0 . . gb

11 . 'A in, it will be possible to document any major pattern of

differ nce bAtweenthese two YEDPA programs, and between them and

other CETA activities with large .youth enrollments. . A

ti There have been three issues in this extension. One has been

whether and to what extent to try/to develop p,psychological profile

of enrollees as well.as the sociodeonomic one, and whether V measure

attitudinal changes after program participation. The Office of Youth

- 73 4
11.00. Program has wanted this. I've been dubious about it and the Census

Bureau pas been, too, in part, because of some skepticism about its

utility; moreso because of concern that the attitudinal questions

would affect responses to othei questions, would adversely affect

response rates, and might have substantial biases, since we anticipate that the interviews often will be conducted in the presence of

a parent, or others who might influence responses.

We've compromised and we will add such questions on atn experimental

beisis for the fiscal year '79 sample on perceptions about work, occupational aspirations, self esteem, other attitudinal 'matters, using

fundamentally, the same ones as in the National Longitudinal Survey

and a number of other youth studies, so there will be some comparabilty..

, We are going to present the questions to half the sample, and therefore, we'll be able to test the efircts of such questions on response

rates. We will also reduce the risk of hurting the whole sample if

the questions do significantly alter response patterns and response

rates-.

4 The second issue is on timing of follow-0p. We started the CLMS

with a follow-up about nine months after entry, which is roughly six

months after the initial interview. 00r purpose there was to maintain

response rates and ease'recollection problems.. But on the negative

sicie, the limited time before the follow-up interview has meant that

many, or most, of the enrollees were still in the pr2gram at the time.

of the interview, so that the'initial post program dgla were for only

the relatively early leavers. We are actively.considering deferring

the first follow -up until 12 to 15 months after enrollment, so as to

get some post program information.on virtually all of the enrollees.

At the moment, we will have one follow-Up for/sure On the ?EDPA

enrollees as well as perhaps a second. We'll judge that in light of

the experience and findings .of the fifst follow-up.

v Finally,'we have the basic issue of how you can,estimate the ' net impact of .yccip and YETP on the econorsic experience of participants.

The CLMS datla we're going to get will give us only gross information on

how the participants fared after they left the program and will not

enable us to determine how much of their experience is att;ibutable

to program participation. Developing a comparison group for youth, I '' think is more troublesome than for adults because.it's not feasible

to. get a reasonable match in terms of prior labor force experience,

because during the unsettled teenage years, there is.considerable

movement in and out of the labor force and in and out of school. We

h4ve the further,diffiduity created by the scale .of the program. As4

. the programi have been expanded over a period of,several years, they

4o in fact, reach a considerable proportion of the low incoele population

to w,nom they are.targeted, and,-so, some ippreciable portiOn of your4

comparison group` will at some 'Point enter a program rithgr than remafn

n4 the untreated category.

. 6 The size of the programs also meads that the'labor markets are

being effected. If you-march in to Baltimore with a-saturation program,

7

04 4

_ 04
,it's fair to.Say that the kids who aren't inthe program, 'in school

or out of school, are functioning in a less competitive larbor market

at the time the programs have enrolled substantial numbers of people

and can't really be regarded as unaffected by the programs.

Be that as it may, we will undertake some comparison with several

types of reference groups. We'll see what we can do by referring to

the Job Corps comparison group, the supported work comparison group,

and the CPS youth group. We may have some considerable questions

about matching, but if the cumulative evidence from, a variety of groups

adds up fairly clearly,then I think we have a more solid basis for

any estimates of impact.

MR. BORUS: The National Longitudinal SurVeys have been going on since

1966. What we are doing under the Office of Youth Prigram funding is

1,, starting a new cohort. This will be 12,000 young menTand women, equally

(' divided by sex, including a heavy oversempling of blacks, Hispanics and

non-black and non-Hispanic poor. They will be interviewed annually for

the next six years, beginning either'the last day of January or the first

day of February, 1979.

We are going to'have a number of background variables, since, in

the first year, we aresgoing to try to get detailed informatibn on what

happened to4the young people, 14 to 21, up until the time of interview.

So we're getting family information, schooling informationf-details

on past training, military service, health and a little bit about

4). previous, work experience.

\ We're alsoincluding a number of socio-psychological measures in

order to get some fix on what ormhich of these measures may be good

predictors of future behavibr. We are including the standard internal/

external locus of control, a knowledge of he world of work scale, a

new thihg that we just invented,-which is similar to a.demand curve

that we use to determine willingness to°work, the reactions of "significant others" to 'various types of behavior, attitudes toward the role

of women in the labor market, and questions on aspirations and expectations,

Finally, in this first year we will have what happened to the

youth during 1978. There will be detailed work history, which includes

all of their major jobs for the years, including more detail on their

present, or survey week, job and its Characteristcs,including such-c

things as their satisfaction with the jobsome of the fringe benefits, and the quality of th work. We will get information on their

and their fathily's income f 1978. We havea section on participation in various types of tra ning programs, particularly DOL programs;

and in these,JPrograms, wi 1 seek to fin out what services they

received and what their reactions were to he pr We will get

information on family status, on educational expereinces and on military service duig the year:

4.*IW ,./

We had'a lotAore that we were ho ing to ask in this first inter-

. view, but our pretest ran almost twice a's long as our OMB ..4pproval

4would allow, so that we have cut back o r survey instrument:' However,

we'have a number of questions for succeeding years.

- 75

80
0.0

One feature that we are tIstinci for the second year that might

be of particular interest to p ople is a tithe-use diary to find out

how many young pe?ple, particularly those who are out of school, are

spending their time. We have also been asked by the Department of

Defense whethe we would consider adding a special additional group

to the sample- young people in the same age grtlips who are now serving

in the milita , si4ce they are interested in 'nding out what -happens

to veterans when compared with youth who da not go into the service.

We are beginning to talk to DOD about adding approximately 1100 more

individuals to the sample.

MR. WURZBURG: The National Council on Employment.Policy is involved

in doing a process evaluation of the implementation of YCCIP and YETP.

Most of the questions that we are addressing in the, evaluation

fall within the ten principles that were outlined in the .youth charter.

We're, looking at knOwledge development. We're looking at quality work

experience, aqg what sponsors were doing to improve the quality of

work experience over pas,t programs, what improves supervisiop and

what tangible outputs result. We're looking at youth 'participation;

what'roles the youths are taking other than as participants and what

forms of participations are being used, other than the forms that

were mandated by the Labor Department. We're looking at targeting

to see how prime sponsors are using their discretion for participant

selection and what the,practical implications of the different eligibility standards-4e. We're looking at, the kinds of fiscaripubstitution

that are occurr9ng with YCCIP and YETP and we're also looking for

different kinds of substitutiontther than fiscal substitution, that

is, other things that may be diminishing the impgct of the youth program.

With respect to institutional change, we're trying to determine what

effect YETP and YCCIP are having at the local level, on the relationships between prime sponsors, the schools, unions,' and private sector

employers. We're also trying to figure out some of the implic'ations

of these changes. We're looking at the delivery agents, how they

were chosen, and how they can be differentiated._ Then we're looking

more generally a some of the barriers.to imprementing the,yduth policy

at the local level--the systemic barriers, as well as some`cf'the local

4istorical factors that have an effect in shaping the fimpadt of the

youth programs.

For the project desigh, we decided to take the oase study approach

for a number of reasons. In spite*of what a lot 'of priple sponsors say,

YETP is still essentially in a deGentralized; decategorized program,

with, a lot of variation. The program is nesok and issues are oonstantly

emerging, changing, and evolving. 4We feel that this is the time for

observation rathsr than any kind of hypothesis formulation and testing,.

The case studies are covering 37 prime sponsors. We have ten.

field associates, each one covering three to five prime sponsors.

They're preparing three sets of interim reports as well as a final

report. The interim reportse, along with an overview goes to the

Labor Department. So farrwe've completed two reports, one in'Febivary,

and one in August. The third report is going to be submitted around

the first of the year and :lathe final ,report is due next _Spring. The

final be the'wrapup covering implerr ntation through early

76 -

8v . s
winter.

We've encountered some difficulties. Inevitably in a case study

approach there is tendency to look for generalizations and sometimes

it's difficult to separate where the 'generalizations are,important and

where the variations are important. Some of the field associates

are also running in to a little bid of difficulty getting data. After

the first two reports, ome ofthe prime sponsors are getting a little

bit more protective of their turf, and they'ie not happy with some of

the findings.

In the first two reports, we have _come up with what I think are

some fairly interesting findings. Some of them may be useful in

changing the programs as they are operated xight now; others will be

useful in addressing reauthorization issues.

The planning process, which t.00k place last fall for fiscal 1978

programs, for the most part lagged behind schedule. It was rushed,

corkfused and marked by a very disturbing4ind of dualism.

There was, on one hand, an exercise where prime sponsors collected'

numbers for th*e.grant application package which was submitted to the

Labor Department, lachille the real decisionmaking was going on in a - separate arena. The decision making process was not in any way reflected,in the grant application package. The numbers used for the grantapplicktion package were frequently seen as being inaccurate and unreliable while other variables were considered much more impoftant in . determining prograM mix and target populations. The grantapplication

also required standardization which may have been useful to the

Dmartment of.Labore but was not particularly useful to the primesponsors. We are recommending that the grantapplication package be

t. reviewed and modified to get the information the Labor Department needs

,without making any pretenses of being useful for prime sponsors in

their planning process. . - 4 With respect to knowledge development, we found that just about

all the prime sponsors. were trying something. Most of it appears to

-be things that they would not have been tryint otherwise, but, the

results have bdkm4: n mixed in the plans and the progress that we've-seen

so far. While so have tried very structured experiments, others

are sticking Wl simple variations in their management process.

This year we are expecting that there'elboing to be limited payoff

from knowledge development. In the first year, prime sponsors had very

limited capacity to give knowledge development enough attention. They

didn't have tHe'time or the expertise, and Allquenfly there was competition for the dollar's-that might go to knowl6dgedevelopment. A lot

pf local pressure was put on primp spojnors to.keeri,the money in job

klott and in training.

I ..

. . / There is still a great deal of confudlon overVhertknowledge

dekelopment is supposed to be at the local level and how ,prime spohsors , are supposed to be using it:p SoMe are trying veiT ssophisticated projects,

but, in attempting to. do a.lot, some of them may wind up accomplishing

, v

1 7 77

I (

,

ss,

S 0
very little. We've recommended that the Department of Labor provide

more technical assistance to both the egionaL offices arid the prime

sponsors. The "regional offices have been found to be a grWt source

of misinformation about knowledge development.

We found that prime sponsors are putting an emphasis on work

quality. A lot welcome the emphasis on work quality, and they are

doing things differently. They are organizing ant improving supervision. They are paying much more attention to the quality. of the

jobs; however, not too much attention is being paid to creating jobs

that are 'fully relevant' to the interests of the kid. Prime sponsors

found it to be impractical. They found that a let of,the kids are

too concerned about where their careers are goin., Instead prime

sponsors have put a lot of attention in to more,classroom oriented

career exploration programs. We've recommended that career exploration

activity and the idea of getting enrollees familiar with different

occupations be done in a classroom setting, rather than by on-the-job

experience. It's cheaper and it's mote efficient. Jobs should put

strong emphasis on simple bAics--teaching a kid to show up to work

on time and teaching him good; basic work habits--the kind of thing

that can be transferred anywhere. --__ r

Youth participation has)ap.een very limited. The,youth councils

have been ineff ctive channels for youth participation for a number

of reasons. Ki. S have not been partidipating very actively on the

councils and,. the councils themselves are frequently ineffective forums

foe% any kiria of decision making. It's encouraging to find that the

prime sponsors have tried a lot of alternative techniques for getting

kids involved in ways other than just being participants. They've . iIVolved them in Some administrative aspects, they've involved them

in doing some evaluation, and they've involved them in peer counseling.

This has been a much more effective route and we've recommended that

the Department of Labor encourage alternative forms of participation

without mandating any particular approach: ,

. Most prime onsors we've looked athave.concentrated6their

services on kids wh are the most economically disadvanted. Very

few prime sponsors., for example, are taking advantage of the 1Q%

option under YETP to experiment with mixed income groups, partly

because they have had difficulty with regional offices in getting v research plaris approved, but also largely because of political

pressures to confine the prbgrams to the most disadvantaged kids.

We did find' that females are grossly under-represented. We're not

.y. Sure whether this is due to discrimination in selection or due more

to the applicants for the YCCIP jobs. Because most of them areplabor

Antensive, and a lot are in cone truction jobs, perhaps_girls just

°°7 haven't applied for them. . , ( -There's also some confusion abOut labor force status. We're,

not quite sure who some of thege pWicipantstre ih°terms of whether.-

they were unemployed or whether thej, came from outside of the labor

force. From what.We've seen, most of them ar,e coming from outside

-the labor'fdice, althbugh the definitions are confusing. The implication is of course, that anyone looking for the effect'Of youth programs

- 7R 9

L
in terms of reduced unemployment are not going to see an effect.

They will see instead increased labor force participation.

One of the concerns of the national office,wadorthat expenditures

for youth under Title I programs would be reduced. In fct, we found

very little of that. Somebody really put the fear of"God. in prime

sponsors on that particular point, so there is very little fiscal

substitution of YEDPA funds for Title One. We did find, however,

other kinds_6f substitutions over which the Labor Department had

little control. Community block grant r49911-ey, for example, was being

pulled out of some areas, and YEDPA money was being,substituted.

We found that the schools, in some cases, were substituting money

to provide services that they would have or should, have, been providing

otherWise.

We found that community based, organizations were extensively

involved. We found it rather difficult to diffetentiate in any

systematic way among CBO's or between CBO's and other services

delivers, with respect to cost or tatget'populations. We did find

that the CBO's had the effect of broadening the base of"community

involvement and the constituency of the youth programs.

As far'as institutional, change goes, we think the CETA local

educational agency links have born some fruit. In many cases, however,

the CETA prime spgpsors and the LEA's are not talking directly, and

there's a lot of room for improvement. What we think is that the

Labor Department ought to be looking for some alternative routes for

encouraging LEA participation, and for getting the educational

community more involved.

MR. LACEY: After selecting about fifty youth work sites around the

cointry to look at, interviewing around three hundred youths, and

conducting about two waves of site vis s, we came up with some

hunches about wpat was.occurri out in the field. But'before turning

to those hunches r would like t give y u a brief overview of tha----

research methodology MDC,used in our process evaluation.

We were required by the Office of Youth Programs',to provide some

specific kinds of data to them on supervision, attendance, disciplipe,

and work activities. tae also were to provide some comparisons supported by data and observations on the traditional youth programs

under Title One as compared to YETP and YCCIP. The thing that wR

were attempting to arrive at here was whethOT YEDPA really mad a differ nce. We were also to provide assessments on he kind of work

that wa being done from the standpoint of the commu ity, the e ployer )

and th youth.

. We interviewed youth, went to work sites, and interviewed ptime

sponsors. We developed a,research'instrument which gathered'information from the prime sponsor,about the kinds of activity that were

',occurring, including enrichments, supported services, Ilionoring,

and supervision. From that/Point, we went to Work sibesadd asked

some fairly specific questions about activities. The thing.that

I've heard repeatedly here, and that has probably been echoed for

the last 10. years, is the issue of illeaninghil' work. We attempted

( - 79 -
to look at different kinds of work activity,..and really find out

what is going on, if there are any tangible.olgszoductg, if youth

are really learning something; if their experiences are good. We

were also interested in looking at these activities to determine the

wages that youth were being paid, the hours, and the like. Lastly,

we were,also to identify some exemplary efforts around the country,

hoping that these particular efforts could be replicated by prime

sponsors. We saw a loe of kids involved in work that would not have

been possible had not it been for YEDPA. Whether the work was meaningful or not involves a lot of subjectivity. We saw kids involved that

would never have had a job had it not been for YEDPA. We saw good

programs and bad programs--and in many instances, the good programs

and the bad programs were within the same prime sponsorships. We . collected data on characteristics, on supervision in terms of ratios,

and we walked awey .th a thought about that data: numbers are neat,

expressive and pu , but their simplicity an be deceptive. We

are very conscio sot, the limi -tions of the tabular and statistical

data that we gat

Also we looked at what 'we called good work sites, or identified

some good work sites. We concluded that, first, work sites were good

where the youth, as well as the supervisors perceived the work or'

activity as meaningful. Second, where all agreed that some learning

has taken place; third, where wages and working hours and conditions

were agreeable to all; fourth, where supervision was a motivating,

rather than a demotivating factor, and fifth, where the youth perceived

some future use for their experience.

These are things that constituted, in our minds, a g od work site.

While looking at these particular kinds'of things we also examined

whether there were conscio attqmpts by the prime sponsors to somehow

insure that good supervis o would exist in terms of weeding out bad

work sites and helping su !i.sbrs,trelate to the youths. We wanted

to know if the prime sponsor provided any kinds of training to supervisors. In most instances we found that theydid not.

Also, we saw a fot of CBO's, involv0 in YEDPA, more involvement

than in Title I. We saw OIC's, we salAybrban Leagues;, apparently,

were able to get a piece of the action.

Perhaps the most startling thing was the lackpf supportive,services..' .

, Youth across the country, from Savannah to Pasadena, are Saying that

° there are so many disinc n iv to participating'in tie program that

you find them terminatin b ause of costs, the need for transportation,

Cf.Jorichild care. The prime sponsors do not.provide these thin9s..f

You also find that public assistance payments,are sometimes being

reduced because they participate the program, causing some to drop

out of the program.

The researchers are now out in the fie for a, third wave of visits. We're going back to the good sites, and tearing them apart, looking

at supervision to determine why it's good. We are talkAg to the prime

sponso to see if it has made some kind of conscious attempt to provide

good s pervision. Hopefully, we will come out with some pointers on

-how O build a successful .work site.

_T 80
* MR. SEILER: ,Among the lauhdry list of priorities of the Office of

Youth Program is to have a-network of reviews of 'the processes of

programaimple entation that reflect on CETA pr ) me sponsors. Now the

programs I'm talking about are not the specia 1 demonstrations, but

the ones that arefinanced-by prime'sPonsors with the formula allocated

funds--that is the YCCIP, YETP,' and SPEDY., the Summer' Program for

Economically DisadVantaged Youth.

Mr. Wurzburg and Mr-. Lacey haye reported on their efforts- at

studying YCCIP and YETP, but we've-also undertaken some other contracted

studies. There are several special efforts to have our own.staff go on

site to review sponsor prog&M"planning and operations.

. ,Jy's

. . Undet YETP at least *2% oftheo.fundsimust be spent on in-school . programs where spohsors,- have

- come,toterms in the form of a written

agreement with local education:agencies:about how that program is

'going to be conducted. That peresumab y is a mechanism for trying to

i

get sponsors and education agencies t better work together, to mesh

.what they are doing.. In the initial' mplementation of the program,

it wasn'.t expected that they wOdid have very sophisticated agreements,

and they were given lots Of extra time to,develop these agreements.

We have had a staff effort to look into those agreements and analyze

them for their strengths and weaknesses a-5,-e means of giving-some

guidance in strengthening'the agreements. We have'also had a joint

DOL-HEW team examine CETA- -LEA interactions in the field. We also

. have a staff effort looking at. urlion involvement in the youth programs,

and there will be'rePorts on, theMtKo things that will be coming soon.

. "We'Ve also launched'a wide ranging effort to get a-broad view of

what the SPEDY program is all about. First, we've had an analysis

done on a national sample of the SPED plans which were funded. We're

trying to use those plans to draw a picture of what theprogram was

supposed to look like, based on what those plans said. Another purpose

of that plan review is to look at"the accluracy of the grant applicatiOn itself as a structure fc, or as 4!planning tool for summer

programs. )

IY, ,to -f

4 i ,'4 Our own staff has studied a sample of primv.sponsors by interviewing them early in June to get a piCture_of the processes that

were used to develop that written plan, and to find out' what were

some of the underlying basep-for their decisions that' were reflected

in the plan. We then returned to the,same sponsors in July'andlAugust

to study how the program was being delivered; to see it in action,,,

.to look at the relationship to the original plan, and to identify

the reactions-and perceptions of youth about the program. As'an

additional effort to learn about SPEDY operations, ManpoWer Development

Corporation is looking at the character oft4work being- performed by

youth in the summer program. In addition, to get a better understandihg

of the SPEDY program, an organization called the National qhild-Labor

Committee has done a review for us Of SPEDY implementation to identify

the kinds of dynamics and the factors in the big city situation. In

addition to that, as part of the National Council on Employment.Policy'

study, there are participant observers in about 40 its apd we will 144 , be getting some feedback on thq summer program from that study. We

are also'stpdying the Vocational Exploration Program, whieh,is.admin- 't istered through the: ,National Alliance of BusinessMen and t,he'AFL/CIO

Q

I
Human Resources Development Institute, as 15art of SPEDY. It operated

in about 66 jurisdictions this summer, 'although there is some evidence

that priMe sponsors are not implementirfg VEP in the way that was planned.

We are analyzing how.the VEP program was planned and implemented

in two ways. We have some staff on NAB and HRDI taking their look

and we have an independent study being made by the St. Louis Univeristy

Center for Urban Programs. In addition; we developed a special instrument for attempting to measure Some behavioral changes that might be

expected out of the program; work knowledge, job search knowledge; and

attitudes and aspirations on the part of the youngsters. ,We.dw.,,eloped

a special instrument which was administered1bn a pre-post program

basis within VEP to try to assess behavioral change. We are also

trying to do a special analysis to zee if there are any ,particular, , program organizational factors that tend tb be related to any behavioral.

changes we might find. .Also,.we've had the same instrument administered . within SPEDY programs, and we're going to attempt to do some comparisons

of changer in summer programs, and the types of programs that sponsors

put under)SPEDY as compared with VEP. Another effort we had was a

national search for some SPEDY program models, and utilizing..some of. . the perceptions of our regional offices, and othej)S, and there are

five monographs being prepared onthose programs.

All that's being done on SPEDY is in readiness for'our launching'

of a series of regional conferences around the country on- SPEDY. These

will try to get.5cime early feedback to prime sponsors about what has ',,. happened in-the program, and to give the prime sponsors themselves an`` opportunity to' share that knowledge while it is fresh; in their.ininds,

Hopefully, all of this effort will help,us improve the summer program

regulations, the grant applications; procedures, the-technical,as5istance ptogram, and.ultimateAy, the SPEDY design and design delivery.

MR. BRANDWEIN: 'ice the YEDPArprogiamsfocused On by most of this' _%i_

conference, th- Job Cot 5 survived some 14 years of ups and downs.'

in public att,tudes towa yestment.in clisadvantaged youth. 'Btit . ,it's.hung- and evolved. into a relatively stable program with a

good not': of what it's doing. Two,years acorwe decided to fpcuSTon

jCfor t e p-rincipar impact evaluation effort in the yd'Uth field-,

because, vt was' a prcSgr.Am, that riadOelleld becauseti had,a

sizab . inveStkeht or.%participantls aad you could look for sbme . me 'red, results;. cause litt3se was known about the magnitude or .,

timing *of its'ebonom,164e7fet'on participants; and jzoecause we had ' thought 17pAhad learned` more 'Wait the evaluation art than when the'

-f Jog CorWWas aNgkihh,Vt of solicontrOVdrs'ial evaluations in. h, 14'60'cs: .. ,, Ap i . , i 1" ,,, Ihe evaluation foCtlegteon f,i.vetype,A,,Of questerys:' F4g.st, doA. °A

the.lia;b Corps partic,ipantS'd610ettdT in.t4e;:laboriMailkdtthan4comphrk,,

able non-pA-tiC,ipa,nts1 Second, bow dci they compare 116 tyi.Rnbn,.;

'participants tin some othej glivelant"par,ameilv such as'riet*vto,10- , school,,entry into milistary serA,oe,'"entry-ip 'other.ppPgrari ' :;/- u. :, 'A-. * degree of reli rice on:welfare-paymdilts, alna*dOtee d eantiebaial % , . behavior, par icurarly*rest records and drugab4S -"Third.,''were

there marked d'ffereInces in effects by type-oepacipant,duration 0: particip.ition.#'the pros 'am,' and by typ4' of. ,fob ,Corps center?

' .

. \A,

.t- 82 - , (41.4 -0 "\ ,

.

r
d

. , Fourth, how di the Job Corps participantd rate the program and what

did they see a its principle strengths and weaknesg,es? Fifth, did,

program' benefit to participants and society outweigh the program

. costs? :

0 Thq comparison, group was drawn from school dropoue'andeftbloyment,

- service applicant ligts in geographic areasywhiot had il'eratively low

. Job Corps pafticipation, selected because they were not adjoining

Job Corps centers, and yet'fn other respects-were tfmiIar to the.

'V areas from which most. Job Corp enrollees came.

. , ,. ,

- ...i- k . '. i . .. ,.. , . ".,Nv In the Spring of L977, base.line interviews-weroa,Conducted'at

the Job Corps centers with a sizable sampld'of'Obb Corps enrollees-4-4'p: ,. about a third ,of the enrollment at that-time.. 'The. sample was made

.

, . purposely large so that we could ,tr4kover a much longer period than

the initial follow-Up, 'and pat still have -sufficlentrY Mange sample °

, to draw some realistic conclusions. 'c'About nine%months,later, those

who had terminated and_had been out at least,ftve months Were,tracked

and interviewed%againAo obtain information on their post program %,

experience. That follow -up sample was'2400. There was a-comparison'

group odge line'gample of 1500 with a,follow-up.of 1300.; ',.:.

.. . "'' '4- n Ancf eaqx report on the base,lihe nterview diat6,0a:s pub4shed.

Tt describes thenature of the Job Corp program, documents the 4.,

. - disadvantaged nature of those. it' enroll , reviews ,why they saidthe , , , chose to enroll and presentt'.their satin of the program. Vie

follow-up study is not in fingl roil but so eiiimport'hnt.4ndin ' t are already evident. . 7 t2, . ,. , ".',. ''' 0 , ki q '..; f., . e' On pe aVerage, the' participants did gain..m9 e'in empl,Oyment..': i

earningsthAn_thecompariscrklup. The,progr'aM.c9mpletdr.g,d1AArik..

wel,li3OtweA to tlie comparisn'grOUR; ;,. , -. ,' r. Q, .. ,

.

, 4

i ...-- .., w ..., .A ' . ''' 4

.0 , . 0 ,I! 4i, ), -Fttpres an interesting .timetdim 042,n,,,hep.-4n:tne ilitian Tropths - 4 the p4tic loants Jpg0dbekhind the qompari*a.n .'' '

groIs. . 11162co"a ton group had.tizable,,,ntpkWr§..who,,,Are emRloyedat

w 77the %ta riod,:while,i?rtQok tme, for- heiCorpsmembers,",;

"to -r,eseftleand ge ,-into. After the first t o month's', the%pogitiv%

,economWimpacts began,fo predominate. Using the?week-bef,$)re thC.

';,, V4abw-up%survly as-the Comparison point, we found that .ail of the

kdIticipalts had gained $12 More illyaverage Weekly .earnings tan the

,,(16Mpirison group had, for the:gaMe.p4.iod..,,For,:the completers, it

was o$23 more. On other measures. of Impact, too, the-completers-ha

.4,:positil.fe, 'Large, and 'statistically significant benefits:,; More were

in ,the.rabor force, more'were employed, they averaged More hours at

! work, mare were in mi,litary service,,more.had picked up a degree in

thejbb Corps or by,gpihg bajo, to.schdol; more had gone on to college;

they had less welfare dependency;%and tiley had fewer,-aire'tte and sles8

. . 0

',,- abuse of drugs. -.2. -,'-.. ,' . .:

, For those epho part:i:all:completed the prograM, such net impa'ots,

were small. In. act,.tbe'dropioiats 3,agged'somewhat behind,the comparison A A group. -

-,. 0 ; .0..

,

- .
0'

.

. No sharp differences emerged at this point beten ypes or,

f..- sizes of centers. Seven Months'after leaving Job Corps, 77% of

Corpsmembers rated it, as pretty good or okay, as against 6 % who

1

said so while they were in the program. In part, however, this is

a courtesy response. Over 90% said the training was good, or at

least okay and fewer than 10% had some criticism of the training

aspects. When it cameo food, halfofthem said it was bad and

there was some criticism of the relatively small pay allowances.

Also, 60% said that after they got out they could haves used more

help 'in'finding a job and didn't get much of it.

Now, the final part of this study sought to relate benefits to

cost, and the report does develop some comprehensive estimates of

the dollar value of various measurable benefits to participants and

to the society as >a whole. At this stage, I think that part:of the

report really offers more of a framework for analysis, an explanation

of the types of estimating that have to be done. The end estimate8

do depend significantly on variousassumptions about the continuation

of rates of gain in subsequent years, ab9ut appropriate rates of

discounting future dollars, and about the reliability of data. on

societal savings from reductions in criminal activity. But the

initial estimate is that the dollar benefits to society for each

Job Carps enrollee based just on the first 7 months' post-program

experience, are'larger than the economic costs of the program, though

not by much. ./

- The key limitation of the study, a necessary o e at this stlge,

is that its findings are based`just on the first s ven months, so

it's by no mean's clear whether the participants' positive gains in

the early period are maintained, deteriorate, Which is what has been

built in to the benefit cost assumptions or whether they increase

further, which has been the pattern in the 2nd through the 8th month.

So welwill now carry forward this evaluation.by following up on

the participant and comparison group a year after the first follow-up,

by which time we'll have'an average of 19 months out of the program

from that sample. We'xe also going"to track the base line interviewees

who had not yet left the program by the time of the first follow-up, or

had been' out fewer,than five months. So we'll have both an enlarged

sample of 'terminees and some check on.whether findings hold'up for

participants who left the program at.a later point in time. About

this time next year, in time for the 1980 Congress, we should have

readings on this somewhat longer term follow --up.

I close by saying that without fussing about whether these

findings are the definitive findings,for all time, we have established that it is feasible' to do an evaluation, to do it on scale, and

to provide sound, meaningful evidence which hasn't been present before.

DISCUSSANT:

, MR. SUM: 'In the last year,'I've been a position of working with

a godd Many prime sponsors in Massachusetts in developing YEDPA plans,

r both by providing labor.market information and other information to

help identify target groups, in drafting sections of plans, in ',

reviewing plans 'in a number of prime sponsor planning councils, and

- 84 - '
.in monitoring some of our programs for the Governor's-Youth Employment

Training Plan in Massachusetts.

First, let me comment on the finiding that Pthnning in the first

year was "rushed, confused and characterized by a degree of dualism."

If any of you had been involted'in pflanninig at the local level, what

was said about YEDPA also held true for Title I, Title II, Title VI

and STIR last year.. Anyone involved in planning at the local level

with Title VI funds, which will be withdrawn, if you don't begin to

,spend there quickly, is not going to place higher priority on spending

a lot of time with your planners and your administrative, staff/in

developing YEDPA grants. So I'm not surprised that basically we

find that the process was rushed and confused. Most CETA planning

last year was rushed and confused, and I believe that YEDPA was given

a Telatively low priority by many--given the importance of pAting

Title VI money out in the field.

But with the plannins process, there were two problems that need

to be addressed and revieiaed over the next year to see if we've made,

gains. One is'problem identification. Problem identification is

generally weak in grant applications of prime sponsors. Previously,

we were talking about using CPS data to identify youth unemployment

problems. But how would you like to use data base programs for

Berkshire Minty, Massachusetts, when the number of matdhes you've

got is equal to about 4 peisons? A basic part of our Problem is the

limitation of the localized data base. We don't really have a goody

fix on local youth emploirent and unemployment problems. The data

base that we've got is v y fragmented;' it has to be pulled.from a

great many sources; and it's not well understood b many prime sponsor

administrators yet. '%

The problem ,with problem identification is that it's not clear

*who we should have been targeting our service on during the first

year. I think we made a number of mistakes. In the first year, it

seems in Massachusetts at least, that there was an overemphasis upon

out-of-school, unemployed youth. Basically what we found is that we

had problems filling slots because we just exaggerated the number of

unemployed out7of-school youth'we felt would be out there.

Wh.atI think isa worthwhile issue to examine this Year is the.

extent to which our experience in the first year led us to shift

target groups during the second year. Are we focusing on certain,

different segments of the youth population? Part of that learning

experience may well be learnir4 who your target groups really are.

The objectives of youth programs are a second issue. Part of

the problem with many of your youth programs is that theyq. were

poorly identified objectives in the first year of plans. The objectives in most plans were more of a statement of_intent to deliver

services, rather than to produce various type of outcomes. Most

YEDPA plans don't state objectives in terms of educational objectives,

high school retention, job placement's, etc. The problems with a

failure to identify objectivesin terms of outcomes is that it

- 85 9',

L

1

z
hampers the ability of prime sponsors to undertake any type of

. systematic evaluation. If you're not clear what your objectives

are,, how 4re you going to evaluate yourself? So knowledge development at,the local level is only going to take place if we begin to

have a better understanding of the problems we're trying to address,

and if we are more explicit within our plans about what our objectives truly are.

S

I see progress in these areas this year. More of the plans

seem to be stating explicitly what the objectives of'the YEDPA

tie system are. I think that ought to be monitored fairly well to see

if the prime sponsors are beginning to state objectivesin a more

explicit fashion.

# There.is a relatively low emphasis on the preparation of youth

for career jobs by prime sponsors, and particularly low emphasis on

on-the-job training and vocationally oriented institutional training.

The priMary emphasis is instead on basic work experience, remedial

education, counseling, career guidance and job search techniques.

The question, though, is why? Why does such a low share of our

YEDPA funds get devoted to those activities? The explanations are

quite diverse and are'at times confusing. For example, it's frequently

stated that "youth are simply not ready for-suth decisions. Well,'

basically, the young age of participants is the thing that is influencing the services mix, and the question would be--do we find prime

sponsors serving different age groups having different mixes of services?

Do, those focusing more on the older, out-of-school groups tend to

have larger share Of their funds devoted to OJT and institutional

training? If not,--then how do we explain it?

I would, like to know to ,what extent was the services mix for

youth influenced by the services mix under Title I? Do those prime

sponsors that run the larger share of Title I programs involving

institutional skill training and OJT also run the larger share of . their YEDPA programs in the same types of activities? I'd like to

know, then, whether that's a learning experience? Do prime sponsors

that have good Tifle I programs transfer that experience through YEDPA

programs?

There'Sla third issue around the question of how much of the low

amount of training provided:to youth is due not simply to youth attitudes., youth preparation, readiness for jobs and job training;. but

father to *employer reluctance to.hire youth in OJT programs or to

hire them out of institutional training? Too often it is assumed

that the problems are simply with youth themselves; when youth change,

the unemployment problem will go away. What seems to me more often

the case is that employer's attitudes and stereotypes have a lot to

do with this.

FinallyN}n regard to institutional' training, prime sponsors

are reluctant at timet to put money into that, because your outcomes

have to be stated in terms of unsubsidized placements. 'There's an

unwillingness to devote money to, activities from which people are

going to expect placements at the end of the pipeline. The way to

be safe in gUaranteeing,yoursel,f against condemnation by the regional

86 -
office is to put your money into work experience programs, because

nobody really has expected much in terms of unsubsidized placements

out,of those activities. If we want to change it, we.might have to

have the regional offiFes take (a much more sophisticated look at

.expected placement rates out of those programs so that prime sponsors

'wlio are willing to take the risk of putting more of their funds into

training youth are n9t punished

Youth Planning Councils, in general have not'played-a major role

ii the YEDPA plans. For example, in the Balance of State, it's hard

to get two people on a youth subCommittee to sit:down land vote on an

issue because all of them have conflicts of interest. Ohe of the

problems is that, Youth Planning Councils are dominated by the people

running the programs.) The problem is not just getting youth to

participate on councils: the question is how do you get councils to

be active and play a major role in providing inputs into the system?

The question of private sector involvement and how it can be

achieved remains unanswered. How active a role have employers played

in the planning and design of the YEDPA programs? If"you want to

get the private sector involved in YEDPA programs, then you'd better

rip institutional OJT programs. Private sector representatives are

not going'to sit down and talk about the greater glory of work experience in the publid sector. For the most part, they're not interested.

But if you want to get private sector involvement, you've got to ,get

.private sector involvement in the hiring and training of those youth.

I would like to see YEDPA programs try to develop over the next year

' tie-ins with the STIP programs. The kinds of employers we're aearing

olOrselves to under STIP, the kinds of jobs we're gearing ourselves

t4;;,, the kinds of wages that we're gearing ourselves to are 'the bdst

in the employment training system this country has ever had. Is it

possibleifor youth to participate in YEDPA, go through that screening

device, learn those basic attitudes, get that credential to be rated

as stable,' and then be sold to employers in the STIP program: I think

tie-ins of that type are highly desirable; I'd like to see more 0,4 it

in the future.

I will not say much about community based 'organizations,

except that in Massachusetts, a numbor of primes are drawing youth

programs in house, because they are concerned.abouf _the fraud and

abuse provisions in,the regulations.

The experiences on LEA agreements appear to be quite mixed:

For the most part, they're largely non-financial in nature in my

area. But I think even getting non-financial agreements signed.is

evidence of progress in this area. Getting the schools,to understand

what their roles should be and ,trying to gear their activities more

towards students who are not college bound is a success. The question

that I' would ask about LEA agreements is to what extent are we monitoring them to guarantee that the services that were to be provided

are, in fact, being providdd? I would like to see more involvement

on the part of YEDPA with cooperative vocational programs and work

study programs in high school. Cooperation with those programs

has a number of advantages. They tend to have good contacts with

- 87 1

ei
, a t Th e , r , 1 ,, employe 'l . They have worked out the probleMs'about. work schedulAg,

, work arrangements, hour laws, a d the like. Academic credit is ': - given for a good deal of that experience..

Op the substitutidh issue I'm going to be a heretic here this

morning. I am not as qoncerned about the substitution issue as other''

people are. It doesn't appear to be a problem. What I would like to

have .known is whether itivaries by prime sponsor. This year a number

of prime sponsors are going' to ask to be given 4emptions in the

maintenance of effort, and I think some 6f the eSceinptions for tie

maintenance of effort make tense, particularly in those prime spontqr

areas where over the last four years we never built up an institutional

or OJT component under%Title .1-2.Whenme!re.ppending_65% of our money

on, youth under the age of 24, one has to recognize that.it violates

the equity provisions in terms ofj delivery of services to ,adult

economically disadvantaged. Whel prime sponsors can justify, he fact

that they have in the past been perving a substantially large'nuMber

of youth, they should be gilien some flexibility. We ought to view

it n. a case by case basis at,the local level.

The last issue is local evaluation. I believe that the reason

why much local evaluation is not being done is simply that little

substeefrtive local evaluation has been done for Title I prograMs

where there are more explicit and clear objectives, such as the

placement of people from on-the-job training and skill training into

unsubsidized employment. Since we have generally not required , 4 detailed systematic follow-up studies of our participants knTitle I,

how do we expect detailed follow-up capabilities to be developed in

programs whgpe objectives are far more complex and more difficult

to measure, ouch as those under YEDPA? r -4

. rthink there is potential for changes ih local knowledge development. During the last few years in Massachusetts we have been working

with prime sponsors to develop follow-up capabilities'for all Title r

adult programs. Thid year we're trying to bring in all Title I

programs for youth. Those programs have been supported by a cpmbination of state money, regional office support, university tie-ins,

and some support from the natiomal'office this year. .- '4''

,

. ,

. With rdspect to the expansion of pLMS to cover YEDPA procirams,

I'm optimistic about the potential value o that, because

I thinkit flight help to provide a-comparison group for those activities

that youth are involved in under those' programs which experience a

long term follow-up. For example, we do follow up youth that are

involved in institutional training, OJT or PS over a 36 month period.

Now what we might be able to do is get a longer term follow-up to

compare youth in wok experience with those gbing through regujar

institutional and OJT programs.

What L would like to see considered, however, is the following.

If work experience under YETP or YCCIP is significantly,different , froM what it has been under,Title I, then I would suggest, that we

ought to include a longer term follow-up for regular work experience

'88 - 1 (.)L

6
programs under Title I to see whether or not this'd fferent type of

work experience has any longer term payoff on employ ent and earnings.

ti o In the last year, Bennett Harrison andlI prepared a paper for the

National Commission on Employment and Unemployment Statistics in which

j N$we made a recorpmendation that there be an expansion of the National . Longitudinal Survey for Youth, with major focbs upon disadvantaged

youth, to learn more about the mobility process. That's why I take

a certain degree of personal satisfaction to fee that we'have in

fAct: agreed to'fund such a study. Yet, one problem with the NLS is

the following:,' The question is whether we ought to'contiaue to draw

on .a national probabirity.samble, covering most states throughout

-the. country, or rather should concentrate on a few major metropolitan

areas to learn,more about larger sampl'es of-youth at a 8elected number

of sites. . Now the results in the localized longitudinal studies will

be less generalizable; however, I woutd argue that I think they could

be More rich and more informative about how youth behave in the labor

force, and what determines their'success in the labor market. If you

want to answer questions about the family,ithe'schools, and the labor

market, they're best understood in a rocalized environment. So, I

would, have liked to have seen a NationaL,Longitudinal Survey that

was geared to a selected number of metrop litan areas to give us a.

richer understanding of what really Kapp nsto youth.

On the, questionnaire design, I would have liked to see in the

new NLS more information on the characteristics of establishments

whe.te yopng people find jobs. We have a tremendous data base on the

characteristics of'young,people, on tAe characteristics of their

'families, -on their human capital-attributes; we have almost nothing

on the attributes of the establishments in which they find work.

For instance, recent evidence suggests that continued tenuee with

employers for black youth does not have the Same payoff that it

does for white youth, in.either skill development or wages. Why

is it that staying with your employer does not Matter as much for

black youth? There are a number of alternative explanations for that,

bilt one explanation is that black youth end up in establishments that

have substantially less market power, less integrated internal labor,

markets, fewer opportunities for Movingup on the job. The question

is whether or not the only way we're going to get employment.stability to pay off is to pub black youth into different types of

establishments. Knowing more about that, I think, will give a

rlicher-story for people trying*to design empldyment and training

policy.

The last thing I. would have liked to have seen is a sample of

yduth participating in vocational education programs throughout. the

country, to see how vocational education programs work, relative to

the .employment training pragYams under YEDPA.

Concerning the Job Corps evaluation, my Only comment is that

'while the aggregate analysis is useful for national policymaking, I

- 89 -
1

would be more interested-iA results for each vocational, area in order

to determine the best training occupations..-

GENERAL DISCUSSION:

MR. WURZBERG: I would like to comment on the notion that kids should

be matched with jobs somehow 'tied to their career interests. It's

one thing to talk after the, fact about akid liking a job because

this correlates with his or her erager interest; it is something else

to try to determthe beforehand what an enrollee'or a potential enrollee's

career interests are and try to set up jobs according to that. We're

not recommending that they de-emphasize work' experience. But we're el recommending that they de-emphasize work experience as the principal - vehicle for finding out about other careers.

MR. TAGGART: One of the really interesting things the MDC study

is that it seems to indicate on the average that the new jobs under ' YEDPA are better than the old jobs under Title I i terms of the'

emphasis on supervision and on the quality of wbrk The way we pursu6d

that through our regulations was to insist that prime sponsors have

high supervisor-to-youth ratios. And yet, another conclusion drawn

froth the MDC study was that the Title I slots tended to have more

supervisors per youth. But the supervisor-to-youth ratio regulation

that we ,bad used to, achieve an objective was not correlated with the

objective. Zhe question was,whether putting it in the regUlations

achieved better work-site,supervIsion even though there was no'

correlation between the supervision ratios and the quality.oft

supervision. That is, did we achieve the objective by d4ng .pomething,

even though .it had no direct correlation with the outcome..- The ratio

may be the only way to get 0 superVision in the regulations. It may

then 'be wrong to say that'yolu should not have in ,the regulations the

supervisory=to-youth ratio, even if the studies find no relation to

outcome. To judge from our MDC study!" on the average, the new

programs had better work sites than the old programs. What is it

that made that happen? The only mechanism that we had in the regulations was the supervisor-to-youth ratio.

MR. LACEY: We found that the prime sponsor in a lot of instances

had never even visited the sites. Even though monitoring was stressed

there were good work sites and bad onesiwhetherrar7not-there was tonitoring. I think there's a Larger issue here doncerning how you get

institutional changes. People do the right things for the wrong

reasons. And I.think the prime sponsors, a lot of them, looked at

the letter\of the regulations and reinterpreted it.in,their An way.

NoWi what's critical is how the regional office reacts to that.

In some cases, regional offices took the,regulations and applied them

very mechanically without thinking about the objectives. I think a'

lot .of, prime sponsors are'inclined to say, okayi.wenol. going

to//'get that specific ratio, but instead we're doing something else,

which we think is qualitatively'aibetter way to get at that objectiv..

MR. LACEY: I thinkyou give them a simple objective that they can

FiE4iht on; a simple ratio; and then you also build flexthility

lu
into the regional offices so that'they don't enforce the things 4 mechanically.

MR. STR9MSDORFER: In assessing institutional change and the relationship to government policy I have rarely seen an'awardness that when

you do a process analysis, apart from monitoring the system as carefully as possible, you're going to have to look at things which policy

can.affett and things which policy can't affect.

-There are some variables or some policy tools which can be exercised at the federal level and some which cannot. Those are rarely

identified.t:if you don't clearly distinguish between those things

which are amenable ta,policy and those which aren't, the process

studie.ayq, doing tp be sterile, and the government is left with the

only too/- its disposal-for institutional change, just dumping one

hell'of a lot of money out there. Presumably, we're trying, to .create . some sort of directed institutional change, and not just any old change.,.

I have the impression that the 'studies so far ave not focused ig what

sort of change could be accomplished withi e current system.%,

The second, point I'd like to makes to get back to an earlier

&pmment made by Mr. Sum, 'that theNLS should be targeted to a few

areas. The purpose of NLS is quite different from the purpose of

other kinds of case studies. You can have case studies but you also

need information abput how youth are doing across the nation. That

is the kind of informatioh on which national conaressiopal

policy is going to be made, not on the basis of specific,cases. I don't know if we have the proper balance in case studies, but I do

not feel the NLS is misdiredted.

MR. DIAZ: , We're just beginning our work site evaluation on

Entitlement, so we really don't have anything developed on whether

or not we have better work sites or not. It would seem to me,

thoucih, if they are better'work sites, that my 'Sense is teat the

primary reason would be that there are very lrge dollars involved

in the program, and it was stressed from the outset that the quality

of work sites would be followed up by an evaluation.

t

YIP

I

- 911-fj:;

04.

4

4!,
O

CONFERENCE SUMMARY AND A LOOK. FORWARD

t

MR. HAUSMAN1 . There ary three questions we should address in seeking

to summarize these discussions: First, .what information is needed

to enswerthe questions which Congress irrterested in answering

be-fore it re-enacts youth legislation? Second, will the research

that currently is initiated under YEDPA yield the numerical or nonnumerical answers to those questions? 'And third, what programs'

either ought to be'expanded or newly developed to deal with the most

serious problems of youth unemployment?

I think that the conference has yieldedfor a few of us some

perspective on our research--some .reco%pition of the limitations

I have a sense that some peoele have.Unusallyhigh expects ions

about what research might yield. For example, I have hear&repeat

references to controlled social experiments, and I guess I would

wonder whether there ever has been one. ,Thus I don't know tha,t the.

youth program should be held to that standard when; in my vied, and

I think in the view of many othets, it has never been done, before.

- PRESENTATIONS BY 'CONFERENCE PANELISTS:

MR..NICHOLS: , I don't have the list of specific Congressional

questions. Thp law spells out specific kinds of programs that had

to be run and differeint kinds of approaches that were to be teqed.

Their comparative efeectiveness is really the'big question which

.must bg addressed. But the information that's neecjed is information

of the kind that I think we are Going to get from the Entitlement

Program, though not necessarily within the deadlines. We must

build the calendar into tte planning of research, but not promise

unattainable research'results. On the Entitlement Program, We

should knots how many people went bask to school or are expected

to go back to school. That's a simple kind of question., What is'

the impact on loCal unemployhent rates? That's a little more

difficult to answer, but that's ultimately a Zauestion that Congress'

has in mind- -can we,reduce unemploymentaand, if so, at what cost?

this is ea question that looks for simple hypotheses that can be

_answered, given the state of the art. We cannot demand definitive

answers to questions that science is not up to answering, but this

is one'issue we must addresS.

Congress' goal is to reduce the unemployment rate, and that_

requires reducing youth unemployment now. Even if the jobs programs

do not provide skills pr output value, they serve a purpose at the

present time. I think youth programs are useful-- they arc a source'

of income. Also, in simple terms, jobs must reduce youth unemployment.

We once had an economy where four percent was full employment.

We now have an economy where it seems we can't get near that without

inflation stirring up. We ask Why this is the case., The answer

everyone gives us is thaie the structure of. the labor market is

changing. It's not too/big a question, 3I think, to ask in what way
4

, . and how that structure has changed. If we can tcl in what way it

has changed, I think that's the first step of know ng how ___.o . unchange it. . ,

,

. _ .

. , . I have no.simple answers, though I do thirik the research , . community has the responsibility to define questions more simply

than they've been defindd in the past. I recognize problems of

nonhomogeneity that exist in any of these programs we're involved

in, yet4I think we have excused ourselves tqo long by saying our

job is difficult. There are some things that can be donet We

must move towards greater refinement and specification.

Nfil. TAGGART: Within DOL, we have debated the research 'questions

to determine which 'were prominent and highest-priority issues. We

' stated priority issues in the1978 Knowledge Development Plan.

We have now revised and augmented them in the 1979 plan. In fact,

we have another $215 million of discretionary activity, much of

which is.alreAiunderway, and the plan is soon to be implemented.

Aat has surprised me is how little cognizance there is of this

-broader framewbrk even among researchers involved in parts of the

activities. The question we have to address is not just whether

our little part works or whether we can make it more effective, but

how we bring that totality together. What's potentially different

from wfiat-Ne did in the '60s was that we began by stating the

issues, and then structuring activitie-p to address them. There must

tie .cognizance,of structure if synthes is to be achieved..

Over the last couple of days we have had argumentS'about "Pieces

that should not be assessed in'isolation. Let's take the'simple

issue df work evaluation., We -have four different assessments ongoing

of work valuation. We have a theoretical assessment of work valuation

methodologies. We,thus.should have a number of was of getting at

. these issues, and each must complement the other.

i So the questioris are: What are the magical questions and what

is neede to answer the magical questions? We haVe a continuing

revised ist of what are the major issues, and we have a set of

knowledge development efforts that are directed to those specific

issues. Will research solve those issues? I'think they'll solve

it as much4as research, evaluation and demonstration can ever resolve

such things:

There were a number of challenges to us to improve the content

of those activities. Many- of these are,vaid. Others have been

addressed.. The plan for next year lays out a whole set of programs

which fill many of the gaps that have been identified. What are

kids in the ghetto really doing with their. time? What should be

done for special groups--the handicapped or young mothers, for instance.

Akvariety of activities to address such issues are written into the (

plan for next year because we already know we have to address those

issues.

So somehow, everybbdy who is doing parts of this has tb grab

hold of the totality. We have to be concerned not just with what

t.

- 93 1..0G
'we're doing but what everybody else is doing. We've got to coordinate

empirical and theoretical research with evaluation and with demonstration activities. We've alsb got to integrate institutional change

measures,' econometric measures, 'economic variables, psychological

variables. We've got to have pure research combined with advocacy

t research'as, well as publicity efforts. All these things have got

to be integrated. ..

-There seems a reluctance to get to the 'big policy issues, to

get to this synthesis. Everybody seems to be focused on narrow

questions.

I think the direction that ASPER suggests is Crucial. First,

what are the big issues, how axe we going to solve them, and in

what time frames? It's most appropxiata and mecessary_for_everybody

here to also'adaress those same questions and also look at the big

picture. Anti 1 think there's,absolutely,no question that if

everybody does that, we will have the resources and the flexibility,

to achieve the major things that Congress wants.

I think what we have to do is look at how the total effortis

structured. We've got to adjust our plans, if they are. deficient.

For instance, we did not-play out the time frames for results or

provide for their integration in the initial drafting of the 1979

plan. Appropriately, wp were sent back to the drawing board to do

that.

As we go along, we're going to have to adjust and refine these

plan . e're going to have to go to each evaluator and say, "Can

yo-Qspeed,up on this, can you get additional kinds ofdata at this

time; 'be6ause it's what we needr And all of us have to do that

together.

MR. O'KEtFL There are some very significant questions that Congress

wants answered in the Youth Employment and Demonstration Act. They

want to know what the effects are on educational participation,

youth income, increases in employment, and changesvin labor forceparticipation rates. And I think that everybody in this room has

'to lake an effort over the next six to nine months to pull together

wha ever information we can.

However, I think part of our reSponsibility is not just responding o some very narrowly-defined questions. Part of the process

is t raise for the Congress and for the public-at-large the other

issues that we should be concerned About as we pass a law--a law

that's probably going to dictate our youth policies for the(1980's.

For instance, I'm not sure that youth policies in this country'

should be directed solely at reducing current youth, unemployment.

We do know these kids are going to grow up, we do know that they're

I 47only look at

going to

every policy that we put in place should no

be in the labor force for the next 45 years and therefore

think

t its immediate effect in terms of unemployment or labor force

participation or education, but it should also'ask the question,

how its it affecting the longterm prospects of these indivi'du'als?

- 94

11,2

.
. . $

. If I had to give-A example of the kinds of questions.that

we're being pushed to answer right now, I would say that we've got

to find ou,t over the next few months how the programs affect -the

labor force participation rates in narrowing the differentials that

exist betWeen minorities and whites. How did they affect income

derived from,the employment? But, I think that assessments of

unemployment changes are much more problematic.

For Atance, I think there's a real question for educational

policy makers as to the extent,to which labor market interventions

are able to induce youth to return batk to pchOol. I've oft6h

wondered why, in our education policies, we haven't been able to

'experiment more directly with paying kids to go to'school, especially

kids we can identify as being at risk. It doesn't seem to me that

hat-Ls-a-labor market intervention but rather an educational _. intervention.

The leg i lative cycleswill be very unyielding. If there are

going to be-any big budgetary changes -in the next-budget cycle they

are going to be in the youth area. The Adainistraetion haS)to respond

to the inflationary pressures that it's confronting and what,were

going to find is that there are going to be decisions on both the

Hill and the Administration abouecutting the budget.

- In the youth area we've got to be very careful about makihg

a case for how much money we want to take in a categorical fashion

for youth labor market interventions..

Over the past two clays, we've.been told about a lot of good

research, evaluation and demonstration activity that'S been going

on. *lit now there seems to be a lot of people out there re- writing

he various books of the Old and New Testament. HOwever, a lot of

policymakers and policy Analysts like myself never really did read

the Books. We've alwayslooked at the Clssic Comics. There isn't

anybody right now pulling' together the Classics-Comics version of

what we're doing.

The time frames arejairly narrow, nd therefOre, what we've

'got to think about, as each of .you do yo individbal studies, is

how is it you can relate it back.to what you've heard here, to what'

you-knowt1i4t's going on.4

But, I think Its-- must do more than simply respond to legislative

objectives'. We must put on the table the extent to which the

objectives we had fOr the current program are realistic.' Is it

realistic to expect that a billion-dollar', program is going to

,reduce or seriousl cut into the problem that's been with ut for

20 years, a problem that we've, spent sofnewere in the neighborhpod

of $50 billion on over'the past 15 to 20 years? As we go forwalad,

we've got, to try to decide the extent to which our expectations for

interventions in the labor market on behalf of youth are realistic,

and whether those expectations can be carried out in .a system that

pulls on a string at tlizi national level and expects r6sponses at

6
the state'and,local level.

a

MR. BRIGGS: If I were a Congressman, I would ask first abbut the

impact on unemployment. I think the law w' passed largely as a

reflection of concern about youth unemployment: But I hope that

that's not the first question someone will ask, because' I think

that would miss the whole contribution of thiS legislation. I.don't'

think the Act and( the programs will,likely have much impact on

-youth uriemployment. Thatdoes not mean that i think anything iswrong with the legislation. The question is, "What is it we expect

to get?" Is it job creation? Is it reduction of unemployment?

Or is it getting people back i& the Schpols, and back into learning

situations? I think the prime sponsors see the latter in this

legiglation, and I think this is an accomplishMent. Entitlement

Programs and.a lot of local efforts' strongly emphasize getting

-people back in, the schools, or to stay in schools, whether these

people were unemployed or.-out'of the labor force,. So', either they

were At school to begin with, or they were. potential dropouts'and

are"now being identified so that they'dOn't become dropouts.

Therefore, it's going to be pretty hard to figure out just what the

impact might be on the unemployment statistics.

ybu'really try to focus on out-of-school youth, thpse in the

labor force full -time, then the problem4,is, of course, that you

encourage people to dropout of school, and that's what keeps the

prime sponsors from doing it. c I have run right into that in a

number of areas. One of the reasons they don't want to swee:k:e.n up

OJT for, young people ,mss that they fear that,itill be an enticement

to leave school. , And so*they want to stay awa'S, :from offering too

goon an enticement. /

There are some mixed feelings in these discussioniabout whether

or not. the purpose is to get people` back in the schools or into

educational training, or.into alearning environment of some sort,

or whether we want to actually get them into jobs right away.

This paradoxical question runs through a'lot of the legislation

and a lotpof the programs; the prime sponsors are faced with the

same-question.

I think the knowledge development activities that are being

conducted Toctis on two separate themes. There's one theme of

spohsoririg research on youth in general to learn safiethrng about

youth. It is marvelous that we're finally able to get attention

to youth problems/ and yo th labor market conditions and to-identify_

who the yOuth are and to sess the effectiveness of institutions

that impinge upon their li es. On, the other hand, we ,also%have a

lot of activities tryihg to evaluate, study, and assess programs

that have been crekted under this legislation. We have to keep

those separate sometimes. I can support the basic research, the

idea of doing basic'resea0h for young people in general, and I

think that's very helpful, 'and positive. V

On the other hand, I think we also'have to be careful,that we

may not set the same standardswhen we get within the programs.

4

g6
The programs run into a tremendous amount of institutionalresistance in trying to bring, about change, as in the CETA program

last year. Quite frankly, in most of the Grime sponsors I have to

,deal with today, the YEDPA is a backburner These.people are

now concerned with private sector-initiatives at,the local level.

Since we were there last fall we've had'Title VI buildup,the HIRE

and ST.IP programs, etc. The most important things in the second

year are to. keep the youth progi'am at the local level 'and to see

to it that the prime sponsors and some of the other program people

takeitasseriousandhavethesameinterestthat'theyrhad last

year.

I think that the educational institutions, in the schooling

7 process are the ones on which this Act may have the greatest

impact. The educational establishment is massive and very hard

to chadge. By definition, an "inptiftution" means something that

doesn't Aange very quickly. If'you try to find measures for

desired outcomes, you are not 'going to find observable change in

three months or six months. In fact, .I.really fear that you're

going to run into the story of the man who wenttout to-pick his

'flowers every day to 'see if they were growing, and killed thers.

The flowers have just been planted; the programs have just gotken

started. Now, it's important to track them and watch them grow.

But At's not demand totally definitive answers so soon after many

of 'these programs have just gotten off the ground.

That leads me back tb one of the other institutional problems

'that we've got that, affects all of these progfams, the whole

CETA system. YEDPA, like all CETA programs, suffers from a lack,

of planning at the local level. .They're trying to plan; there's

a lot of encouragement to local planning. But maybe it's just

endemic to Zhe whole process that there can be very little planning

for CETA ma power programs. The lack of national planning in many

ways filters down to the local level.

Under YEDPA, there was great pressure to get started last fall.

In most cases, it really didn't get off the ground vnt4, well into ' the spring, and all the pressure to get it going only clomplicated

the process. The first six(months were chaos at the priMe'sponsor

level. About the time it did get off the ground, the private sector

initiatives were the big things pressing on these sponsors. It's

very hard to isolate exactly what Vpie impacts of these programs

are.from looking at either the program data or even some of the

case studies. Thbre are a lot of other things going on.

. At the same time, institutional changes are somethirig that we

do need to look for. I think they are coming with the schools, with

the vocational education establishment, with academic credit for

work experience, GED progr4ms in rural Mississippi. and the like.

This legislation has been a wedge for introducing a lot of new

ideas li,ke alternative schools for sponsoring some of these .and

saving some programs that were about to get cut out because they

were too expensive f'or local school districts.

- 97 -

y
It's a grab baa of accomplishments, but in ffany ways I think

that the Youth Act, by focussing so much of the attention on

education, has taken on the institution that is most important in

youthS' lives-and that's the educational system itself. It's not

,going to change the educational system entirely. It's not meant

to affect all youth. But for many of the youth that have become

the clientele for many of the programs in the past, this legislation

may ha've some impact.

I don't.know how you actually convey that idea to,a Congressman

in any precise way, because in one community it's the vocational

educational establishment that your're dealing with, another it's

the dropout programs, and so on. It's .a variety of individual types

of accomplishments.- But I think we shouldn't lose track of these

accomplishments and in the second year tae ought to continue to push

the prime sponsors into exprimenting more in trying to reach new

groups and to try to document in some sense some of'the'institutional

change that's taking place, as well as some of the normal quantitative

measures. We're asking a variety'of quantitative measures. Maybe

we ought to ask researchers to tell us a few stories, and give us

some non-numerical accomplishments and changes.

I've seen institutional changes in the communities hat I've

been to--major changes that can'be credited directly to this Act.

Someone can pinpoint this, and Say this Act has helped us/make this

change. As an example, consider a work program in El Paso.

People.there have been,pushipg for yetrs to get it; with this

lergislation it came about. A Albuquerque, there is movement into

a whole'new,tindfttry,, the hotel industry, by the cooperative education

community. Nothing ever touched this industry, the largest one in

thZ city, and haw they're doing it.

There are a lot of 'things that this 1(tgislatic\n can accomplish,

andt I don't knOw whether you can capture them in the quantitative

data, especally if their real impact is likely to be on,,the.

institutions, and especially on educational institutions.

MS. WILLS: I think to understand these youth .programs we need''"'

sen e of their history. There's a very smart man we have in

Was ington, Ray Marshall. He went down to an island before Carter

wa sworn into office and he figured out fairly quickly-that the

on y-new money that was going to be in the budget in the first year

the Administration was somethinwthat was going to be packaged

d was going to be called an economic stimulus program. Mr. Marshall

ad no illusionsithat it would solve all the problems of youth. You

an't do that inla one-year program as part of an economic stimulus

program. He seized upon the opportunity, and thank God he did, and

we had the youth legislation.

Another thing happened, though, on the way to this legislative_

forum. One of the criteria they used when they were trying to put

together that early package was to avoid,new pieces of legislation

by using existing authority. It was correctly pointed out to

- 98 - 1"... 4
Ray Marshall at that time that he could use Title III, the Secretary's

discretionary authority. But there was a very basic and fundamental

problem: 1\16body trusted the DOL to use the discretionary authority

of Title III in CETA legislation, including myself'and the organization

I represent.

Interestingly enough, he could have done it with one paragraph

in appropriations legislation; he could have gotten all the money

thait he needed, $1.5 billion. Instead, you have 10kpages of

legislation. In the process ogetting that 100 pages of youth

legislation, a variety of missions came into being. Quite frankly,

it was a plea; it was an act of desperation on the part of concerned.'

people on the Hill and within .the Administration.

1.

Indeed, there is not only the,economic stimulus to put the kids

back to work. As .1 recall the statistics, the range was- between

600,000, up to 1.5 million youth in this economic targeted population.

And indeed we can serve that'number of aeople with programs coming

out of the Department of Labor in a yeaP or a year and a half. The

question is, how do we touch all those people, and with what kind

of programs?

I remember well,'because I sat there and talked with the lobbyists

of the education community that pounded on the door and pleaded with

the interim task forces that.were trying to put' legislative agendas

together. They wanted very badly t' get an increase in terms of

fl)nding fOr their programs so that they could begin to affect the

lives of youth. They were willing et) admit they hadn't done a very

good job.. They didn't get it, because it couldn't be a part of the

`economic stimulus package.

"racer kind of question that came out, and is one of the reasons

that there's so much Secretary's discretionary money, is, "how do

ou really target?" You knQw there are inadequacies in terms of

the formula grant programs in terms of targeting. eIs there any

possible way within the political proCess to.target the funds on a

formula grant basis that's goipg'to make any difference to the kids

in the .inner-city ghetto and out there in the rural areas? -

.

. Clearly, another mission of that legislation is institutional

change. It's the publicly financed structures that' were very clearly

the target. If you bother to take the time to read the -committee

reports and the conference committeeeports,jt'§ there, and it

(

r i a plea. That's why you have the 22 percent set aside. I'll neve

orget the day that I was saying that you,can't do a 22 percent

et-aside the way you have it written because there are 1200 public

districts or so in -a particular state and there are only 23 prime

sponsors; now how are yoll going to mesh it? The point was that the

Congress, in an act -of desperation, because we, representing-the - institutions in the manpower world, and the educational world hadn't

bothered to get ourselves together, decided they'd do it for us.

Indeed they did.

0

99

1 1A=2
Academic credit wasn't in the first draft of the legislation.

Three different times it was taken out as a goal. The Department

of Labor did'not want academic credit placed in that legislation.

As you an see,' it is in there.

So I think, that there is a very clear plea for a varie\y of

, missions, a,variety of"tests that the knowledge development planning

is attempting to address. By the way, I think Ray Marshall was

right. He knew that there would be continuing money and that it

would not just last one year, and I think he's probably smiling

about that.

Ther2are so many missions thatigit is necessary to keep an eye

on all of them. FOr ,instance, J think you're going to have to take

look at some numbers on female participation, for example, in

YACC and YCCIP. That obviously has some implications in terms of

'progranj design, whether or not you have a separate part of the title,

for that kind of program. More than that, throughout that legislation,

When one bothers to correlate that with the legislation going through

now, you will find that the Hill is getting much stronger in terms

of the language used concerning elimination of artificial barriers

and. sex stereotyping. I think we need to pays some srcial attention

tq the issues of the handicapped. That'sjust,a policial observation.,

The Minority Leader in the House next year, for this committee, is

very concerned about the handicapped and I suggest you cover yourselves

on the issue. Besides that, it's an important.issue.

I`

. I think a question that 's going to have to be answered is

whether and by what means we reach the target population,. which

basically translates into,blacks, Hispanics and low-income 'whites..

What are the implications; for instance,,of using the 70 percent

lower Aricome living standard? Should we raise or lower it?

I.Would also suggest that we will have to consider univerSa-1

coverage. responsibilities.. For example, there's the vocational

education.program and all the educational systems. I'm hearing when

. I'm talking to labor market peopre that kids are dropping out because

of the real and/or perceived opportunities in CETA of the income

transfer payment program and the family income and participtption

in this part of CETA and the rest of CETA. What loes that really

mean and what implications would that have fori de4igning,policies

perhaps more strictly on the income transfer side, as opposed to

the manpower side?

Another issue is labor market and occupational information.

Congress got frustrated with the lack .of response on the part of

the federal fiscal agencies, and the lack. of coordination between

HEW and vocational and CETA systems. It created the National and

State Occupational Information Coordinating Commiqee. Rest assured

you:re going to be asked to respond back to CongresS- about what you

have done with the mis-Sion,.of the NOICC and SOICC, in.terms of the

development of the career occupational information systems. o Also,

the local primesponsor system in 90 percent of their responses

on Oat they want out of state nd fedyal government say more

40.

a

J t.

- 100 -
information on significant segments of the population. They're, /).

desperate for that kind of information. They don't have any help

and nobody's giving them any help on that. And if NOICC,isn't'the

Tight solution, then some of you say so and give them better' alternatives next time around.

I. would hope that one of the things that you:re going to be

able to take a look at in the case studies are ways that would make

things easier. This is really a plea, having operated some of

these programs out there. One of the responsibilities policymakers

have bAck up here in Washington is to try to write a piece of

legislation that will make it easy as possible to administer. We

often don't do that and we end up having to have five million differerit

administrative councils, regulations, and the like.

MR. POULARD: What information should the research provide? '1 believe one major concern shOuld be delivery institutions. Questions

should be forthulated, raised, and collated that address the mechanisms

and the institutions that seem to be effctive, as delivery agents,

as operators of programs and advocates of the right kinds of'policies.

Ithink any.research well done would, or should, acknowledge that

even certain of the sacrosanct institutions that have been a market

for the Department of Labor haveflubbed over the years. There's

nothing to be seen in their participation at present that can

adequately recommend that they be left untampered with. Conversely,

there might be seen some institutions that have had to fight their' - way in. These might be identified and the research finding used to

support their position so that thli no longer have to force their

way in. I ha40 in mind, in particular, community-based organizations.

Ideologically and geographically they are nearest the poor. Almost,

all of them came into existence primarily, if not exclusively, for

the purpose of helping the variety of individuals who are chronically

and structurally unemployed.

On the basis of that generalization, of course, one can proceed

to argue that they ought not tiohave to rely on circumstance and

pressure tactics to force their way into meaningful roles. At least,

as research bears out the useful role the7're 15erfbrming at the local

level, perhaps their task,should be made easier regarding their

future participation.

I would cite, as one particular example,ithd fact that out of

the limited discretionary money 'che YCCIP 'program, $12million

is being spent by discretigp and 4iven to HUD, to pass on to ten

community-based organizations. Some of you wen here might,be "14 startled to learn about the additibnal millions of dollars whic

these ten community-based organizations parlayed by virtue of t ir

small grant .s. For instance, the Watts Labor Council leveraged IA.

money from' Washington with additional money ficom the state, the

county, and the city: It was able to affect-ery quickly the

appropriate kind of working relationships with the union and that

was needed because housing constructionewas undertaken. It_wasli

able to train a large number of personp in the construction trades.
a As a result, there is a housing stock consisting of new construction

and major rehabilitated units..being produced at a pace that might

embarrass our largest contractors. It's there for the skeptics to

see". Beyond that, Most of these are home-ownership programs, so

that in 20 years 'the people, who currently pay theirnotes, will

own the housed. Where do you get that kind of bang for the money?

And who's doi.ig1it? Not some sophisticated institution but, rather

"a ghetto-located community based oxgaiizatio.

I'll admit there are some that can't do that, but many can.

My point is on behalf of those who can. That CBOs have to spend

too much time and too much of programmatic energy fussing, fighting,

struggling, and trying to get in.

The point again is thaI I hope the research Will yield information,

that will profile adequatel' the meaningful role that some of the

CBO's are now playing and might play in the futuv.

I would hope also that the research would yield some views of

participants. Frequently I hear it said that there are many thousands

of jobs going begging,. that unemployed youth and others don't care to

accept. Why is it that unemployed youth don't want tije jobs that the

newspapers advertise, or other jobs that are supposedly available?

Might these youth tell you, if questions were posed to therd, what

these jobs are like, the kind of wage they get, which when received

represents an inadequate payment for what is expected of them?

0 I would suggest also that questions be formulated regarding

the views of user-agents. What do some of these user-agents th-at

get, the subsidized labor think of people sent to them under these

programs?

If you find out what I think you're going to find out, I would

not ask you tb shift gears-or wear a new hat as advocates and no

longer be scientists. The rest of us will do that. If you were

to learn the views of some of these user agencies, I have an hypothesis

regarding what you would find. If you were td make public these

findings and hold them, up to public debate, I think at that point

the advocates would then assist the scientists in pushing that

issue on behalf of the chronically unemployed.

MR. LOVE:.,There's °lie thing conspicicously missing in the debate:

the role of the private sector. We have heard talk of "increased

demand," but I don't know how we're going tb increase that deMand.

For instance, there's alot orPublicitvibehind pi,ivate sector

hiring. But the HIRE program never got1off the gr'ound. In the

Labor Department, its known as How Industries Refuse to Employ.

So, if you're talking about expanding job demand, it's got to be

in the public sector. That's got to be where youth legislation is

geared. The AFL-CIO, organi;ed labor, hats been spending many years

training youth, as, for instance, In the Job Corps. This is important.

But the end,answer is that there are simply not enough jobs.

- 102 -
There is also the problem of adequate wages. Everyone seems .

to agree:that the minimum wage is fine for kids. If all youth see

is working at McDonald's, is that going to inspire f*ure prospects?

The biggest.problem I've found in the last ten years, is that \. organized labor was not contacted and involVed in program development

activity.. You wait until a program is detigned and then say ;sign'

off4on it-" That's wrong. Apprenticeship programs are the biggeAt

training mechanism in this country. TlTey are jointly operated by

management and by labor. You'are not going tc get into that system

unless you deal with labof.

So, if you're going td go out and do research, read the .basics.

Organized labor is written into the legislation, and we'r going to

be in it. If not, we're going to rap up and down the La or Depinment's

halls and even go to the White House: We hke fought for this CETA

and we are going to play a role because organized labor is here and

we're going to stay here.

DISCUSSION:.

MR. POULARD: I would hope also that the researchers are able to

formulate some questions that would,provide some insights regarding

the factor that many tf us feel is most important in youth programs- -

the personnel directly involved in program operationsy Nobleland

high-sounding goals of these programs notwithstandfng, it -cbmes

down to the selection of people who provide counselling and training

and support services:

f remembe. one case in Chicago where a group was asked to,place

only 20 percent of the graduates of its very complex training program.

With unemployment_rampantc a particular lady ended Up placing 8Q%.

She did so well that the prime sponsor appraSached her and begged'

her on behalf of the city to accept another contract to do placement

for other agenci doing the training or the.city. It's not because

it was,a community-based organization alone, it was because those

people were committed, were trained, and,.wee skilled. It was because

the director of the, organization did not atsume that; just anybody

`was qualified t9, do it. They were careful in whom they hired.

Again, I would suggest that the researcti, if posSible,.formulate

some questions to get some data regarding how critically, important

personnel hrepin these'programs. It's not just the legislation,

it's not just the volume of dollars, it's not just the program

design. SucceSs primarily is determined by who's running j.t,_

I think Sometime , at least at the level of regulation or field

memoranda, cre entials or requirement or qualifications for personn

should be add gtsed. 1 MR. TAGGART: T his legislation is quite different from he rest of

CETA in terms of labor union and community-based organ ations'

involvement. YEDPA requires front -end involvement:of labOr unions

and special consideratioii'ofIcommunity-based orgar4zations. T

reasonv,thak special considerhtion was given4 the reason labor' on

- 103 -

4
front-end involvement was put/in the legislation, was because that

it was believed these alliances would improve youth,. programs.

It is our job to assess whether these assumptions are true. 1 ), The prodess evaluations restructured to assess whethei- communitybased organizations do tter than established institutions. In

addition to what, we h ve earlier discussed the HUD CDC project;

in which we had compari ons betWeen the CBOs and other performers.

We have,found that,'indeed, community-based organizations have had

good leverage. That is the finding, and it will'be researched,

further. The school-to-work transition demoRstrdtion which has not

been discussed has a group of CBO's in varying circumstances offering

well defined, well structured, and sensitive services with itpacts

measured by past-tests. The Employment Servide is doinig,th.e

same thing so that we can test comparative effectiveness.

P

fe Next year we're funding CSA to work with Farmers Home Adminisrtration to test the effectiveness o rural CBOs as opposeA to the

balance-of-State delivery institutions.

We're iloing the same things with'labor union involvement and

whkt we as researchers have to-address is"whether special considerations

and labor union in olvement have measurable benefits.

MR. OSTROWER: I can relate someothings tha- we saw pl our studies.

we'baw that CBO's, of course, have greater participation under YEDPA

than they did under the Title I program. Stt we alk) saw that 'CBO's'

were more successful and more effective when-they dealt with things

like motivational training and job preparedness than when they were

beginning to deal'With 'things like training and other types of things

/that'had the compete against the traditional agencies. Much of ' ithrrmay have been learriijpg curve prob;ems.' Those groups that are

getting money for the first time and Ibing into construction are

having more start-up problems than those that, have been at ,it for

awhile. But what I think we're noticing is that they` -re developing

Isimilar structures.

Another point is that wl.i.er the community. based organizations

have,a geograLhic.focus, the have a"constituenc arithey basically

work with'thcige in that constituency. In other ords, they* are more

likely to target: 0'

MR POULARD: I'm beginning toffeel that I'Ve d9ne CBO!s a.great

disservice. The last thing I' meant to do was to make Cdirtmens which

would pit CBO's against other institutions. That is not 'act all my

point. AyjDdint is simplythat they've had to fight and force their

way in., Let all the oth* grandta andgrandpaarganizations that . have been around singe Day One and that are still effective remain.

If they can train better, God bless them, let thet train. The

only was seekng to make is that there ought to be, a

comprehensive mix, the utilization of all prbgram components a

institutions as well- If CBO's are effective, then use them

p

= 104.74'

(
y

they can only do outreachr^let them do outreach. If they can also

train, let them train. At the level of pgrception, of policy, of

legislation, regulation, of field memoranda,'let there be some

proyision which recognizes them as an'entity, as other traditional

entities are recognized as entities, so that they are. more easily

involved in the process, rather than outsiders who have to always

fight their way in. 'It's not one or the other; it's a,partnership.

MR. OSTROWER: ,Wp're in tilt proces$ of doing the comparison studies

right now to determine whether CBO!s do a better job of targeting.

It depends on whether the community-based wganizations have their

own territories. In cities like San Antonio, where the groups are

funded through the prime sponsor and are'expectea to have a larger

territory, there's less targeting.

W. LEVIN: When you look at the

44d youth problem, one of the reasons

we're in it to a significant-degree IS-becalise of the schools and

their inability or unwillingness to db significant tlaining job

with kids. There are ideas about alternatives: One that I'm

thinking about is school votcherg. This "market" approach might

make more sense.

MR. TAGGART: We've made a major commitment td testing educational

vouchers. We've completed implementation studies, and it'sra.

complex issue. We have seven million dollArs net yeas set aside

to provide vouchers for participants in selected' work experience

' programs. ,

i We are, in fact,.testing all the alternatives" to schools we

can,think of. We've got the Career Intern'Pr9gram

alternative. The Job Corps is the largest alternative education

program in the country. We've got educational vouchers, which is . another Alternative. We've got the,CETA'system_creating all kinds

of alternatives; We've got schools .reating alternatives under the

Exemplar' In-School'Grant Program. The question is, which works

beet and for whom? Some will do be'tter with'some types of kids

than other types of kids. We have to iailpFatp all that research

to assess comparative effectiveness. Ypulye got to have all tliN

angles on things. to assess them.: That's why the structural m foundations-in the knowledge development plans are-so important

and that's why everybody has got to know what.- everybody else'isY

, doing. . .

. 0 -MS. GUERON: . I'd just like to add that Entitlement pis the most

targeted of all programs; it is open only to people on poverty

levels or if the family is on welfare. Theie are some,difficulties

in income Conditioning the program, but I4think there is going to

be quite a, good job of enforcing the eligibility criteria. To that

extent,,itmay be more tarAted than'may be the other programs.

MR. TAGGART: The most important fact is that there are kids out

:there that want to work and'are coming out of the woodwOrk, if

we offer them even minimum wage jobs, much less jobsAhat pay

- 105 -

a.
s higher. -One thing 'we're going to prove by.the raw numbers is

something realistic bput the universe of need.

MR. HAUSMAN: One thin that amazes me is how many-agencies--and

now yours, too--are ilthe business of determining eligibility by

income and other thing . One time or another, we're going to fund

a research prabject to ee how much money we are spending per family - to look at its eligibi ity for several different progrqms, whose

eligibility standards on't vary too much.

MS. GUERON: The difference is that you can bring in yotkr welfare

stub and that's the ticket to eligibility fOr'Entitlement.

MS. WILLS:. I'd like to perhaps speak the unspeakable on this question

as to whether we're serving the hard core. I'm not sure that it can'

be done in this next ye,,ar or two years, but I think we're going to

have to look eventually*to reasons why the dropout dr the younger

individuals simply do n$t participate in sate of these gfOgrams.

* My conversations, interestingly enough, with a lot of CEO's

indicates that they a e seeing a tendency for what they're

calling the "CETA husj le", in the local areas, where the kid comes

in and he drop,s, out nd knows how long he needs to bp unemployed,

and/or is notgoing to take the job,vbecause frankly he can make

more money in other economies of our society.

I think you're oing to have to address that. One can get

fancy and talk about reservation wages and all theseopther things,

but I dOn't think it s really necessary. I think tha t-In the next

year 'or two years,s,y u're'going to have to think AbOut those you':

are really calling e hard core. ; thinie''We're'going.to be forced

to begin to address ose kinds of questions, I think it raises

weStions from DOL perspective,-as to whvg are the limits of -, government intervention in terms of employltent and training programs,

and I thinklit needs to-be approached from that way.

One other comment in terms of dissemination. Lheard a lot

of words about,replication yesterdaT. And I would suggest to the

representative's in the Department of LabOr that perhaps one of the

thif4s that .1A need to think through a little bit more is to think:

through, a strategy fOr real dissemination bf information. The

project operator or the user out there in Mississippi may

or may not havp access to or interest in documents. Everyone in

the world ca'1Th for a better training and technical assistance

from the Department of Labor, but perhaps one of the things that 0 we can don4is with utilization and the regional training'grarits

and a whole syste4c strategy is go beyond that.

And I would think also that the approach of developing a cohwsive

knowledge development plan, itself, needs to be replicated in Other

parts-4,5f ETA.

106 - 110

41' 4

r
)

i

k

. .

0

r

t I

t

PART II

CONFERENCE PAPERS

a

.......

/1 p

I

c

%.

I

4

4

z

M

ar

s.
,

V

SUMMARY OF RESEARCH ACTIVITIES ON THE

YOdTH INCENTIVE ENTITLEMENT PILOT PROJECTS

1

4

I

Prepared'byi, Manpower Demiinstration

Research Corporation

September 15, 1978

This paper summarizes research #ctivities on the Youth Incentive Entitlement

t Pilot Projects that will be conducted by the Manpower Demonstration Ibsearch Corporation

(4DRC), and Abt Associates, Inc. (Abt), MDRC is responsible for the development and

oversight of all research activities on Entitlement. At MDRC, Joan Leiman is the

research manager, and Joseph Ball and William Dia1i are responsible for the design

and condugt of the implementation analysis. At Abt, Ernst Strommlorfer, the

principal 'investigator, and Robert Jarrett, III, the project manager, are responsible

for the impact analysis. ,
,SUMMARY OF RESEARCH ACTIVITIES ON THE

YOUTH 'INCENTIVE ENTITLEMENT PILOT PROJECT

The Entitlement Program (Titlegv Subpart T of the yEDPA), is designed asa demonitration--ah experiment--to (1) test whether School-year (part-time)

and summer (full-time) jobs can be guaranteed for 16-19 year old, econom cally

disadvantaged youth residing in a designated area who arp in secondary school

or who are willing to return to school or enroll in a course leading to

certfticate of high school equivalency; and (2) test whether such jobs

. . -c will be an incentive td increase high school-retention, return, and completion,

t

--. ; and'longer term employment and earnings of program participants.

Within

part-time e.

desi

V gnated areas during the schwa year, otherwise unavailable

Aoyment or at combination of part-time emPloT

ent and training

is guaranteed to thops economically disadvantaged

of 16 to 19 inclusive3r4to are in secondary school'

I ' a certificate of highschoolequivalency. During

employment or a co nation of employment and training is provided to all

youth between the ages

or a program leading to

the summer, full-time

eligible youth.

Because of the high cost of Entitlement, ZOky a fewfull

scale programs could be launched that would test the saturation of an

em titre central city or multi-county area. In order to test # nuMber,of

secondary experiments and innovations proposed by the legislation, the

Department of. Labor specified a "two-tier" approach. Seven large-schle,

or Tier I, projects test whether CETA prime sponsors can feasibly

10 -74. implement programs that will encompass all or -a large segment of their

service, area. Tiff demonstrations' Are expected to pro` ide_iobsfor

between_3,500 and 10,000 youth. Somewhat tmaller Tier II projects, each

1') r/

- 108 -

0'"
a covering the area served by a particular school or small school district,

test a greater variety of programma4c approaches. Each will enroll _ no more than 1,000 youth and the.majority will enroll less than 500,:participants.

To meet the legislative mandate which specified diversity and required

estimating the probable costs of a nation-wide entitlement, sites were

selected to represent a wide economic and geographic rang- -both large and

small cities, high-density urban areas and sparsely populated rural regions,

and areas of varying unemployment rates, school dropout rates, and lLacial/

ethnic composition.

In section 329 of the. Youth Act, the Congress has directed theNSecretary to report findings on the efficacy of the entitlement projects with

respect to:

"(1) the number of youths enrolled at the time of the report;

"(2) th cost of providing employment opportunities to such youths;

"(3) the degree to which such employment opportunities have caused

out-of-school youths to return to school or others to remain

in school;

"(4) the number of youths provided employment in relation to the

total which might have been eligible;

"(5) the kinds of jobs provided such youths and a description of

the employers - public and private - providing such employment;

"(6) the degree to which on-the-job or apprenticeship training has

been offered as part of the employment;

"(7) the estimated tost oT such a program if it were to be extended

to all areas;

109 -

1 o

re
"(8) the effect sah employment 'opportunities have had on reducing

youth unemployment in the areas of the prime sponsors operating

a project; and

"(9) the impact of job opportunities Provided under the project

on other job opportunities fOr youth in the area."

Additional research concerns can be inferred,from the Statement of

Purpose to the demonstration projects Title of the,.Act (section 321):

'(10) "It is explicitly not the purpose of this part to provide

makework opportunities for unemployed youth; instead, it is

the purpose to provide youth...with opportunities to learn and

earn that will lead to meaningful` employment opportunities'

/7 s after they have completed the program."

Finally, the Act specifies congressional interest in understanding

the importance of the service delivery system for implementing entitlement,

with a statement that the purpose of the demonstration projects shall be:

(11) "to test the relative efficaCy of different ways of dealing

ir* with these problems in local contexts...(section 321), and by

providing for tests of 'a variety of administrative mechanisms

to facilitate the employment of youths under an' entitlement

arrangement...' " (section 327 (b))."

The Congressional mandate is reflected in the Office of Youth Program's

Knowledge Develcpment Plan which addresses the requirement that the'demonstrations test the efficacy of Entitlement under differing socio - economic,

regional and governmental circumstances (p. 10a); assess, the feasibility

of Entitlement for entire prime sponsor jurisdictions and estimate the

probable costs of such programs; describe the beeps and nature of jobs

which mould be created; and assess the program's impacts on labor markets

- 110 -

S.
and school completion and retention rates (p. 11). In addition, the Plan

anticipates that the research effort for Tier iojects will also examine

the long-term impact of the program on participating youths in terms of

their post-progr4m labor market and other behavior. Finally, the Plan,

notes that the research is to be concentrated in the Tier I projects,

with a qualitative or process assessment of the smaller Tier II projects

which would examine the range of new and improved approaches, methods,

and techniques to be explored and demonstrated at these sites and would e(

also generate knowledge to complement the Tier I analysis (pp. 12-13).

Three major issues will be analyzed to meet the congressionally mandated

research findings and the Knowledge Development Plan:

a. The impact of the program (on participants, eligible youth,

and other youth);

b. The costs of the program; and

c. The operation and coordination or implementation of program

service delivery mechanisms.

The Department of Labor has contracted with the Manpower Demonstration

Research Corporation (MDRC) to conduct these impact, cost, and implementation

analyses, and to coordinate the implementation of the Entitlement demon-

,Stration under the supervision, management and direction 'of the Office of

Youth Programs. MDRC has, in turn, subcontracted with two firms respectively for the conduct and analyses of program impact surveys with eligible

youth: Mathematica Policy Research of Princeton, NJ and ABT Associates

of Boston, MA. Implementation and-cost analyses will be conducted by MDRC

research staff. Each of the analyses and the congression'ally mated ated issues

and questions which they will address are discussed below.

1
4

THE RESEARCH QUESTIONS..

IMPACT,ANALYSTS

Several of the mandated research areas concern the impactof the

Entitlement demonstration upon youths in entitlement areas and upon the

labor markets which they comprise.. With reference to the legislative

questions enumerated above, the impact analysis will address the following

.iseues and hypotheses:

A. Participation rate of eligible youth (1, 4) f a. What proportion of eligible youth enroll in the demonstration?

b. What are the soCio-economic-demographic characteristics of

enrollees?_

c. How do participants compare to'non-participants?

d. What program design and environmental factors explain partici-'

7-14 pation?

B. Short-term educational attainment and school performance (3)

a. What is the impact of Entitlement on the performance ol,

students already enrolled in school and on their retention

in school?

b. What is the impact of Entitlement on the return to high school

or GEaprograms by former high school dropouts, and their

performance in such programs?

C. Short-termimpacts on employment( unemployment, and labor-force

participation of poor youth (8)

a. What is the impact of Entitlement on the employment and unemployment rates of students?-

b. What is the impact of Entitlement on the employment and unemployment rates of former dropouts?

112 -

-1. ) isA., O.

.
A

D. Longer-term impacts on earnings, post- secondary education and

expectations (10)

a. What is the impact of Entitlement on the post-high sChool

earnings of students and comer dropoutg?

b. What is the impact of Entitlement on the college entrance

rates of st1den s and former dropouts?

c. What is tie impact of Entitlement on participant expectations?

E. Effects on the labor market for non-pooryouth (8, 9)

COST ANALYSIS

viz

:"The cost analysis, to be conducted by MDRC, will address issues (2)

and (7) in the legislative mandate. Data will be drawn from the Entitlement

fiscal reporting and information systems that have been designed and are

being managed by MDRC. Data from these systems will be used to determine =

total program costs, costs per participant, and costs per participant-

- year for different subgroups of youth. These unit cost Measures,

combined with the sample survey estimates of program participation for the

different groups of youth, will be used to estimate the cost-of extending , Entitlement to all areas of the country.

IMPLEMENTATION ANALYSIS

The purpose of the implementation analysis is to understand and explain

the programmatic development of the Entitlement demonstration at the sites

and draw lessons concerning the demonstration's operational feasibility, both

generally and under varying site circumsti=ces. Although there is a basic

program design for Entitlement under the management of'CETA prime sponsors,

there is considerable variation across sites in the administrative arrangee
rents established fur operating the program. The implem'entation'analysis

Ls examining the influence of these varying institutional arrangements,

, 1 processes, and decisions upon the program's development, as well as local,

political, social, and economic factors that may also'affect the way in which

Entitlement operates.

The general research on the implementation of Entitlement will cover

a number of key areas which are discussed below (for each area, the correskl, ponding question in the legislation is indicated parenthetically):

A. Program Description (1, 5, 6).

This will be straightforward description of program content and

operations:

a. HoW many. youth were enrolled?

b. What are the characteristics of enrollees?

c. What kind of work did they do?

d. For What kinds of employers?

e. What kind of schools did they attend?.

1. What roles did participating agencies perform?

In addition to providing narrative descriptions of what happens in the

Entitlement demonstration,,this will be A principal data base for other

areas of implementation analysis discussed below.

p

B. Factors Affecting Participation (4)

Participation at each'site will depend on the specific form in which

the -design is implemented as well as underlying lopal conditions and youth

preferences. This component of the research will describe those, aspects

of program 'implementation considered most relevant to the youth's decision

4

114. to participate in Entitlement and, as such, will complement the impact

s'

analysis in explaining the participation rates at the sites. The data collection will focus on recruitment and enrollment procedures, the nature of jobs

- 114 -

1

9 (7:

Ave O
I and the worksjte experience, the educational alternatives available and

the implementation of eligibility monitoring and grievance procedures.,

It is assumed, for example, that participation will be higher where there

is, an aggressive outreach effort and where, for another. example, youths

are offered a variety ofeducational choices.

C. School -Pr*me Sponsor Linkages Under Entitlement (11)

Ts study will document and analyze the roles played by the two

primary agents in jmplementing the two "halves" of the Entitlement offer.

Although school systems and prime sponsors have generally established a

number of working relationships over the years, Entitlement will provide

these two institutions widithe opportunity and necessity to coordinate

work experi nce and school services to a greater extent than has usually

been the case with in- schoolwork prOgrams. In some "cases, the relationship

will be an entirely new one. Since the schools. and prime sponsors have

overlapping but different senses of mission with respect to youth and employment, Entitlement offers the chance to explore the conditions under which

such coordination works effectively and to assess the lessons this partnership provides for carrying out a guaranteed youth job program elsewhere.

+.7 D. The Adaptability Of The,CETA System to Implementing\ Large Job

Guarantee Demonstration (11)

There Will by an effort to document the problems encountered in, prime

sponsors' implementing a job guarantee concurrent with ongoing (and possibly

competing) program responsibilities. Problems may arise, for example, from,

an assumption of resource, scarcity on the part of the prime sponsor derived

from past experience with programs whose limited resources.have required

,careful selection of sub-populations ("significant segments") to receive

program services.. Such an assumption in Entitlement, which is designed to

provide all eligibles with jobslupon request, may lead to overly cautious

- 115 -

a 12J
prOgram implementation. Other problems,may arise from the imposition of

a demonstration prram on the decentralized CETA system where program

discretion for prime sponsors will be more limited than with other CETAprograms.

E. Tier II Innovati8ns,(5,6)

. OF

Most of the implementation analysis will be concentrated on the larger

\ %

Tier I sites. However there will be a systematic documentation of the ' program innovations introduced

, at the Tier II sites. Reporting on the -.

, specialrinnovations will concentrate upon comparison Of experience with the

seek program feature across sites, and upon-lessons concerning the replica-

. tion of those innovations that appear most effe\ctive. Effectivehess will

be considered not only in the ,context of a job guarantee under. these special

arrangements but also in the more general context of exemplary program

services for disadvantaged' youth.

The general implementaiion.research will be supplemented by specia

implementation-studies in three,importhnt areas. .5 t F.

11 ivate Sector Work Experience (5,6)

e

Given the unique wage subsidy provisions of the Entitlement demonstration which allow a wage subsidy of up to 100% to

/

private septor firms that

/ provide program yorksites, Entitlement sponsors may seek to develop a stronger

private sector involvement than has typicallybeen the case with youth.

work experience programs. A special study will examine the role of the

0 privet sector in the-demonstration, the subsidy arrangements that were

utilized,, other arrangements to make participation in the Entitlement

0 program more` attractive to the private sector, and the effects of these-

. arrangements on the volume of private sector,job creation..

- 116, 1 3 0
o G. Rural Programs

The demonstration includes programs in rural areas Of Mississippi,

Steuben County (NY) and Monterey (California). In studies of job creation

and enrollment; there will be particular emphasis on the difficulties in

creating jobs in rural areas and the arrangements that developed between

thelemployers and the schools to deal with

nation problems. 6f particular interest is

type these problems are in relation to

transportation and other coorAiehow differentlinscope and : that may arise in urban, sites.

H. The Quality of Work (10)

."

As part of ibs monitoring responsibilities, MDRC will survey a sample

of program worksites to devea a profile Of the quality of the work

experience at the different. programs. The iniportarice of the Entitlement

worksites lies in their ability to proyide an exemplary work experience that

z.0 develop and and reinforces the positive aspects of work for youth. Therefore, :

,

0 . we hav tentatively identified factors on which a sample of at least

50 worksites for each.Tier I site.,z,yillo.be assessed. Thepe factors,' derived . and adapted from the literature on jobquality of earlier work experience

programs, are:

9

,00 a.. The content of the work provided;

b. TheKlanization and management of the worksite;

c. The 'level and nature .of supervision proyided;°.

A d. Youth perceptions of the. valUe of the assignment; and

e. The viiilue of the work to the sponsoring" agency or firm.,

0

t. The discussion that follows addresses in some detdil the research

design and analysis strategiesInd ),imitations, for responding to the

above issues and questions. These will be discussed. first for the impact

analysis followed by.a discussion concerning( the implementation analysiS.

- 117 -

13

tP
ESIGN ISSUAS

IMPACT ANALYSIS ." Research Design and Analysis Strategies

The key issues and hypotheses identified above for the impact analysis and

coherent time-sequencing of program treatments and effects present the unique

. opportunity to trace the entire process of prqqram enrollment and program41

operation through the receipt of treatments ultimately, td observe,both-

. the intermediate impacts and final impactS of the program. 'This process

and sequence of events comprise the followilg general analytic models of

behavior:

//

/4

1. it model of pre-program behavior.

The purpose of this model is to exploit the baseline survey

co'

data to describe the work/schooling experiences of the sub-

: jects prior to Summer 1978. Particular attention will be

'given ,o part-time versus full-time work experience:

3

e versus jobs held durihg the school year: wages. received,

the nature of reservation wages and their effects on leabor

market behavior; and the nature and extent of time spent in

school.4

2. A model of participation.

The purpose of this Model is to:

41; IP "-

- a. estimate initial demand for .he program;

\

b. model continued enrollment, late enrollment, and.:

atiiition;,and

A c. -provide parameter estimates whereby program participation

can be used to generate an instvntal variable to serve-.

as one control for self-selection bias.

118 - Ism
t is,expec'ted that participation will be examined

separately for Spring 1978; Summer 1978; Fall 1978,

< and thereafter. Of particular interest will be the

.propensity pf Summer enrollees to honor their commitments to attend school during the Fall.

3. A model of intermediate time allocation program impacts.

This model examines laborUpply, time spent on schooldork,

and time-devoted to other activities dUting the secondary

School years. Summer activities and school-year activities

will be considered separately.

h. Labor-supply. Daily, weekly or total program enrollment

b. Schooling (educational production functions):

bl length of additional weeks or months spent in

secondary school

b2 the probabiliti\of secondary school graduation

--b3 change in grade level completed

b4 change in grade point average

b5 change in.claRm standing

b6 change in school attendance:

hours spent in the labor market;-

i i) change in times tardy;

ii) number of days absent, given the required

attenganceperiod;

---& i

, III) change in weekly hours spent in school building; and

# c. Leisure and other activities. _/\\

4---\

e",

-119- 13Jr,

/
4.

0

-- These are intermediate program outputs which become final

program inputs in models which ektimate postsecondary educe-

, tion and labor market impacts. It is important to examine

these. intermediate program impacts separately during the

summer and the'school year. Close attention will be paid

to the quantity and mix of school courses or other types

of training or education that occurs.

4. 'A postsecondary labor supply and employment effects model.

The purpose of this model is to measure the size and statist tical significance of the intermediate program impacts on

final laboiLmarket impacts. Thus, the secondary schooling

or other training and education effects which were intermediate outputs above become final program inputs in the model

of labor supply and labor market effects.

5. A model of the demand for postsecondary education;

The purpose of this model is to complete the estimation of

desired- program final outputs-or-impactsiAgaimithe-secon---

dary schearing or other training or educational intermediate

outputs become final program inputs in this model.

6. It should be noted that the labor supply model and the

postsecondary scho ling demand model can be combined into

a simultaneous equ tion svtem whereby hours spent in the

4. labor market and hours spent in postsecondary schooling

can be jointly estimated.

A variety of statistical and econometric approaches will be used to

perform the above analysis.. It is important to note that while relatively

la" tJ`-.4
AO

complex behavior will be modelled, the salient features ofanalytical figdings

will be syste4atically displayed and discussed with appropriate crosstabulations developed either from a standard analysis of variance approach

or through more complex regression procedures. Complex econometric models

will_.pot' encumber the main body of any report.

The data required for the impact analysis will come from several

sources. First, a household survey was conducted in four Entitlement

sites and-Your control sites.* By means of screening interviews administered

to an estimated 120,000 households, a sample of about 8,000 eligible youth

has been selected and intervie ed, 2/3 from the pilot'sites and 1/3 from

the control sites. The first (baseline) interview was administered in

early 1978 to the eligible youth and their parents (if they'live together);.

the second interview will be administered in early 1979 to youth and their

parents; the third inthrview will be administered only to Youths'in'early

1980. And the fourth and final interview of youths will bp administered

a in 1981.The exact mix of youths among'seniors, juniors, sophomored, and

freshmen has not yet been decided for the second, third apd fourth interviews.

A second data source will be the recods of the high schools and GED

. t programs in the four. Entitlement and four control sites. Finally, data will

be collected through the demonstration's information Idtem designed and

operated by MDRC.

Research Limitations

Perhaps the major technical_ problem in this Study-is that of proper.

estimation of the program participatidn function and the use of this function

to-help overcome the problem of self selection bias.

*The four Entitlemqnt sites are Cincinnati, Denver, Mississippi (selec-.

ted counties) and Baltimore; the four control sites are Louisville, Phoenix,

Mississippi (non-Entitlement counties) and Cleveland.

- 121 -

135
.s Participation breakb down into two components. One must first predict

the p4obability that a person will join the program. Once having joined, one

must predict the extent that one participates. While it is possible to use the immediate benefits of the-program, such as earnings on thsubsidized

job, as prediptors of the extent of partiirtion, it is not possible to use

these benefits. to predict whether one enrolls or not. Rather, independent

or exogenous variables which are not directly related to short term program

benefits must be used to perform this task. Here we come to the problem of

self- selection bias.

Self-gelection bias is essentially an omitted variable problem. In

other words, crucial variables needed to explain the likelihood of participating

and succeeding in the program are missing. To give one example, if "ambition"

,.,..-- is omitted from the analysis to predict ultimate educational attainment, and

401

-1.,, other things equal, more ambitious youth selectIthemselv into the group

of program participants, then the estimate of net prOgram effects will be

biased upwards. at is, the self-selection effects of ambition are

a$%ributed to t effect of program services.

In the Entitlement study, this problem is substantially corrected through

it* the use of a household survey to identify eligible persons in both the program

site and a control site prior to program start-up. site contains

eligibles, but, as figure 1 shows, in the program site, there are both participant and nonparticipant eligibles. Under the assumption that the prog

has..no effect.upon nonparticipant eligibles, we can attribute any differe ce

which is observed between-tlia-group and the controls to the fact that

group consists of the individuals who did not select themselves into.the

program. When the participant eligibles are joined with the controls to form

a data ft for analysis and analysis is employed in which a dummy variable

identifies "experimentals" versus "controls," the measure estimated for this

- 122 -

1
dummy variable is the sum of the program effect and a selection effect.

If we knew the magnitude of the selection effect, we could subtract it out

and thereby find the true program effect, But we can measure the,selection

. effect by creating a different pooled sample, one containing nonparticipant

eligibles and eligible controls, and estimating program effects for them,

with the rral groups identified by a dummy variable. Then, the measure

. associated, ith the dummy variable in this calculation is wholly attributable

to selection.

Figure 1

Program Control Site

Non- Participant

Participant Eligibles

Eligibles

Eligible

Controls

Dummy Variable Coefficient

E. Treatment Effect + Selection Effect

Dummy Variable Coefficient= Selection Effect

The result is an unbiased estimate of program impact.

0.1 Another limitation to be noted is that rigorous measurement of the impact

of Entitlement job opportunities on other job opportunities for youth, including

non-poor youth (question 9), would require that a random sample of all youth

in the selected areas be surveyed. It was determined that the costs of this

sample would be prohibitive, more than doubling the sample size and cost.

The sample size was determined, therefo-re, by the need to answer with sufficient

- confidence the questions of short and.long-term impacts only for poor youth.

Other data sources will have tca be utilized to develop an approximate estimate

of Entitlement's impact upon non-poor youth employment, and these estimates,

will not permit the same level of statistical confidence which would be

*See page 3, above.'

- 123 -

4
possible, through surveys.

IMPLEMENTATION ANALYSIS

4

Research Design and Analysis Stfategies

implementation analysis addresses the behavior of key "actors"

(institutions, agencies, firMs) who participate in the delivery of Entitlement

services to participants, and the effects of o er local circumstances on the-

, program. In order to describe and account for the behavior of service

deliverers, it is necessary to undeXstand major features f their structures,

missions and organizational stakes in Entitlement,- previous program operating

erience, and pre;iious working relationships with each other. The way in

whi these factors come together for an organization will help to account

for the way in which the organization carries out its role in Entitlement

and the extent to which the organization's members perceive an incentive to

cooperate with the other actors delivering Entitlement services.

'A dynamic program implementatibn process in unique local settings,

particularly when this process involves many different actors,with different

histories and administrative arrangements in the several Entitlement communities,

. does not lend itself to highly quantitative measurement:- It is important to

derive an understanding of Entitlement processes through the collection of

data about program decisions and actions, alchlg the range of qualitative

va;iables mentioned above, from the viewpoint of all participating actors, and

where possible,, by "triangulating" the observations of more than one outside

observer. 13 ding such an underst4ding is a holistic process, and multiple

actor viewpoints do not constitute redundancy. The data sources which are

being utilized in the implementation analysis reflect this conception.

The sources of qualitative program data include outstationed field

monitors assigned to the Tier I sites, MDRC central field staff who periodically

- , visit the sites as part of thei: oversight function, and consultants to MDRC.

4 - 124 - %-),S
All of these have responsiblity for documenting and assessingvarioOts aspects

of program implementation through periodic,structu research reports. The

outstationed field monitors responsible for desibing and assessing basic

program activities such as recruitment and enrollment, schooling, job creation,

and others in a series of structured research reports. In addition they will

4 carry out the field research for the quality of work study (see below).

Central MDRC field staff, and a group of consultants under subcontract to

MDRC to.monitor five Tier II sites, contribute. to' the research through sharing

of their field visit reports and through periodic debriefing with the research

4

staff. In addition, they contribute through structured research reports on

the management and coordination of the demonstration by prime sponsors and

their managing agents.

To enable a more thorough analytical assessment of the development of'

Entitlame ii at the Tier I sites, the implementation ari4lysis design also calls

for a series of interviews with the prime sponsor and other program agents

during the course of the demonstiation. These interviews, carried out by MDRC

consultants during a week's field visit, are structured to address background

factors at the sites that may explain the program's development and content.

Such factors incl e local political conditions, the quality of the relationship among the arious program agents, their perceptions of the Program and

of its problems nd successes,..and the rationales behind administrative

decisions and o anizational arrangements.

Finally, quantitative data on progrdin participants and content is being

provided by MDRC's Entitlement' Information System on a monthly and quarterly

basis with special reports produced as needed. This system, which in addition

to informing the implementation analysis serves other MDRC units and reporting

requirements, produces data on the number of eniiallees, their demographic

4 and employment backgrounds, their source of referral, school enrollment data
L

by of school and type of curriculum, participant activity in terms of

initial job or training assignment, termination and re-entry where applicable,

characteristics of the worksite or training sponsor's; monthly levels of

youth participation by types of joissAkraining, and wage rates,

and total wages paid to individual:participants%

Research Limitations

One research limitation that should be noted is that he quality-of work

study will not be a highly rigoroUS and quantitative evaluation of the work

experience provided to participating youths. 'Due to the part-time nature

of the majority of the work prOvided during Entitlement, the fact that it

is a you program, designed to primirily provide a good work experience

O rather than to impart specific °Vocational skills, and other cost and data

limitations; a qualitative and,extensive worksite assessment strategy, is

planne\ It is anticipated that from 350 to 500 Tier I worksites will be

assessed by the Tier I monitors u&ing a structured assessment instrument.

Another potential. limitation is the reliance of the Entitlement

Information System on data recorded and processed by local prime sponsor staff.

During program start-up, there lave been numerous and serious delays in the

delivery of data by prime sponsors to MDRC's,central infotWagli.on system staff.

While the problems have largely been alleviated and data from most of the sites

are now flowing in a.satisfactory:Way, the system obViously still remains ° vulnerable to unforeseen difficulties.

.- 126 -

.-//
w

TIMETABLE OF RESEARCH PRODUCTS

The. Entitlement demonstration and research activities are still in

an early stage: the demonstration began enrollment in late spring 1978,

the baseline survey for the impact analyv4.5 was completed in the summer of

1978, and a variety of interim reports and interviews on\V!p start-up period

have only recently. been completed. The following research reports aretentatively schsduled for the dates indicated:

Impact Analysis Reports

Baseline Report January 1979

Participation Study, January 1980

Report on In-program Impacts February 1980

Report on Post-prgrn Impacts January 1982

Implementation Analysis Reports . f General Implementation Report Janury 1980

.!----The Quality of Work in Entitlement January 1980

The Role of the Private Sector

in Entitlement October 1979

Entitlement to Rural.Areas February 1980

In addition, MDRC is preparing a report on the early program experience

as specified in the YEDPA (section 329).

1 0

141

- it? -
POLICY IMPLICATIONS

The paential policy implications of the research obviously follow

from the questions and issues articulated by Congress and the Office of

Youth Programs as preSented earlier in the paper and as explicated above.

Among the major issues to be addressed are:

1. The operational meaning And feasibility' of operating a guaranteed

-161bs program under varying local circumstances and administrative

arrangements.

2. The impact of such a program, when school-conditioned, on the

school performance and labor market tehavior of participating

youths both during their participation and beyond it.

3. The costs of operating such a pso7ram and the potential cost

should it be extended.

4. The ability of)rime sponsors to create jobs in the private

sector under wagesubsidies ranging up to 100 percent. . . 5. The quality of work. that can be provided in-a large scale, rapidly

implemehted work experience program for youth.

.142,

128

6%.
s

I

REPLICATION * VALUATION * IMPACT

THE VENTURES IN COMMUNITY IMPROVEMENT PROJECT WICI)

1 A YCCIP DEMOSTRATION AID RESEARCH PROJECT

SUPPORTED BY THE OFFICE OF YOUTH PROGRAMS

UNITED STATES. DEPARTMENT OF LABOR

ti

Paper''prepared by Johri'M.Kelley

Corporation for Public/Private Ventures

Philadelphia, PennsylvaDia

For presentation at the Office of Youth

Programs' Knowledge Development

Projects' Conference.

October 5, l978

, - 129 -

1 4 rj
41.

INTRODUCT1QN TO VICI

I. OVERALL PURPOSE OF THE DEMONSTRATION RESEARCH

II. SPECIFIC RESEARGHQUESTIONS,-ISSUES, AND APPROACHES

INDEX

1

II.1 THE FEASIBILITY,AND EFFECTIVENESS OF A REPLICATING METHODOLOGY

11.2 TO DEVELOP AND REFINE A,"WORK EVALUATION" METHODOLOGY

11.3 TO COMPARE POST-PROGRAM EFFECTIVENESS

III. TIMETABLE OF RESEARCH ACTIVITIES

IV. KNOWN, ANTICIPATED, OR LIKELY RESEARCH LIMITATIONS

ANTICIPATED RESEARCH FINDINGS

WHAT DOES ALL THIS MEAN TO YOUTH POLICY PRIORITIES?

VII. WHAT.DOES ALL HIS MEAN TO RESEARCH?

APPENDIX A ESSENTIAL AND REQUIR) MODEL PROGRAM ELEMENTS ' APPENDIX B.THE,CPPV DESIGN

"

I.

- 130 -

, / S
1 Alb I. :` OVERALL PURPOSE OF -THE DEMONSTRATION RESEARCH.

I As noted in the Knowledge DeIelopMent'Plan for YEDPA, "the underlying

objectives of this demonstration project are: I %- (a) to demonstrate and. est the feasibility_ and effectiveness

. , of.a replicating methodology itself (i.e., hoW and Under

what conditions can the prOgrat model be adeqately replicated in various communities.)

I

.

. . 1 ,---- (b) to develop and refine It "work evaluation " methodology which

may eventually be incorporated into the fort'la-funded community

Improvement type efforts.

(c) to comparethe post-program experiences of youth participating in the -replicated projects (which will mainly provide job experience) with those for.a comparable group of

young people of similar backgrounds who have participated

in other manpower programs (which have emphasized classroom

training, manpower and support services) in the same localities." (p.17)

SPECIFIC RESEARCH 9UESTIONS, ISSUES, AND APPROACHES

II.1- The Feasibilit and Effectiveness of a Re licatin Methodolo

Consistent with the Knowledge Devdlopment Plan, CPPV conducted "a review

of both appropriate post programs and designs of projects financed through the

formula allocation for,Community Improvement" (p.17) and selected a model program, be Emergency llome Repair program (EHR) of Portland, Oregon. This Aro-

, gram utilized in-school and out-of-school youth to complete home repairs (Or

poor, elderly, add handicapped homeowners. In addition to meeting basiCYCCIP

criteria (e.g., the target population and the production of-tOngible, long-lasting community improvements), the EH program boasted several aspects that were

especially effective:

Close union involvement: ERR youths work in small teams. Each

team is supervised by a crew chief who is a journeyman member of

the local carpenter's union. This aspect not only appegred con-,

ducive to good skills training, but- was deemedas\a positive asset in subsequently motivating youths to seek entree into the build- ' ink; tirades" unions.

An effecticommunity aaencY_ linkage system: EHR, in its planning

and operatio effectively involved a number of local agencies

(schools, unions, courts, and a-variety of municipal in

order to:

provide sound program management

refer ample and interested youth from the target population

generate a titable inventory of work projects in a timely

manner , - 131 -
obtain building and licensing permits/approvals

.-N,provide post-program employment, training and edudgion,

The successful transfer of the EHR model to diverpe jurisdictions mu.ot be

';based on a clear distinction between essential, characteristics of the model and

'incidental features of particular program operations and particular jurisdictions.

For examp while direct union invd1vement via the designation of journeymen

crew chi as deemed essential, it was nottdeamed essential that the one

particular union need play the lead role in all replication efforts. This distinction between the essential and the incidental is critical, because circumstances vary from one locale to another. The process bf replication must,thereforemaintain the essential principles integral to program success and yet be

frexible and sensitive to the needs and conditions of divev communities.

4 Based on an intense analygis, CPPV distilled nine-(9) essential program

elements which sites who participated in the demonstration are mandated to

incorporate. Thege elements are listedin Appendix A.

d' The efficency of the VICI replication process will be assessed primarily , by means of process evaluation

.

. , which will yield a documentary analysis of

issues such as: .

. ,

. .

. The4xtent,;to,which linkages requi=ted to implement the model are lie-existent in ibcal-communities :aftd/or the processes involved

in creating, th6Se linkages loc'ally _ . , ., -, fr

. . a1 replication"

,., The "politics of,1.4eplication" in local areas-i.e., the extent

'to which idic/nOraticiocal factors (quality of leadership, prior

a history oflinter-insOtaional cooperation, local politics of

si CETA, etc.) enhanceNcor imOdepuctessful replication.

. The speed with which local sponsors,mountprograms, their ability

, to attract, and retain the fargei'popula in the program; And

their ability to perform valuable WOikAtimely basis.

, The extent and natureofthequalityinter-institutional linkages exhibited during program operation;

Assessment of ways in which 'local economic c

climate, and city scale impact the prograM.

itions, political

Ways in which local preferedce inittaiive-br conditions produce

local innovations and augmentation t7 the.model;

Evidence of "spin-offs" from succ's program implementation.

.

. To accomplish this process or documentary re arch, cPPV has retained the

services of two (2) independent evaluators whohave demonstrated pompetenca and

experience in this type of undertaking.

- 132 -

31-

,- 146

.

be
0

-11.2 To Develop and Refine a "work Evaluation"

Ali important outcome of the VICI demonstration will be the production of

tan e and valuable physical improvement to local communities. The growth of

public employment programs has added urgency to developing reliable measures'

_of the dollar_value of work performed. In order to address the effect of VICI

in monetary terms, a practical and workable methodology must be developed, To

date most work valuation procedures have relied upon expensive and time consuming fiscal audits or have utilized cost /effectiveness formulae involving adomittedly arbitrary and subjective components.

CPPV, has designed and will evaluate, a fairly straightforward method for

determining the value of VICI work products. For each VICI work project, local

estimators will compute an estimate of the cost that an alternative supplier,

specifically a typical private contractor, would charge for producing the same

product. The price that'a, private contractor would charge will be equated with

"dollar value" of a VICI product with possible adjustments made to reflect differences in the'quality of outputs.

)CPPV, through its subcontractor, RBS,

** Will select an independent estimator who will visit VICI sites and compute. estimated alternative supplier costs

for approximately 200 randomly selected work projects. These independent'estimates will be compared to local estimators, figures in order to determine the,

degree of agreement.or concordance. If systematic error is apparent with re-

, spect to local estimates, correction factors will be used to adjust-figures

across all estimates. Assuming that there is acceptable agreement between local

estimators and independent estimators, dollar values will be assigned to VICI

uojects, The dollar values will reflect what the same work product would cost

if it were supplied by a private contractor. Taxonomies,will also b!! constructed

to explora'whether dollar value differs significantly across various types of

Work projects (eg., rehabilitation vs. repair).

Work Evaluation Methodology" in this context has been operationally defined

as attributing a dollar value to the outputs produced by VICI participants.

"Work valuation" is perhaps a more accurate term rn describing the nature,

of this ,;ask. ** CPPV has selected Research for setter Schools (RBS) to conduct the quantitative portion of the VICI research. 1

4

- 13 31-4 71

A.
11.3' To Compare. the Post - Program Effectiveness of Youth Participantink in

the' Replicated Projects with Those for a Comparable Group of Young

People of SimilarBackarounds Who Have Participated in Other Manpower

Prostrams

The key question to be addressed in thl s: "Does participation

in VICI lead to getting a better job than participation in other manpower

, programs?" Clues to this answer;will be obtained by simply comparing VICI teriiiaation data with the termination data gleaned from other YCCIP formula programs,

and HUD demonstrations that may be operating at each locality. CPPV, has asked

RBS to gather existing data from other YCCIP and HUD programs that resemble

VICI-in each site, and to present this data in a desEriptive, summary fashion

in order to compare VICI and other programs along such dimensions as:

youth 'characteristics

nature of work performed

. strength and extent of linkage systems

. retention rates Ny

. cost per positive termination

C.. . post-program placements

In 2.dition, it is anticipated that this inter-program comparison effeq

will provizie information that will aid DOL in assessing the effectivenEss of

different kLnds of deliVery systems in mounting prograliis of this type. Specifica115, co7paratidedelivery system data will be presentepelative to the use

of:

. an intermedtate unit (CPPV)

community-based agencies (HUD)

. prit,e sponsors

4.\

It was recognized, however, t in order to draw confident statistical

inferences, about the impaet of 'CI on youth, a research design would have to

be developed that would better assure equivalence between VICI and comparison

youth.

Indeed, unless there Were an equivalent group of yetng persons with whom

to compare VICI paticipants, it would be impossible to confidently attribute

"outcomes" to participation in VICI since a number of other plausible explanations c.ould account for outcome patterns (e.g., local economic conditions, the

type of yo involved in VICI.) However, the generation of an equivalent comparison oup has been a problem which has continuously plagued manpower research,

, ond p aps all social services rodsearch. In addition to programmatic problems,

scriou, .thical problems arise when one attempts to launch a "social eXperinent."

CPPV.sraff, as well as: its VICI research subcontractors (RBS /ARB) * and private

(ow,ultants, worked for veral months ip order- to come up with a desiT,Mh1Lh

would satisfy research criteria as well as ethical and programmatic standards. ' * tlTV has approved 1:1;S' selcction of Associates for Research in Behat,ior (ARB)

1,, A 5u1icontractor to perform the follow-up interviews for VICI research, lo

maior in tlu ,onpift-t rized infat..,Lton base, and to offer Ongoing input reAarding the VICI r.watLh.

- 134
Consistent with DOL instructions, CPPV presented the design to the appropriate representatives from each VICI site as an option, not an order. Seven

(7) of the eight (8) approved sites opted to adopt the "CPPV design" as presented

or with minor variations. Our design is explicitly described in Appendix B.

III., TIMEABLE OF RESEARCH ACTIVITIES

More than -half of the fight lay7Inci programs have-begun enrolling candidates. All sites should be operational by mid-October, 1978. CPPV has designed a comprehensive set of uniform data collection forms and manuals for

use in all VICI programs. Data from this information base, as well as from

the process evaluation, will be incorporated into brief monthly progress re-'

ports and extensive interim reports. The final research results will be available in May, 1980.

IV. KNOWN, ANTICIPATED,kLIKELY RESEARCH LIMITATIONS

In addition to those potential limitations, which all research efforts

encounter (e.g., accurate and complete filling out of forms, being able to

find and interview sufficient youth in the post-program follow-up), thb main

limitations of this study may occur around Renerating an equivalent comparison

group. The design_that was earlier mentioned assumed that VICI programs could

recruit and screen almost three-hundred (300) youths within about a four-month

period. Preliminary feedback indicates that problems may occur in finding

stIfficienx. youths. In addition, the added screening burden may cause one

or two sites to abandon this design, beebuse they lack sufficient intake

/resources.

Shotild problems like these arise and scuttle the design in certain sites,

CPPV'may sWt its follow-up resources to tracking yodths who have'participated

in other .ianpower programs without going through VICI screening. Admittedly,

such a fal.back position jeopardizes the equivalence or comparability between

groups since othex programs may have screening processes or programmatic goals

which differ from VICI. However, comparisons of this nature should yield clues

as to the impact of VICI compared with other manpower programs.

V. ANTICIPATED RESEARCH FINDINGS

SinCe data collection is in its embryonic stage, few directions or trend*.

can be discerned at this time, which would give evidence for anticipated findings. One critical area wherein a specific finding is apparent relates to the

replicating process. As noted above, all VICI applicants Were required td meet

a set of fairly arduous program criteria. In most sites, ibis ca led for the

forging of new inter-agency cooperations. Union involvement, and curing supplementary funding, were especially viewed as'potential stumbling ocks. How-

, ever, of the acceptable applicants (13), * ,eight (8) sites have met all criteria,

and one (1) site is still in the running. This lends strong evidence to the

feasibility of replicating even those model 'programs which make strong demands

on local linkages, procedures, and other resources can be replicated. CPPV is

paying close attention to the five (5) programs that could not meet requirements

in order to document obstacles and formulate strategies that might enhance replication.

DOL eliminated two (2) of the original applicants for reasons not directly

related to VICI.

135
j

VI. WHAT DOES THIS ALL-MEAN TO YOUTH POLICY PRIORITIES?

If the VICI research yields certain findings it is possible'that these

could be the basis for several policy.and procedural changes. For example,

little evidence has been gathered on program replication. The term keeps crop-

:ping-TT and inherently seems to make sense. However, to date, there is a paucity of hard data in this area: Should.VICI.demonstrate that replication is

not only viable, but economically practtcal.and effective in aiding youth enter

thg mainstream of employment, it is possible that planned replication may gain

increased propinence as a program planning and implementatipn policy.

. . The research that addresses alvernative delivery systems (prime sponsor,

'intermediateylts, and community based agencies) should yield preliminary

evidence es--to the efficacy of each variation. On the policy level, such evidence

could influence future program implementation strategies.

On the procedural level, work valuation has been a problem which has befuddled economists, program planners And,policy makers alike. CPPV's methodofogy, should it prove valid, would provide a long-needed mechanismithat would

yield extremely valuable information to policy-makers, planners, and administrators for purposes of program development.

VII. WHAT DOES THIS ALL MEAN- -10 RESEARCH?

' In terms of adding to 'fast-growing body Of research knowledge, the VICI

praject could provide substantial offerings. First, VICI will constitute one

doof the very few replication efforts that was well documented and evaluated.

This, in itself, will contribute significantly to a better understanding of the

replicating process. Second, the "CPPV design" irs-one that has not, to our

knowledge, appeared in the literature. Yet, given the real world of the progrzlm Oerator, this design seems to strike an acceptable compromise among the

deans of research methodology, ethical considerations, and program operations.

The design may be especially useful in programs within the human services sector,

and could possibly constitute an advancement in action-oriented research. . Thirdly, and perhaps most ATortantly,.the VICI data collection forms and

thg "CPPV design" were riot conceived in VACUO by a "bunch of researchers who

dont.t know what running a program is all about." Both the data collection forms

and the design resulted from intense interaction and numerous meetings among

CPPV research staff and field representatives, DOL national and regi nal staff,

prime sponsor staff, front-line program staff and youth. To be sure this made

the design process more time consuming and at times somewhat frustr ing. However, CP#5-believes that such a process served to Rroduce not only better products, but tended to engender genuine feelings of involvement and commitment

to the research efforts of VICI. CPPV.Kill document this process and explore

its effects. From a planning perspective, it is possible that this process can

dIfero model to fgture research efforts which like VICI are dependent upon the

cooperation, effort, and commitment of literally hundreds of people if the data

is to be property defined, recorded, transmitted, stored, analyzed and promulgated.

, - 136

4
APPENIAX A

ESSENTIAL AND REQUIRED MODEL PROGRAM ELEMLNer

1. Participants 16-19 years old who are unemployed, underemployed, economically

disadvantaged and confronted by severe difficulties in obtaining access to

jobs.

oi

2. Work projects which provide both needed physical community improvements and

varied work and training for participants. The work projects must be completable within the life of the demonstration; and the work must be of a kind

whiclymuld not be routinely done in the absence of the program.

.3, Project participants recruited frowthe geas where youth work activity will

take place.

4. Well defined work crews led by skilled supervisors and a supervisor-participant ratio small enough'to permit skills training.

5.- Linkages of public and private agencies with clear assignment of (and agreement tib) roles and responsibilities Eo include:

(a) A local management agency with the-j.experience and capacity to conduct

and manage the program.

. (b) A youth referral agency with demonstrated access to and experience in

working with youth who are the,target population for this program.

14

(c) A ybrk providing agency with the dempnstrated capaqity to provide a

suitable, inventory of projects and work orders in a timely and continuous fashion. .

4

(d) Labor unions and trade organizations'cooperating in the referral of

journeymen instructors and provision of apprenticeship training credit

for youth who work in the program.

(erEducational institutions which can. provide yOuth participants with opportunities to receive a G.E.D.or enroll in other forms of continuing

education.

6. Provision for obtaining licensing approval and/or permits to undertake physical improvements work, inspection of completed work, and valuation of'work

performed. . 7. Provisions for post program training, education or employment.

8 A commitment to providesthe necessary data and information to meet the research iequirements of the demonstration.

. '9. Local funding to !mppleent-the amount provided by DOL, particularly for

building materials and supplies.

1371T
I,

APPENDIX B.

THE CPOOJDESIGN

"\-

The proposed design calls for 60 participant slots. In addition rthe VICI

structure calls for two cohorts of participants at each site. The first sixty

will remain in VICI for up to twelve months but no more. A second cohort or

wave of youth will be enrolled after the first wave has completed VICI. Thus,

there will be at least 120 (60'4- 60) participants. Assuming a 25% or so dropout rate, it is predicted that there will bet about 150 (60 + 60 + 30) VICI participant's per site.

Because of the urgency involved in getting VICI up and running,'the "CPPV

design" opts not to deli), intake until a sufficient pool is built and applicants

can be randomly selected for VICI (i.e., the classical experiment) or selected

via other procedures from such a pool (i.e., quantifiable scores/ratings.)

Building up a large enough pool before VICI start-up would simply push back the

schedule inordinately. Therefore, the "CPPV design" calls for progrAms selecting the first youth cohort (n = qpi) and several waiting-list youth (between'10

and 30) according to the program own procedures and own time frame.

- lig - After the first VICI cohort is selected along with a small waiting list, the

"CPPV design" calls for each program to contihlue intensive intake for several

months until an additional 220 or so youths have been found who meet all VICI

criteria (both DOL's and the local program's criteria.) These youths will have

undergone the e!:act screening as thefirst.cohort and will beapsigned, by

lottery, a place on the VICI waiting list. It is predictable that the first

60 of these 220 youths will be selected fOr VICI before the demonstration is

over.- It is equally predictable that those youths who are placed towards the

end of the waiting list will not get into VICI during the demonstration period.

Approximations of the chances of getting into VICI can be made for those youths

who are toward the iddle of the waiting list. The "CPPV design" calls for

honestly and prompt y informing the youth of

ItiE hip/her odds of getting into VICI

as soon as the you has a waiting list number. 1 This decision, namely, to

be totally honest with youthst is called for in order to avoid undue raising

of the youth's expectations. For the youth who has little chance of getting

into VICI, or for the youth who may well get into VICI but chooses not to wait,

full effort will be made to place the youth in another,appropriate employment/

training program as soon as possible. CPPV expects that a sizeable number of

youths will be placed in ongoing programs. It bears noting that the CPPV

degiv neither directly nor impliCitly encourages the formation of a "no treatment control group" whose waiting list position in icates bt e chance o I entry. There must be a real cleance, not just a sham chance, of receiving alternative manpower services. To better insure this option, CPPV has mandated

that no site wherein there is not sufficient potential to place youths from the

waiting.list into alternative CETA programs'Will be permitted to use the "CPPV

design."-

CPPV,-through its research subcontractors, RBS and ARB, would follow-up eighty

(80) VICI participants and an equal amount of waiting listorouths who did not

get into VICI. Follow-up will be done/ /at a standardized die. For example,

,"" - 138 _ 1

4,
APPENDIX B (Con'd)

three months after a'youth terminates VICI, both this youth and a youth selected by chance from the waiting list will be followed up. It is foreseen that

this design will yield a comparison-group that is sizeable enough for conducting statistical analysis and boasting a level of homogeneity with VICI youths

That is adequate for drawing confident conclusions. Homogeneity between VICI

And comparison groups Will be double-checked by examining scores from a standardized,test of ability which will be administered to all VICI and waiting

list youths.

s.

153

- 139
Itt

YEDPA RESEARCH VIA INTERAGENCY AGREEMENT (AND DISAGREEMENT):

4 STUDY OF CONFLICTING OBJECTIVES1

Roy E. Feldman and Jay Ostrower

Papet presentedo U.S. Department of Labor, Office of Youth

Programs, Knowledge Development Projects Conference, Sheraton

Conference Center, Rest, Virginia, October 5 - 6, 1978.

The research described in this paper is being conducted under

HUD Grant No. H-2881-RG.

1 5.e:t

- 140 -
Qa. 41111, Three agencies might appear to have some stake in this

YCCIP demonstration project: DOL. and HUD because this demonstration project is being formally operated inder an. interagency

agreement, and CSA because it has provided money for the opera-

_ tion of the project. According to the DOL Knowledge Development

Plan the evaluation of this demonstration project is to determing

whether: "(a) CSC's can more effectively link youth Community.

Improvement efforts to other funding sources and, if so,

whether there is a multiplier effect reflected in project out-- `

comes: (b) CDC's can facilitate particular productivity by

youngsters which is not generated by formula funded projects:

and (c) the nature and value of project accomplishments which

distinguish the CDC efforts from those of formula- funded projects.

,2

A YEDPA document dated 9/11/78 changes the second item above to

"value of community improvement work" and the thil.d item above

to "the impact of.these projects on communities and participants

in comparison with that for YCCIP formula-funded projects at

CETA prime sponsors.

HUD's objectives differed from those of DOL. They were to

give priority to (a) impact on the CDC, (b) impact on the community,

and (c) impact on the participants. 4 CSA's objectives are not

clear. CSA representaties never attended any of the briefing

meetings in Washington which were attended by the evalUators and

HUD and DOL representative. Because of the structure of the

evaluator's contract with HUD (and riot with DOL or CSA), HUD

determined the basic priorities of the evaluation.

a 2. U.S. DOL Employment Training Administration, Office of Youth

Programs, "A Knowledge Development Plan...", 1977.

3. "YEDPA Knowledgement Development Project Funding as of 9/11/78."

4. HUD Grant No. H 2881-RG Contract
4

The project design and methods of analysis are in two parts.

First, there are three site visits of about two weeks each to

five of 'the ten demostration projects- These are to observe

the programs, View the work done,,interview operators of both

HUD and formula-funded programs, interview community ,leaders

and youth participants. The second part of th s udy is to

nalyze data from YCCIP intake forms, ter ina ion forms and a

small sample of participant int ewson_their-perceptions of

the programs. We hope to compare ata for both HUD and formulafunded YCCIPs but it is not clear whether we shall get access

to All appropriate formula-funded YCCIPs because they are under

the auspices of DOL whereas the evaluation is under the auspices

of HUD..

Among the major limitations of the study are

the missing items on the YCCIP application and termination forms.

Because we are funded by HUD, we are unable to offer any incentives

to the DOL formula-funded'YCCIPs to cooperate'or to improve data

quality. Because HUD/YCCIP operators know we are reporting to

HUD." they have generally been very cooperative.

On the positive side, the amount of..time we were able

to allocate'to field research at each site has meant that we

can arrive at a detailed description of each of the five sites

we are visiting and adequately, describe the impact of these

demonstration projects on the CDC, the community and the participants.'

Nevertheless we have come to come clear but still tentative

conclusions within the limitationsIdiscussed. 5 .

5. The dmonostration projects included in our site visits were.;

South Bronx, N.Y., San Antonio, Atlanta, LA., Mississippi, and

Boston. CF., J. Ostrower and R. Feldman, "Evaluation of the HUD/YCCIP

Demostration Projects, First Quartly Report", Boston University

June, 1978.

142

156
1. The programs are working and in most cities useful work

is being completed. In some cities the programs are well organized,

work is going smoothly, and participants are pleased. In most

others, work is going relatively well but the programs are having'

some organizational problems.

2. There seems to'be a trade-off between institutional

impact on the one had and participant and community impact on the

otheL The greatest institutional impact of the HUD grants

. will be oh those organizations that havethe least experie ce

and consequently have less impact on the community and part11\'/ cipa t. Conversely, those organization wii high levels of

perience and a sophisticated institutioha structure . will be more able to haye a significant community impact (e.g.,

number employed and skill developemnt). These distinctions

lead to the suggestion that the administration of the program

should be resnonsive to the differences. Potential"fbr ex- . pansion seems directly related to previous organizational experience. Those organizations with the leacrgtonstruction.exper.ience need to do the most planning.

3. Those programs that are faring best have had construction experience; housing experience, and are able to combine

both the youth and work aspects of the program with little

difficulty.

4. The programs th t have the most impact on the organization are those that either change the direction of the organization or substantially increase the organiations resaurces.-

5. Those programs that seem to have the most short term

impacts on their communities are those that have decided to

concentrate resources in the most limited geographical areas.,

6. As expected, among the most important participant

impacts of the HUD project are the receipt of money by the participants and the work experience for these youth, many of whom

have never' been employed. In addition, many of these jobs demanded,

the kind of work performance comparable to unsubsidized employ7-4135-:
ment and provided signi. icant skill training.

too.

7. The job supervi rs are particularly influential in

working.with youth: Union journeymen in particular are highly

skilled and seem to serve as economic role models. Examples'

can be found_of supervisors Who helped participants get unsubsidized jobs and they have also helped a limited number of participants to enter the unions.,

8. Obviously, programs Where participants are engaged in

complex work (e.g., housing Construction and rehabilitation)

are providing greater opportunites for skill acquisition than

those with relatively simple tasks (e.g., lot clearance and

building demolition) At this point impact seems significant

where par cipants are engaged in complex tasks and craftsman/

participant r tio is not high. Where this ratio has been

observed to be high, e.g., 1:10, a variety of problems seem

evident. These include: a) slower skill acquisition, b) more

discipline problems on the job, and c) ,% lower likelihood of the

supervisor providing an effective role model.

/ '9. Althoughit is still tooearly'fordefinitiove comparisons

with he formula funded YCCIPs, the following observations can

be made: a) In general, the HUD/YCCIPs are substantially larger

<1.).an the comparison groups. The larger scale of these projects

and more limited definition of " commnity" predisposes questions

abouth community impact in favor of the HUD/YCCIPs. b) Where we

have been able to make site visits at the comparison ,projects,

it appears that the HUD/YCCIPs are having a larger community

impact. Obviously, these conclusions are still based upon interim

observations and interviews in the community. They will be pursued

at greqrterjength during future site visits:

10. Given the current state of our research., we believe it

is important to develop ideas on how neighborhood units can be

integrated into an overall employment strategy.

One important purpose of an interagency demonstration is

probably to permit the silbtantive priorities of the second

- 144 -
agency to impinge on the demonstration project. It is probably

reasonable, therefore that the evaluators take 'serious account

of these priorities and depart somewhat from the ojectives Of

the primary agency (DOL).-On 'the other hand, communication

between agencies With regard-to DOLs Knowledge Development Plan

could have been better.

Estimates of projecNoperators of the likelihood of project

renewal probably has an important effect on their administration

of the projects. Their belief that the demonstration will end

in six, four, or two months affects the programs during their

critical final phases. Basic activities of the programs-such

as finding new work, participant and staff morale, job development and departure of staff are affected by preceptions of when

the project will terminate. This is a critical input,for a

demonstration project designed to last only one year..These

deponsKations are probably of too short a duration to answer

many of the questions of both HUD and DOL. In'the absence of

factural information, demonstration project directors ,have already

made judgements of when the project is likely to terminate and

are acting according 1Y. This factor needs serious consideration

in the design 9f program duration and in the design of evaluations:

A
I

4

YOUTHWORK, INCORPORATED

Learning By Trying

Identifying the Strengths of Exemplary,In-School

Youth Vmployment Programs

A

A paper p esented'at the:

Knowledge Developme t Pro'ject Conference

of the

Department of Labor

4 Harvey Pressman

Youthwork, Inc.

October 5, 1978

k

80515th Str et,.N.W. Suite 705 Washington, D.C:

146 -

4

202) 347-2900

4.

V

t
For many 'low-income teenagers, the gap between school and work

seems to be widening. The dumber of years between leaving school and

finding a full-time, "adult" job iincreasing. The kinds of preparation provided by the school for the "world .of work" seem to be

getting even less relevant. Very little is really known about how

schools can effectively combine school...and work experiences to

facilitate future .employability. Even less is known about which

particular-in-school models and treatments work. ell, and why.

The allepartmerit of Labor, as a means of.encouraging'innovaticiri

in school-related youth employment programs, has launched a special

exemplary-school demonstration project. This demonstration project

is identfying some 55-60 program models in four primary areas of

focus:

A (1) Career Information, Guidance, and Job Seeking Skills, (2)

Academic Credit for Work,Experi:ence,. (3) Expanded Private Sector

Involvement, and (4) Job Creation through Youth-Operated Projects.

The goal in narrowing the demonstration to primary areas of fOcus

is: (1)' to identify key areas in which more knowledge is needed-,

then (2) yto support enough program, models in each area, and (3) to

learn a lof`about each area from an analysis of variety of related

program models.

TO help the Department of Labor select appropriate program

models, provide technical assistance to funded projects, coordinate

what can be learned from the demOnstration, and prepare "how-to-d4-it."

guides for in-school employmen related efforts, a non-profit,

.intermediary corporation was fotmed in the winter of 1978.. This

corporation, called Youthwork, Inc., has undertaken the responsibility

for insuring that' we learn as much as possible from what the demonstration

projects are trying.

. ,

. , Youthwork's special mandate from the Labor Departme Is to focus

attention on the employment problems of inschool .youth,' on the

capacities of educational institutions toihddress these problems,

and on critical issues emerging from the evolving relationship

betWeen,the educational and CETA systems. With regard to the probleMs

of in-school youth, Congress wants to know about (I) the feasibility

of, and procedures for, implementing, expanding, and a*pting

exemplary programs for in-school youth slid returning dropouts, ) the number and characteristics of youth served by these programs

and-of those rejected, (3) tlye- kinds of experiences, training, and

other services being provided to youth, and the kinds of work being

performed by them, (4) the time spent in programs, and(5) the

costs of variou's programs and approaches. Youthwork's knowledge

devel5pment plan places high priority on obtaining this information.

Gradually, Youthwork has also developed a knowledge development strategy

designed to maximize what will ultimately be learned. This paper is

an effrt to describe the outlines of that emerging strategy.

The strategy begins at the\ver,y beginning. It assumes that s

lit0e is currently known about effective work-educatiori combinat'ons

that few working hypotheses are justified, intense documentation is

required, and 4n initial knowledge and data base must be developed.

*v.
It attempts to address-the folloWing-broad questions:

1. Can the, school become a more valid institutional instrument for

preparing low-income youth to msi)es successful transitions to 'the

world of work?,

'2. What changes (,necessary institutional modifications) have to

occur to help the school become a more valid institutional instrument

for meparing low-income youth to make successful transitions to the

world of work'?

3. What are the most effective techniques and strategies to develop

closer cooperation and coordination between the CETA system and local

educational agencies?

4. What are the most useful available mechanisms or strategies to

attract and maintain more intensive private sector participation in

in-school youth employment programs?

5. Do the "exemplary" programs, included in the demonstration have the

power to impactdifferentielly upon the educational direction,

training opportunities, and early employment outcomes of participants?

6. How (or why), do the "exemplary" programs impact differentially

(i.e. do a better job) upon the educational direction, training

opportunities, and early employment outcomes orlow-income youth?

7. What is the relative impact of a variety of different treatments

and models with similar goals, in each, of the 'four focus areas?

8. *, What.are the particular components of pro4rams which can be

isolated as most critical to increased opportunities and other

Isuccessful outcomes?

9. What effective techniques can be developed to link what is learned-

.through job-site experiences with academic credit leading to a.

recognized degree?,

10. How can young people be more directly involved in the creation

of their own jobs, and in the planning and operation of employment

programs*designed for their benefit?

4

.11

Ideally, we would want to know the ultimate impact of the various

prograths on participants* employability, future education and employment

records, income levels, etc., but Youthwork's knowledge development

plan recognizes, as does the Labor Department's plan, that the

practical limitations of time, money and research capacity in a one

year demonstratidn program are very r al; and that such goals are

A probably impossible to realize.

Instead, the strategy focuses heavily on the developmental process

within each project: how it evolves, where it accomplishes anticipated

outcomes, how it accomplishes anticipated outcomes, what are the

characteritics of each participant at the beginning; what changes

participants experience, etc,

- 148 -

A

It'
This information sill be obtained' tWiCigh a three-tiered, hierarchical

knowledge development system: (1) local management information

component, focusing on the individual data which will be wularly

fed into Youthwork's centralized data bank; (2) the execution of

.a local knowledge development plan, sponsored on-site by each funded

project, with technical assistance, advice, and coordination

from Youthwork; and (3) a national, cross- cutting analysis of

funded projec , designed to shed light on several mijor issues: whether

(and how) t School can be an effective instrument in preparing

low-income y th to mak success'ful transitions to the world of

work; the means for inc easing private sector participation in in-school

youth employment progr ms; whether (and how) demonstration programs

_ improve low-income youth's prospectg*for better employment outcomes,

future training opportunities, or future educationdl ppportunities;

and whether (and how) steady progress can be made in promoting

closer cooperation between prime sponsors and local education agencies.

The three-tiered approach is designed to have a natural logic of

its own, in which the unit of analysis gets broader as one moves

from tier to 'fief-Tindividual participants, local projects, national,

demonstration). The ultimate objective is to facilitate making

generalizations across projects aboutlt:he value of various treatments,

or spproiches.

Each of the three level's/of inquiry has a corresponding method

of data collection, analysis, and reporting. In addition, the three

levels combine to cross-validate certain impressions apd to form

a common pool of data and conclusions that will be used to answer higher -

order policy. concerns. Inquiry at the policy level will be conducted

with a case study methodology. Data for a case study of key projects

Will be collected by field researchers, using structured observation

and .0terview techniques. In addition to forMing the

individual case'Studies, .these data will be used to cut across

the various projects and focus- ,areas to make judgment about presSing

policy issues.

The intermediate level of inquiry is built around local knowledge

and standard instrumentation

a development plans. Where programs have used designs and treatments, Youthwork has asked that common measure

be utilized in local knowledge development plans. These common

measures will be. used to verify project results and to conduct

special across project examinations to determine if there are differential

impacts on the various populations served. Another major component

of this level of inquiry will be a third party summative evaluation

of each project. These evaluations will be synthesized to maker

judgments about various program treatments and designs, Data from

these reports will be synthesied'to prepare'a series of policy

reports.

The third level ,6f inquiry,will gather data from a series f

one-Ttime-only,report to be submitted by each project. These,r ports

include reports on start-up, unique features outcomes and pro ct

costs. Cross-cutting analyses of each of these reports will be

conducted to determine the most effective program designs and

treatments. Conclusions from these analyses will be synthesized in

a series of How-To-Do-It Guides and policy reports.

- 149 - f(3,

GN
Bow-To-Do-It guides will focus on such specific issues as

how local programs can conduct effective self-st6dies; how to involve,

young people effectively as peer counselors and job placement

specialists; how to tailor on-the-job learning to facilitate the

maximum development of writing, reading, and mathematics skills;

how to reduce barriers to ybuth employmept in the private sector

through intermediary organizations ,ihich deal with bonding,

insurance, paperwork, and other "overhead problems" of the employer;

or how to organize the financial aspects of.youth-operated projects

so that earned income can be utilized for project expenses. Policy

(reports might .cover such questions as what responsibilities the

school systems should have for the education and training of young

people and what responsibilities are more appropriately handled

through another system, such as CETA.

National knowledge development activities began before the funded

projects were known, with the initiation of four kinds of activities:

(1) Knowledge development agenda - setting conferences,in Jach

of the four key focus areas, to help guide early decision

regarding areas of emphasis for local knowledge develop

(2). Reviews of research and literature in each of the focus

areas;

44

('3) The creation of how-to-do-it guides regarding local knowledge

development in each focus area, so that each funded project

could have, at an early point in their history, clearly

written technical assistance manuals regarding how to learn

as much as possible about what they were trying; and -

(i) A corporate self-study, conducted by' an outside third-partyN,

observor, regarding the decisions made in the development of

Youthwork's activities, the factors influencing the, eta.,

to provide on-going data about the progress of an intermediary

organization created for the purpose of implementing a Labor

Department demonstration.

Other knowledge development activities currently, planned include:

(1) How-to -do -it manuals: As funded projects move into their

implemeAation phase in October of 1978, information will be

systematigally.gathered regarding components.emort--ccritical

to successes. This information will ultim4tely be synthesized

in how- to -do -it guides in each of the four'key focus areas.

(2) Policy summariep: Each of the fu ed projects, will produCe

one-time reports on%start-up problems, implementation issues,

outcomes, and-costs. These reports will be synt4sized in

404 four policy summaries, which will be published' periodically.

The first report (on start-up problems) will'be ready in

;February or March, 1979.

(3) Special studies: Special studies Will be commissioned on an

as needed basis, fipcuSing on studies which can be put to

immediate use by loaal program managers and policNplanners.

'These studies should be completed by July or August, 1979.

150
Although the Youthwork knowledge development plan is ambitious,

the limitations and impediments to learning'from the demonstration

have also been recognized. This is not a controlled experiment, nor

even:an uncontrolled experiment. It might best be described as an

dncontrolled non-experiment. It is not Controlled because it was

conducted as a competition involving many relatively unsophisticated

pfogram operators. It is not an experiment because the program

operators have,each formulated their program design with only occasional

reference to the application guidelines; and many program operators

have demonstrated little or no interest in experimental designs.

Many paid only lip service to knowledge development in their original

,.. proposals; and negotiations garding knowledge development have

had to be conducted hurriedl , in order to keep,,to Labor Department

funding schedules.

There is even some considerable questionvregarding the "exemplariness"

lof many funded projects. Although the program is referred to as a

demonstration of "exemplary" in-school projects, it cannot be taken

for granted that most programs will, actually prove to be exemplary

in their implementation. This is an issue which must still be studied,

as all the others. This creates problems, for example, in thinking

about "planned variations" across sites. It is a -lot more difficult

to accomplish planned variation among non-exemplary programs.

Another limitation springs ff6m restrictions on amounts available

for local knowledge development. AlthoUgh the program is labeled a

demonstratioh, regular CETA restrictions on administrative costs

(20% maximum) have been applied; and knowledge development is defined

primarily as an adminstrative task. Since more thaq,10% has usually

been allocated for,a0ministration by prime sponsors and, program

operators, less than 10% is usually left to pay for local knowledge

development costs. This is another example"of what the Labor

Department referred to, in its original (1977) knowledge development

plan, as the tension between getting the money into the pipeline

and carefully orgaffizing to learn as much as sossible from the funded

programs.

All these limitations argue for the strategy adopted at the outset.

To the extent possible, resources have, been concentrated and structured

so that underlying ideas can be given a reasonable test. Youthwork's

national knowledge development efforts have been focused on finding

out how to do,a few key ,things better. We have attempted to direct

(or redirect) a_ major emphasis of each project's knowledge development

plan on an issue or issues identified as national priorities. If

all goes well (a big "if"), valuable information should be available,

for future researchers, progr m operators, and policy makers, within'

twelve months; 'We might discov r, for example, that jobs can be

created less expensively throu h youth-operated projects than through

public sector employment. Or that jobs created in youth-operated

projects providea more valuable transitional'e4erience than public

sector jobs. We might learn more about how -tó build basic skill°

instruction.(writing, mathematics, etc.) into the on-site job experience,

and hoW to measure the academic credit'value of skills learned on

the job more accurately. We might identify techniques for cushioneing

first job experiences in the private sector, with implications for

the expansion of private sector ,Iacements. a - 151 -

1t
Our national study might also tell us more about the value of

placing a certain proportion of youth employment funds within the

schools. What, for example, would lie the policy implicationt of a

finding that, even in "exemplary" in-school programs, institutional

impediments make significant participant outcomes highly unlikely?

What would be the implication of a finding that peer or para professional counseling makes roughly the same 'impact as professional

counseling? What would be the value to program operators of a private

sector treatment which successfully places' small businesses in a mentoting

relatibnship with individual participants and absorbs some of the

usual costs (in time, money and energy) of employing temporary

help?

From the above, it should be clear that Youthwork is dealing

with issues which are potentially very important to program operation

and policy-making in the area of in- school youth employment programs.

If its current knowledge development efforts are successful, it

will not only have provided useful ways to identify and spread the

best of what happens in the in-school demonstration, but it will

'have created a sophisticated data base in an area where the currently

available data are skimpy and incomplete. This may make possible

future research'tasks that are hard to get at in a time-limited

demonstration -areas involving institutional changes within schools,

changes in the definition of the kinds of people most appropiate

for various, staff roles, information about how to facilitate the

maturation process through increasing the operational responsibilities

of young people within a project.

The implementation problems of the first year may also argue°for a

significant narrowing of the knowledge development focus in a demonstration

Prior to the beginning of any competitive or non-competitive process,

in recognition of the,powerful anti-research forces unleashed by the/

simple desire to "put a jingle in the jeans" of our nation's diadvantaged

th. Or, finally, we mayoat last begin to focus on the. question

of whether efforts which begin with the motive of putting a "jingle

in the jeans" of our 'nation's unemployed youth too often end tip

adding-to the rustle of dollars in the trousers of the professionals

who staff 'the CETA systemts and local education agencies.

(

152 -
BACKGROUND: THE PURPOSE OF THE PROPOSED

DOL/NI2 FIELD TEST OF THE ZAREER -INTERN PROGRAM

The Problem and the Program

(1) The Pgoblem: In 1973, over 10 percent of youth aged 14 to 24

left school hatore receiving a high school diploma. The percentage of

those that left school was higher for blacks than for whites: amoing /

18 and 19 year-olds, 27.7 percent of the black males and 23 percent of

the black females had dropped out in 1973, in contrast to 14.1 percent

for white males and 15.2 percent for white femaleS. Dropping out meant

bad economic news. According to the U. S. Department of Labor, 20

percent of high school drop-outs between the ages of 16 and 24 were

unemployed in 1974, compared with 10 percent of all high school graduates.

When drop-out unemployment rates are compared by race, youth from black

and minority groups are doubly disadvantaged: about 16 percent of white

youth without high school diplomas aged 16'toe24 were unemployed in 1974;

32 percent of drop-out youth from black and other minority groups wanted

jobs and could not find the

(2) The Program: A program developed by the Opportunities Industrialization Corporations of America, Inc. (OICs/A) in Philadelphia, Pa.,

gave promise of showing how to turn around these statistics. The Career

Intern Program (CIP) is an alternative ,high school for .16 to 21 year olds

who have dropped out of school or are at risk of dropping out before

graduation. The goals of the program are to enable these students to

complete high school, acquire occupational knowledge, plan for a career

and improve' their reading and mathematics skillsin short to facilitate

the transition from high school to work or further education.

The CIP itself has three phases. Phase I is directed a'improving

student's career awareness. The phase.lasta 21 weeks and includes

ilasses in English (with an emphasis on skills mastery), Mathematics,

Social Studies and Science. During this phase, learning is keyed to

occupations through specially developed materials. Weekly' individual

counseling sessions, combined with the exploration classes, lead todevelopment of an individual learning plan for each intern. !

Phase II may last from four months to a full year, depending on how

many credits interns need for graduation and rate of student progress.

In Phase II, interns participate in two to four "hands on" job experiences and in other courses which fuse academic and career information

in a sequence of individual learning activity packets. In addition,

Phase II stresses individual instruction, independent study and advanced

courses in the academic subjects. In Phase III, students concentrate

on the transition 'from high 'school to work or to more advanced education.

Students choosing the option of vocational or on-the-job training are

assisted by counselors for six months after leaving CIP; college-bound

interns are Counseled for a full year after leaving.

- 153 -16-,
The Evaluation Design, Methods, Measures and Findings for the

Original Philadelphia. Study

(1) Design: The design involved three separate coh)ts of applicants, applying at six to eight month intervals. Through over-recruitment

and a.lottery process, known in advance to all applicants, three separate

sets of experimental and control groups were selected in a ratio of about

3 interns to 1 control. This ratio was selected to permit maximum entry

into CIP with the minimum N estimated to be needed for a group large

enough to be sensitive to educationally meaningful effects.

(2) The Methods: StWy methods combined ethnographic studies and

psychometric data.' The ethnographic study included (a) case studies in

depth, over time of interns and cdhtrol students (b) semi-structured

observations of classes and program events, and (c) field observations

through a participant-observer. The primary purposes of these studies

were (a) to help understand how the student's experiences in CIP and in

the comparison schools differed, (b) to improve the description of the

CIP processes and practices, (c) to identify what about CIP might be

accounting for program effects, and (d) to collect data about effects

useful in their own right to describe program results.

(3) Measures: Four tests were administered: The Stanford

Achievement Test (for reading and math performance), the Career Maturity

Inventory (for career knowledge and development), the Coopersmith SelfEsteem Inventory, and the Internality/Exterhality scale. These measures

are described in the Final Report, Vol. LI, pp. 11-18; pp. 81-149.

All tests were administered prior to selection of all applicants and

after the program to interns and to controls. (See Final Report,

Vol. I, pp. -9-10)

(4) The Findings: All data were audited. Several statistical

approaches were used (variants of ANOVA and ANCOVA) to help rule out the

possibility ihe conclusions would be due to a statistical artifact.

Among the findings:

In December 1975, 67% of the CIP students but only 13% of the

controls had graduated or were still attending school. About 44% of the

interns had received high school diplomas and 23% were still at their

studies, while only 7% of the control students had graduated and only

6% were in school,

'247 Changes in reading and mathematics achievement showed interns

gained more than controls, although the final levels of both groups left

room for improyement.

CIP interns showed very gh levels of career planning and development skills, as measured by th tests, interviews, and the ethnographic

observations.

Graduates were'followed p in Fall 1975. Of 77 graduates, 717. , were employed, in college o n technical school as compared to 39% of the

controls. None of the con rol men had enrolled in college or technical

school as contrasted to 2 of the male interns who were continuing their

education.

- 154 ledL)r
a

'1

These findings are described in the Final Report, Vol. I pp. 101-181

and the data are presented in detail in the Final Report, Vol. II pp.

19-73.

The Proposed Department of Labor/National Institute of Education

Field Test ot the Career Intern Program

The Youth Employment and Demonstration Projects Act of 1977

(PL 95 93) is addressed to the plight of the three million young people

between the ages of 16 and 21 who in 1976 alone were out-of-school and

out-of-work. Many of these unemployed young people can from very poor

families, families Who were unable themselves to cushion the blow of

youth employment. Many came from minority groups. Many had dropped out

of high school., The Youth Employment Act and Demonstration Project Act

( YEDPA) has two main thrusts. The first.is direct provision of work;

the second is learning what could be.done to have a larger, and longerterm impact on the problem of youth' unemployment. The YEDPA provides

authority to conduct demonstration programs.

Under the demonstration program provisions (Section 341), the

Department of Labor, Employment and Training Administration, is supporting tests of a variety of approaches to helping young 'people stay in

school, complete their training, and then to make the transition from

school to work. Of particular interest is finding out whether programs

proven effective on a small-scale continue to be of benefit to low - income

youth when the approaches are implemented on a larger-scale. Such studies

will help identify specific programs that can work on a nationwide basis;

but also the studies might identify what program elements are necessary, .

sufficient, and reproducible and thus inform the development more generally

of workable, effectii.va projects.

The Career Intern Program to the attention of the Department of

Labor as a program of proven effritleiveness on a small scale (Philadelphia

prototype site) in helping low-income youth, who had dropped out ofschool

or Who were at risk of dropping out, complete school and make the transition-frail school to successful post-high school experiences. The

Departgent of Labor has entered into an Interagency Agreement with the

National Institute of Education to manage a field-test of the Career

Intern Pliogram.

The Interagency Agreement established two c ents for the field

test. The first component is the operation ot thceTilladelphia prototype

site and at least four addtional sites (three urban and one rural) by the

Opportunities Indukrialization Corporations of America (DICs/A). The

four sites are Seattle, Detroit, New York and Poughkeepsie. A separate

award from the Institute to the OICs/A hasbeen established for the

management of 'the program operations. The second component is an evaluation of-the field test. Both components are under the direction of

the National Institute of Educations

6

155
Goals and Objectives of the Evaluation Study

A. Goals: The goals of the evaluation study are two-fold.

The first 1773 learn what education, broadly defined, can do to help

low-income4youth at high risk of being unemployed prepare for a success-_

ful transition from high school tCri4ork or further schooling. The

second goal is to find how educational innovations which give promise

of working on a small scale can be transferred elsewhere to benefit the

larger populations whose needs the smaller programs were intended to

serve. These are ambitious goals'indeed for research. The field study

may contribute modestly to reaching them; it is not expected to attain

them.

B. Objectives: There are four specific objective tasks which the

evaluation of the CIE field test is expected to reach. .These are phrased

below as questiohst

1. What happens to the Program itself in the process implemed

tation n additional sites? What accounts tor the changes or adaptations, in

if any? VOr the fidelity, if any -to the original program goals and

practices (process evaluation)? Information on'the implementation of CIP

in the new sites is intended to help answer three sub-questions:

does the program remain the same in terms of goals and

practices?

are the changes, if any, improvements in terms 'Of the 'overarching goal of effectively helping young people who are

drop-outs, unemployed or at risk of dropping out?

is the process of implementation more effective through a

system such as OIC in comparison to the processes of

research utilization and dissemination through the usual- -

developer-public school linkages?

The answer to these questions should help assess how well and readily the

CIP approach can be implemented, a question of some importance with regard

to the possibilities of larger dissemination efforts. The answers also

should contribute to knowledge about implementation in systems which differ

in power relationships, political considerations, incentives for change,

and other variables believed important in the literature on educational

change.

2. Does the Career Intern Program continue to be effective in

helping youth when it is implemented in sites other than the Philadelphia'

prototype? The CiP "worked" in Philadelphia, according to the earlier

evaluation studies. Interns completed high school, improved their reading

and arithmetic skills, improved their career planning knowledge and skills,

and successfully made the transition from CIP to paid employment, continuing

education at the postsecondary level, or, voluntarily, the occupation of

homemaker. Does CIP continue to work in helpingother young people achieve

these outcomes, when the approach moves from a small, somewhat protected

prototype, to at least four other sites?

- 156 f I
3. What happens to young people in the CIP process that could

account for its effectiveness? "The early evaluation identified what seemed

to be essential elements which accounted for CIP's effectiveness for the

interns. .(See Final Report: Vol I, pp. 201-206). These program components,

identified mostly through the case studies and ethnographic research, are

hunches about essential elements. The four new sites offer additional

opportunities to find out if these lements are in fact responsible for

CIP's outcomes and if they are esse tial for program effectiveness. The

task is to identify the essential ts. The answer to this question

might help in designing other programs.

4.

lity, impact

e... Job C

How does the CIP approach compare in effectiveness, feasibiand other factors Important for policy with other approaches

s Action s sored youth ro: ams to help the lation

rou: t e smos. e answers to t is question, ich may ave tentative, should help cumulate knowledge about the relative

demerits) of different approaches to helping la.w-income, highto serve to be quite

merits (and

risk youth.

Summary of Data-Collection Methods of the Evaluation Study

In general, there are four data-collection methods proposed to

gather the information necessary to complete Tasks 1 and 2: standardized

tests, interviews, observations, and ethnographic studies. In addition,

there is one questionnaire to be administered to controls, at the posttesting

session.

Tasks 1 and 2:

standardized tests.) To assess program elements on interns; a

battery of standardized tests will be administered immediately before, at

about midway through, and after treatment. The content of the tests , includes academic achievement, self-perception, logical reasoning, and

career awareness and planning skills.

To determine if program climate is the same in the new sites as

in the prototype site, standardized site climate tests will be administered

to interns and staff Members. , . Interviews. Study staff members will interview CIP interns and

staff members as well as relevant community members to collect information

for both Tasks 1 and 2. For Task 1, data from the unstructured interviews

will be used to create descriptions of the CIP implementations at the four

new sites. These descriptions will be compared to a model of the prototype

program aVeloped on the bases of analysis of the program documents used by

OIC /A staff to train new site staffs and of descriptive information gained

directly from interviews with the OIC/A staff. For Task 2, interviews will

yield data about outcomes of program participation not measureable with

standardized tests, such as how satisfactorily CIP graduates perform in job34

'compared to controls), in the perceptions of employers..

Interview topic checklists were deSignedto structure interviews

as little as possible. The study staff's experience has been that the most

comprehensive and most accurate data can be obtained when respondents are

not limited to a prespecified set of responses to questions. Thus the

interview guides consist primarily of Mnemonic lists to remind interviewers

of the topics about which data are needed from respondents.

157 - NJ

1
I

Observations. In addition to interviews of CIP participants to

determine the fidelity of new-site implementations to the prototype, the

study will include structured observations of CIP classroom sessions to see

if the instructional techniques prescribed by the developers are follbwed.

Data from the classroom observations will enhance the analysis of CIP

operations for Task 1.

Observational data collection requires no respondents, as the

observers do not ask questions of those observed. They simply record

events.

Ethnographic studies.' Another method of data collection for which

no instruments are needed will be ethnographic observation by study staff.

members. Through the ethnographic studies, more data will be collected for

making the judgments necessary to complete Tasks 1 and 2.

Ethnographic,'participant- observer data will be recorded-by

evaluation-team staff members who will spend several weeks in each CIP site.

The participant-observers will attend classes, assist teachers and other.

staff members .and get to know CIP participants in a more casual way than

short exposure allows. The ethnograplfers will record their observations of

CIP processes for inclusion in the study's data base.

Questionnaire. One questionnaire is included in the survey

design. To be adknistered to controls at posttest time, the questionnaire

is designed to find out what the controls were doing while the interns were

in the program and what they are currently doing, so their further education

contacts or employers can be interviewed.

Task 3 entails analyzing the CIP processes an8'its outcomes on interns

jointly to raw inferences about which program features lead to which outcomes. No data beyond the process and outcome documentation involved for

Tasks 1 and 2 are necessary. In essence, Task 3 is an analytical process,

requiring no additional data.

Like Task 3, Task 4 is primarily analytical. Data will be reviewed

about aher programs EFFnded to achieve similar outcomes for similar

groups of youth through secondary analyses of evaluative data collected

/by:these other programs (e.g.) Job Corps, Action-sponsored programs).

These data will be obtained through literature reviews and conferences

with other evaluators. Only cost information which can be obtained

through document review, is.needed from the CIP sites in addition to the

data collected for Tasks-1 and 2.

1'7

- 158 -
t.

CPPV'S PRIVATE SECTOR INITIATIVES DEMONSTRATION

A Paper Prepared for the Office of Youth Program's Knowledge Development

Project Conference (Reston, Va., October 5-6, 1978)

1. Overall Purpose of the Demonstration and Research

The Corporation for Public/Private Ventures (CPPV) is under contract

to the Department of Labor's Office of Youth Programs to design, operate

and research a multi-faceted Private Sector Initiatives Demonstration,

aimed at.testing a variety of approaches for increasing private 'sector

employment opportunitiks for youth.

The initiatives in this deionstration represent five different approaches

to the problem, as followsr

1) New Career Pathways -- building access routes for disadvantaged youth into private sector jobs through restructuring

of entry-level positions; developing institutional linkages

between private sector firms and public or non-profit organizations; creating subsidized training relationships between

employed adults and youth with 'a low adult/youth ratio; and

carefully modeling training career entry sequences tailored

to the needs of specific firms or industries.

2) Youth Pre-Employment Services -- testing the effectiveness

of existing non-profit programs which combine job development, placement and follow-up activities with modest job-'

related education and counseling.

3) Use of local business intermediary units to reduce transactional costs to firms such as private sector-run placement

and job development services, or OJT contracting services -

which eliminate paper work, "hassle" and the necessity of

private employers to deal directly with g9vernment agencies

and programs.

4) Youth Entrepreneurship Programs -- testing the viability of

several approaches to creation of small business ventures

which combine modest front-end subsidy (to keep them competitive) with a youth trainir and employment agenda.

Att, 5) Direct,Incentives -- a formal and limited experiMent to

test subsidy formats,nore generous than existing OJT, to

increase hiring and training of out-of-school disadvantaged

youth by priiiate sector firms.

In addition, CPPV may conduct research on a program which tests tin'

use of volunteers (employed adults) to assist ycith in job placement and

adjustment.
ti

The overall purposes of research are to study the impact of each

initiative, and the programs funded within each, on youth career enhancement;_to assess and document the'program development. and implementation

process followed in each initiative and its component programs, both to

provide qualitative assessment and to provide information useful for replications of successful approaches; and to compare effectiveness (for whom and

under what conditions are programs effective?).

.Th0- initiatives in this Zemonstration are closely derived from the

OYP Knowlede Development Plan, which said,

A demontration project will be undertaken to study and test

four separate approaches to increasing employment for youth

in the private sector:

(1) Direct employment subsidies will be provided for each

disadvantaged youth hired.

a (2) Training cost subsidies similar to those under the

JOBS-contract program will be tried, with varying

levels of subsidy and specification.

(3) Apprenticeship subsidies for smaller employer's to hire

and train youths similar to those tested in the

Community Services Administration's exemplary Open

Roads project will be further tested.

(4) Entrepreneurship options for youths which have been

developed in some localities will be carefully

demonstrated and tested.

The "we-employment services initiative" was added to the typology of

programs described in the KDP to enable close research scrutiny of a program

model tlit Department of Labor has already provided considerable support,

the 70001 program, and to permit.expansion under close research scrutiny

of the well-regarded Jobs for Youth program from its New York and Bo'ston

bases to a third site, Chicago. The themes of new career pathways and

volunteer involvement in job assistance emerged from discussions with Office

of Youth Programs staff subsequent to publication of the KDP.

c. 2. Specific Issues to be Addressed in Impact and Process Analysis

Certain kinds of generic questions will be probed in all programs. These

. include:

A) Impact on Participatin Youth

Here the emphasis will e on economic impacts, including skill

development, increases in w ge levels and increases in employment

stability which can be attrib ted to program participation.

,/ 160 1 `..c
Impact on Firms

Here the emphasis will be on indicators of increased emp loyer e. ,receptivity to the hiring of y9uth workers, as indicated by increased

hiring of youth (as a percentage of all workers), and by such institutional impacts as alterations in yersonnel policies, hiring practices or job structures which have the effect of opening up opportunities for youth*.

C) Indicators of Program Effectiveness

Such indicators include cost analysis (e.g., cost per positive

termination, cost per placement in unsubsidized private sector

employment per month); youth attendance and retention in program;

and ability to reach and maintain projected service levels.

D) Process Analysis and Documentation

Under the rubric of process analysis and d cumentation, research

will provide thorough statistical, bugetary and programmatic descriptions of each program; qualitative assessments of program operations

(e,g., leadership, managements skills, skills of trainers, organizational climate) and an analysis of factors enhanCing or constraining

program success, both within the program per se and in its political/

'organizational /economic (labor market) context.

Process analysis, in CPPV's view, is important not only to he p draw

policy inferences from quantitative findin but also, and especial ly, to

provide guidance for reiJlication of affective rogram models.

While these generic questions frame the overall research effort, each

initiative raises intrinsic questions which are of special or unique importance to it. It is beyond the scope of this brief summary to list them

all, but some of the major questions, by initiative, are:

1 1. New Caee/. Pathways - Can job restructuring be applied in private

firms as a\ conscious policy tool for increased access of disadtaged youth to private, unsubsidized employment (and if so, under

what conditions)? Can subsidized training relationships between

youth and skilled workers,on a one-to-one or small group basis

be effective in increasing private sector employment opportunities

and skill developMent? How can career pathways be developed in

emerging industries/occupations which make youth competitive

for entry-level jobs?

2. Youth Pre-Employment Services - Do pre-employment services

succeed in matching disadvantaged youth with job opportunities

in Amen.- to medium-sized firms for which the%would otherwise'

lack knowledge or opportunity? Can the existence of a pr,e-employment service.impaet firms' hirtng policies in a way that results

in a net increase in jobs for youth?

161

17
3. local Business Intermediaries (Reducing Transactional Costs) -

Doe:, the establishment of a business-planned, business-run intermediary ,increase the effectiveness (i.e., increase private sector

youth opportunities) of such traditional manpower tools as OJT

and job placement/development services (essentially a test of a

delivery system)?

4 Youth Entrepreneurship_-'Under what conditions can the business goal

pf profitability be reconciled with the public goal of increased

youth employment and training? Is subsidy of business development a cost-effective manpower strategy for (some) youth? What

are the most effective ways to provide technical assistance and'

support to such ventures to enhance their stability/profitability

(e.g., traditional At, franchising support by an established

company, linkage to community -based economic development or manpower

programs)?

. . 5. Direct Incentives - What level of incentive is necessary to elicit

significant business "take-up"? What kinds of firms. respond?

What net increase in youth employment among those firms is

achieved? Do direct incentives also constitute an incentive

for fiims to "cycle" youth as the subsidy expires? Do windfall

profits result? Are adult workers or non=eligible youth displaced in these firms?

3. Design and Analysis Methods

In all its knowledge development research efforts, CPPV expects to

combine quantitative analysis with process analysis. The former will rely

heavily on computerized information provided by CPPV-designed reporting

forms which will collect infOrmation tailored to each initiative, on:

demographic characteS4cs of all youth (including prior employment and

training history); prograM reports on all youth (attendance, skill inereases,

job placements, promotions, etc.); termination data (exit interviews on

destination plus nature of termination, reason for termination); characteristics of participating firms (nature of business, size, demographics

of work foi4Ce, etc.); and prograM costs. Such information will be supplemented as appropriqte per each initiative and its individual research

design by data on firms over time, folloW-up post-program data on youth,

and data on comparison youth collected by CPPV research subcontractors.

This data base will be analyzed to assess economic impact of program

on youth, and impact, as relevant in various initiatives, on the hiring

pracrices of. firms. Typically, multivariate analysis will be used to

determine the variance in program output attributable to youth characteristics (e.g., sex, age, prior work history); program characteristics

(e.g., duration, cost); and firm characteristics (e.g., size, industry).

In 'assessing firm response to initiatives, the effort will be to factor

in both the economic condition of the firm and local labor market conditions, as availability'of data permits. In initiatives 1-4, the most

prob=able comparative measure of impact on youth will be either a simulated

<%1

I

SD e

c "

l'"

162
""44, (

I . control or youth participating in public manpower programs. In the direct

incentives experiment, our current (and still formative) thinking cal s

for a control group of firms, with a youth comparison group formed from

subsidy-eligible youth seeking employment in control (i.e., non-subsidized)

firms.

As a way of validating termination data from programs, and as a way of

laying the groundwork for longer-term follow-up if preliminary results

warrant same, follow -op interviews of participating youth (and, where

appropriate, matched comparison youth) will be conducted three months after

termination in each program.

The process analysis design for each program will consist of a series

of programmatic, implementational and contextual questions designed for each

initiative. The data base for process analysis will consist of information

gathered from.sites through periodic observation, semi-structured interviews

and, as appropriate, survey instruments, as well as statistical descriptions

of programs, from the CPPV information system. Analysis of process findings

will be largely qualitative and documentary, although in some cases procdss

findings may be amenable, to statistical analysis through use of 0-1 variables or scaled ratings in explaining program outputs. CPPV believes the

process analysis may be particularly valuable in assessing the responses/

reactions of private sectoi. employers (from whom collection of qu itative

data is inherently,difficult), as well as in providing documents ion and

analysis useful for replication. -

4. Timetable of Research Activities

With the possible exception of the direct incentives experiment, CPPV

expects all research activities to begin between now and January 1, 1979.

In all cases, research Will be underway by the time of first youth entrtnce

into programs (several of which are now in planning stages). Final

analysis of each initiative will be due within tqo years of start date.

In all cases, as well, a "working paper," based on CPPV field reports,

procss analysis or program start-up and information system data, will be

prepared after the first four months of each initiative's operation. A

preliminary research report from subcdntractors will be due on each initiative (covering both process "findings and, as available, preliminary quantitative analysis) after one year.

5. Known or Anticipated Research Limitations

Obviously, this research faces all the - endemic i ues of drawing

strong empirical/sta istical inferences from non -2 ratory settings:

difficulties in as mbling or maintaining approp late comparison groups;'

possible instabil ties of organizations and progr undergoing research;

cooperation of ogram operators; less than omnisolien researchers, etc.

It is further complicated by the twin effort to a) de lop model programs,

and b) research those models, since the failure for on-conceptual reasons

-- 163
of one or more programs in the'refatively small program sample in each

initiative can cripple knowledge-development (upless the failufe is "exemplary".). Knowledge developMent concerning private sector initiatives

inevitable involves considerable cooperation from private sector firms and

individuals, who hove no clear incentive (and my have reITETance) to

cooperate. The hope is, of course, that careful planning, and putting

research.in place before programs,,begint can minimize these obstacles.

Finally policy must ultimately consider the potential macroeconomic effects

of programming appfoaches, but ability to generalize from the micro world

of the deMonstration to the macro level is inherently limited.

6. Other Possible Limitatiois

In general, this demonstration and its Cbpsta ituent initiatives occur

in a fluid social, political and econdmic context where any number of factors

exogenous to the demonstration per se can influence'CPPV's ability to

mount Tffective programs -- and hence to deverlop knowledge. For instance,

the imminence,of CETA Title VII complicates thepossib4lity of testing

local business intermediaries since private industry councils Tay co-opt

the ground; the possibility of a Targeted Employment Tax Credit aimed

partially at youth, as the administration has proposed, may render a direct

incentives experiment redundant; the political issues which surround and

and all privatecsector initiatives -- but especially direct incentives to

the private sector -- could blas ,some programs out'of the wate tc.,

A 7. Likely Research Findings

Since we are at the front end of his demonstration, it is simply

_impossible to predict the likely'directi of research findings in any

initiative. ,

8. Implications for Youth Policy Priorities

Whatever the nature and direction Of research findings in this demonstration, knowledge developMent should reduce the uncertainty surroundingsucbAkeY policy issues as: the cost effectiveness of private sector-targeted

manpower programming for disadvantaged youth; the most promising approaches

to such programming; the ability of government to increase the demand for

disadvantaged youth in the private sector through manpower pr grams; the

levelT\of subsidy required to do so, and the effectiveness of pure subsidy

approaches; the e'-ent to which. increased demand for disadvantaged youth

substitutes for demand for*:Other workers: and the characteristics of youth

for whom various private sector initiatives are most effective in producing

increased employment opportdhfty, stability and wages.

44- A

IF

. Additionally, the careful testing and documentation of program' approaches

shOuld provide models for replication of successful efforts elsewhere in,

a field wheie little.is known about what can.be,attempted, much'less what

,works. 'In addition to Federal, state and loia/ manpower,..officials, the

broad audience for such information includes pri vate sector-Kganizations.

On

7 rs

1'

1 2
Specific audiences which are likely to find information yielded by this

demonstration useful include private inductry councils formed under the

proposed CETA Title VII, and other efforts of Office of Youth Programs,

such as the Corporation.for Youth Enterprises (which will be.launching a

broader test of yollth enCrepreneurshp).

9. Likely Implications for Future-Research

, Almost inevitably, anyresearch effort spawnsissues for future re-

, search, whatever the findings, and so one expects the current effort will

do so also.

. . ,.

. The most import'nnt contribution of ihis,demonstrationto future rescarrh efforts is, however, likely to be what is learned about efforts to

reseaggi private sector behavior in response to manpower programming and

policies. This is ,a re tively unusual eedearch,eMpasis, yet clearly an

important ode- ifpubli/4policy aimed at increasing private sector employment of the disadv.aritaged is to proceed on a basis of reliable knowledge.

, oThe multi-faceted, multi-part nature of the research task for this demonstration is a complicating factor@ in some respects, but it does provide

a Leh field for lear'ning more about the feaSibility of data collection

in the private sector,'the Sourced available for succollection and the

approaches to data collectiln .which are most tolerable to t e privatesector.

.60

.:o .1 a

gig

ti

.1

. . -

. A ,-.41:6 5 .

. r; ( 4

A

4 V./ e t 'i e

'" 0. '%* ee

: :

:

0 0 : _ 4
WASHINGTON, D C 20525

TO: Andrew Hahn UWE: 9/29/78

Center for Public Service, Brandeis University

,FRCM: Iry id MUchnick

Special Assistant for Youth Programs, ACTION

-`10 SUBJECT: Project Overview Statement - Youth Community Service Project,

Syracuse, New York

1. Overall Purpose

To demonstrate and evaluate the capacity of a local community to implement

this model of a voluntary, community-based youth service program.

II-4 Specific Questions/Knowledge

1. Test the demand from and interest of all young people in youth community

service.

2. Test the interest in yogn community service of all potentially eligibletypes of sponsors (pubr1C ageno4es; private non-profit organizations;

community groups and groups of youth) .

3. Test the capacity of all interested sponsors (especially community groups

and groups of young people themselves) to develop meaningful, non-traditional

community service projects, consistent with a representative distribution

among the eligible types of sponsors.

4. Test the relationship between the community service,experience and the

educational process.

5..Test the communitlils capacity to mobilize itself in support of its youth,

and secure the maximum community mobilization in order to: demonstrate

the community's commitment and 'loyalty to its young people; demonstrate

the community's esteem for volunteers per,forming service; enhance the vblunr

'teer's pride in their community service and alleviate their alienation

from their community; increase the opportunities available youth the

program ends; increase the likelihood of some projects continuing r the

program ends; and increase the community service provided by-the limited

federal funds available under the' program.

6. Test the effects of y;uth community service on a diverse group of individuals,

youth, beneficiaries, sponsors, supervisors, facilitators and the communityat-large. et,

PEACE. CORPS VISTA UNLVERSITY YEAR FOR ACTION NATIONAL STUDENT VOLUNTEER PROGRAM

FOSTER GRANDPARENTS RETIRED SENIOR VOLUNTEERS

166 -
III. .Project Design and Analysis Method.

One evaluation of the YCS/S project focuses upon the process of implementing

the demonstration in. all sectors of the Syracuse community. This includes

the administrative and decision making process of the YCS/S board and staff;

the inter-relationships between ACTION and YCS/S and the Syracuse community;

the effects of the political, social and economic structure of Syracuse and

Onondaga County on the implementation of the YCS/S project and vice versa;

and, an analysis of the volunteers and volunteer service projects. Periodic

reports fran thke evaluators will enable ACTION and YCS/S to modify features

of the program's implementation as the need arises. And, they will facilitate

ACTICN's design of future, 9Diaistrative organization and process in a manner

consistent, its aim of maximum local responsibility.

I In addition to the implementation analysis, ACTION has contracted with the

Urban Institute to design several studies which may be used to complete the

evaluation of the ICS/S.demonseation: These include:

1. Descriptive Study: Basic to evaluating and monitoring the YCS are

descriptive data on volunteers and their assignments, sponsors and

their resource commitments, supervisors and supervision provided,

YCS finances and expenditures, and other such aspects of the YCS.

These data can be analyzed independently or employed as variables

.through which to examine effects under the other studies. The ideal

means.for obtaining these data woad be through a management informatip.system (MIS) for the YCS.

2. Volunteer Impact Study: This study examines the effects of tie YCS

on the volunteers during their tenure with the YCS. It does this

through comparisons of YCS volunteers with youths in other programs

and/Or ypuths who are not in any employment program. The study collects data fran the volunteers at several points during their service

tenure.

. A 3. Volunteer Follow-Up Study: This study is an extension of, the Volunteer

Impact Study. It examines the effects which the'YCS has on volunteers,

after termination.. Data are collected from volunteers and comparison

groups by any of several survey techniopeS.

4. Goninuqty. Impact_ StOdy: Thls study assesses the effects of the ICS f on those individuals and fhsx.it,pti4pns which have had direct=ntact

with the volunteers. These individuals and institutions include the

recipients of-the volunteers' services, the volunteer supervisors

and the sponsors.'' If it is desired, this study can be expanded by 0

"gaps substudy." It would augment the main study by detenninihg

whether or not the YCS has acted to narrow the gaps between disadvantaged recipients (e.g., poor) and groups of the more advantaged

non-poor r.

5. Volunteer Activity Survey: De nding.on the type of data which the

MIS sses.on volunteer placements, this study can assume several

confi urations. In gerieral, it is intended to iocument what the volunteers are doing, and to classify volunteer tasks so that they can be

used asp independent variables in examining YCS effeCts..

Vb. 167s-1.

Iv

O
6. Moderator Variable Study: This Study is designed to identify policy

relevant and manipulable project variables which might affect the impact

of the YCS. Where possible, the study will examine the effects of these

variables on YCS impact. A primary focus of this study would be to

examine the influence of sponsor and project organizational and contextual vIriable on the effectiveness of the YCS.

7. Cost-Effectiveness Study: No additional data are collected for this study.

Rather, cost-effectiveness ratios for the YCS are established yia cost

.data obtained through the Descriptive Study, and effectiveness data collected

under the Volunteer Impact and Follow-Up Studies and the Community Impact

Study.

IV. Research Timetable

1. Implementation Analysis

- October, 1978, StartUp Phase Report:

.0 YCS implementation activities qpto the Pilot group and intitial perceptions of the "second wave" of volunteers and sponsors. Initial comparison

of the policies of the prpgram as set by ACTION with their actual implementation.

-March, 1979, Growth Phase Report:

9 YCS implementation activities from the start of the "second wave" through

December, 1978. A detailed analysis of the second and succeeding groups

of volunteers as compared to the pilot group. 'ahe evolution of management

"style" and procedures, the solidification of staff job descriptions, and

responsibilities, key contextual influence during this period, criticalpolicy decisions and their effect, and descriptions of field operations.

The primary emphasis is sponsor evelopment and volunteer recruitment

activities. MIS information for this and the "Start7up Phase."

-July, 19796Maintenancease Report:

YCSimaementation activities during the initial phase of project maintenance: i.e. that period in time in which YOS administrativresoucces

become totally devoted to maintaining existing volunteer opportunities

community. Extensi e lb

and insuring a productiveland meaningful

"site his ry" reports

service }uar for

volunteers, sponsors and the c

on.ectivities to date BPI' 10 to 15 selected projects. An analysis of the

staff transition from a project growth to maintenance mode of operations.

.-(1iscussions Lfcia the three previous report's with appropriate updates or°

ch 1980, Final Report: /)

, ° k ( -,

, field implementation activities in the "site history" project sites and

key ,policy and administrative decisions between April and June, 1979.

A full discussion of -the tyracuse political culture and its unique

impacts on the implementation of the ?CS program. A full explication of

key findings for replication purposes with particularattention to the

implications of these findings for a National 'Youth Service. 14

),>

t_.

4

a 168.-

Li r) 1.,
Q

-Periodic report on the selection of staff for orientation and development,

on the recruitment process, on the process of sponsor development and the

involvement of YCS staff in,project development, on the types of tasks and

activities that volunteers are performing, on the service-leqning agreements and activities of the volunteers, and educational insAtutions,

and on the project supervisors' assessments of the YCS program.

2. The timetable for implementation of the studies being designed by the

Urban Institute will be established on the basis of the,proposed designs.

V/VI. Limitations

1. It is unlikely that data on the youth volunteers' attitudes toward volunteeriam,.ccmmunity service, and other relevant factors can be obtained

prior to their entry into the YCS/S program. To fill this vacuum, the

designs for attitudinal stWielfare likely to employ comparison groups of

youth not engaged in YCS/S. v 2. It is likely that the time involved in undertaking programmatii and

research objectives of developing and evaluating the local capacity

to operate this mcdel of youth community service will curtail the evaluation of the "'saturation effects" of YCS/S - i.e., the effects of

saturating the local youth employment market with a sufficient number

of service opportunities no provide all available 16-21 year olds with

an opportunity to serve.

VII/VIII. Possible findings7Policy Implications

The following are indeed tentative ideas on possible findings and policy

hnplicahons based on the IalTaTciaeXperience to date. 4 1. It requires a considerable period of time and edutation for a local

community to understand and hmlement a program iwolving youth which

follows neither the on-the-jcb training approach nor the income transfer

approach to ,south unemployment problems. 'Ibis implies that this mcdel o YCS could not be run nationally as a "no-strings attached, revenue

sh program, but may require a grant application and same degree

of federal approval of volunteer activities and projects (including

the identification of supervisors) .

, 2. The choice of service opportunities available to volunteers, the process

of individualized match-making between Volunteers anal supervisora,,and

the quality of the service projects available has created a favorable

impression of YCS/S among the volunteers. Some volunteers will require

a substantial amount of direction in making a match with a project.

Also project deVelopment needs to be coordinated with the level of skilols

,among the volunteers being recruited, especially those with inadequate

basic skills.

- ' 4 3.,abe,quality of supervision given the volunteer is likely to beta critictl

factor in the volunteer's service experience. The studies shobld attenPt

to define'the personal charaCteristics °foe good supervisor as well as,

the characteristics of an organization conducive' to good supervision.

a

0,

rte

It4
4

V

IX. Research Priorities

The following three areassare of general interest to those engaged in youth

policy. The research issues surrounding national youth service are not

included. s\ 1. Identification and searching examination of succesful youth projects.

What is "success" and why was it achieved? Haw-can accomplishments

of youth be conveyed to the adult population?

2. What are the expectations of both youth and adult sectorp c the population about the constructive roles which youth can play in this society?

How can the generalized feeling about youth as a "problem", be turned

around?

3. Identification of the personal attributes and motivations of effective

supervisors of youth projects. Identification of the types of administrative and ,organizational structur* which are conducive) to_ effective

supervision of youth projects. The development of programs to train,

and support "non- professional" supervisirs of working youth.,_

Y40 - 170 -

eat

411
SERVICE MIX ALTERNATIVES DEMONSTRATION PROJECT

Drid Gottlieb

.

I I. To demonstrate, the relative effectiveness and thel

longer term benefits derived from three service

mix models designed to increase the employability of

this' youthful target population.

The major goal is to learn which, if any, of the

three mix models generates greater benefits in

terms of costs; attitudinal'and behavioral change;

employability and employment retention.

II. In controlling for age, sex, ethnicity, employability, and pre-program work experience 'will

significant outcome variations occur among and

between enrollees in each of the following service

40 mix models:

A: Labor Intensi/e: 35 hours per week of on-site

job training and work experiencd.

B: Supportive ,Employment: 35 hours per week of a

Combination of classroom type occupational

/training, remedial education, and guidance/

counseling. 0

C: Supportive Service: 35 hours 'per week...

a combination of on-site work experience and

supportive services (remedial educatidn, and

guidance/counseling). 4 '4 Three sites will*be utilized for this demonstration

project: Broward County, Florida, Oakland,

California, and Los Angeles, California. A total

of 900 youth will participate in this demonation

project --- 300 in each of tpe three siftes..*

Assignment to the various trailing models'will be

based upon 'random assignment of participants thatched

by ay, sex, ethnicity, and academic sta.p4: Applicants willbe informed prior to assignment that ,they,

are to be involved in an exiwimental/demonstration

/Droogram be4reguired 't.o participate in a

series of est/evaluation efforts., Every effort will

14 made to.,assure commonality in the attributes and

characteristic8 of participants assigned to each of

N, the. program mixes.
Multi-variate 'analysis will be utilize ---noting

relationships betweeln participants background

character.stics (See II above) and the various

dependent variables utilized in the evaluation

effort.

IV. Total project duration will be 24 months---alr

project sites will follow the same research/

evaluation time schedule and will utilize the same

pre-post instruments. Among instruments to be

administered are the following:

Vocational Attitude Scale

. Job Knowledge St Job Holding Skills

Work Relevant Attitudes Inventory

Zob Seeking Skills

Sex st4reotype of Adult Occupations

Self'Esteem

Academic ProficiencyTest/Evaluation/Data Collection will occur at the

follbwing time periods:

'.a. Determination of eligibility'selection to program.

,b. Assignment for program` participation.

c. Mid-point of program (26 weeks).

d. Program termination.

e. Follow-up post program:

--30 day s

--90.days

--'240 days

0

V. LAK,..pf unif ied control of selection/Assignment process.

4

6

Lack of unified control in administration of test/

evaluation materials.

Lack of available instraments.which are adequate in

accounting for variations.in ache, academic proficiency,'

ethnic /cultural background.

Lack of instrumentation which would tell something.

about variations in.projectsocial climate; staff

variations;"process-interaction

0

S - C

4,,
S

ib

t

ti

VI. There are always unanticipated events and gaps

between design expectations and realities. Every

effort possible is being made in dAer to enhance

the quality and integrity of the'evaluative effort.

VII. None--far too early.

VIII. To learn whether or not program mix does make a

difference. Basically, to'show whether or not work

experience alone provides benefits for low income

youth equal tokor superior to other service mixes.

IX. (1) More care/time in thinking out conceptualization/

design.

(2) More pre-project effort in the selection of

test/evaluateve instruments.

(3) Greater understanding of reasons for the too

frequent conflicts between program operators and

research/evaluation people. '

O

7 173 - 18

1

tIA

fe
An Economic Analysis of the Patterns-and

Trends in Youth Unemployment *,

Richard B. Freeman and David A. Wise

(Co-Principal Investigators)

1. Overall Approach and Purpose

Time series and cross-section data on individuals and groups are

being analyzed to determine the characteristics of the youth unemployment

problem in the United States and the causes of the postwar increase in

youth unemployment rates. We hope to draw conclusions that will help in

the formulation of policies directed toward the employment and unemployment

problems of youth. The principal goal of the researchis to explain the

factors that underlie trends in youth unemployment and employment and participation rates, particularly of black teenagers. We are analyzing several

supply and demand forces that can be expeed to influence teenage unemployment, such as demographic changgs in the relative4numbei- of teenagers,

movement of jobs from the inner cities to suburbs, transitional adjustment,

from full-time schooling to full-time work, and the minimum wage, among

others. We are in the process of determining the wags in which various

characteristics of youth employment and unemployment, such "0,movement into

and out of the work force, are associate with the high rates of unemployment

and of obtaining information on how the young unemployed finance their consumption and spend their time. We are nalyzing time series data, crosssection data, and cohort and 1 itudin 1 data to estimate the impact of

several diverse factors on youth unemployment rates. An effort will be

made to identify geographic areas with especially low rates of youth unemploymtnt to obtain clues as to the factors that might reduce rates elsewhere.

2. ,Questions to be Examined

Several,, types of data'are being-used 0 anal/);zAhe trend and charteristics of the youth unemployment prablem,Tevalilaty:he importance of

v rious supply and demand factors in the determinAtion of the high and

crdasing 'rates of unemployment among young workers, and to seek an'understanding of the micro-economic factors that affect the rate°of unemployment

of young'worl.ers. The specific re14earch'objectives are:

1) To .quantify, the ,thajor characteristics and trends youth unemployment, employment, and,p'articipation rates.. Such quantifi tion is necessary to document carefully the nature ot the problem both ovVall and among

persons defined by various characteristics such as sex, race, and education.

One i ?ort:Int set of i5.,-.1.es rvl:It(-.s 40 differces in. the Ine-ploymcnt and'

011.11o; mint-of young persons sewleing part -thie bpposed to full -t'1 lac work

. - 174 - 1 r
1

and to tto labor market participation as opposed to unemployment among ttie young. Another :s to do with the gross fuel s's or tUTnover in youth unemployment, the shfilme-iuration of youth unemployment, and the extent to which changes

in turnover patterns may have contributed to trends.

O 2) to estimate the quantitative effect of supply-side forces on the

level and trend in youth unemployment rates using time series, longitudinal,

and cross-section data. The principal. supply side factors to be exaimed are:

labor turnover initiated by persons searching for jobs; attitudes toward lowlevel 'secondary' lobs which might lead persons to reject certain types of

empldyment; family background factors, particularly the.role of parents in

obtaining jobs for teenagers, and of the broken home on black teenage possibilities; the increased number of young persons on the market due to high

birth rates in the late 1950's and early 1960's; and problems in the transition from school to work.°

3) To%estimate the quantitative effect of demand-side forces on the

level and trend in youth employment and unemployment rates. The principal

demand side factors to be examined are: the indukrial and occupational

_composition of the economy, which may have changed adversely against teenagers____

in recent years; the minimum wage and other 'rigidities' in the wage structure,

which would reduce their employment relative to what it otherwise woUld.be;

`decline of jobs in inner cities where black teenagers are concentrated;

substitution of female older workers for young persons, especially on Part-time

jobs, induced by the increased participation of women in the labor force; and

of the potential substitution of older workers for teenagers as a result of

potential legislation striking down mandatory retirement la

4) To determine the distinctive factors influencing the unemployment

and employment ratesof specific groups of young persons, namely blacks and

women. With respect to blacks, the problem is that the rate of unemployment

of young blacks has risen relative to that of young whites since the mid P 0's

and early 1960's. There h as bets serious analysis of this pattern et

and, more geierally, relatively little information on the status of black teenagers. One reason for the lack of information is the "negative" nature of the,

unemployment data: it tells us that people aren't employed and are looking

for some work but fails to tell us wh,,t they are in fact doing. tVp..rincipal

goal will be to relate the increased"rAtio of black to.white teenage 111,.m0loyment rates to such factors as the shift of jobs to suburbs and to how unemployed

black teenagers use their time. With respect to women, we want to explain the

extraordinary increase in the labor participation of young women sin recent

years and, in particular, nptint thelrole of attitudinal changes in this

pattern. 4. ,

. 5) To examine the decline in unebployment rates as a cohort ag4s,

thereby casting light on the causes of the high unemployment among the young.

The key question is whether anything beyond pure age and time underlies the

drop in noemplw.,7,ent fr6m 2'0-40 percent anong 16-19 year olds to 7-15' percent

1011/V arong persons in the 20-21 tar old age b "et.' If any.factor.beyond age

were found to contribute to the drop in rates, we would have an important clue

to the structural factors behind the high unemployment ra,e and possible suggestions for policy. An important part of this objective iS to investigate

the possibility of changes in cross-section or cohort/longitudinal unemployment by ave.`

175 -

'14 18(6)
4i 6 Th4 final objective of the project is to produce two related

reports: one lengthy technical report embodying the formal research studies

undertaken; and a less technical and shorter report designed to pull the

findings together to form a comprehensiverpicture. The shorter report will

be written,tor a broad general audience and will be intended for wide dissemination.

0-

1 . 3. Project Design and Analytic Method

The research plea for'-thisqroject involves the work of several

economists, who are using'various.date sets to answer the questions of concern.

Most of the work involves' detailed statistical analysis (described in Table 1)

supplemented by discussion, with personnel officials and others involved in the

youth labor market: While each of the analysts is developing his own specific

mode of examining hit topic, tbfre is considerable interplay amo4 the researchers through frequent group meetings. A major effort will be made by the

project directors to distill 'the findings-into a unified picture.

The use of a varjeiy of data sets is an important feature of the project."

The data range from aggregate time series to cross-sectional data for SMSA's

to longitudinal and crosk- section data on individuals. While our ultimate concern is with explaining'ehe tkendfand level of unemployment rates among the

young.'; the standard problemS witti'time series data makes it incumbent on us

to conduct our analysis with a Lill set 'of individual and aggregate data.

Cross-section and individual data sets an be used to test the effect of

variables found to be important in the, time series analysis.

The data sets to be analyzed are sh9wn in Table 1. The principal

characteristic of eadd body of data is tivr1 in column 1. The way in which

they are being used is set outiin column 2.°

. 47 Interviews with personS)i volved ip xouxh labor market wi be

made at various points in the project. In addition; an effbrf will be made

to bring 'outside' knowledgeable persons to some of,rhe Bureau meetings on

youth unemploy"ihnt. It is hoped that this will help us Co gain insights that

might otherwise be missed and to provide us with checks at) our interpretation

of results. We will condudt our own interview survey of the allocatiun,- of

time of black unemployed teenagers. The survey will be conducted on a'.small,

scale in Boston. The purpose of the,survey is to gain insights into behaVioral

patterns, that we would be likely. to miss through analysis only of 'rriore'remote

data sources.

By examining the potential causes bf the high and increasing rate of

unemployment of teenagers with 'several different types of data and by,bringing

the insights and suggestions of nonacademic persons to bear on the problem,

ue anticipate putting together the most cc-plete and valid picture of the

:outh labor 7.arket ptobler- that has been atte.Tted to date. 4 0 t

The research work will be subdivided into several separate reports,'

each. to be conduCted by specific scholars, and distille4..into'a final report . , 4 by the project directors. The principal specific projects,are listed below.

. . . , . , .

, , .

. 4.

a

t 176 t

I,
Name

1. NLS Young Men 14-24

in 1966, followed

through 1977

2. NLS Young Women

14-24 in 1966,

followed through

1977

3. National Bureau

time series

data bank

4. Current Population

Survey, March J969-

1977 ,

5.. Current Population

Survey, May 1976

6. NLS, High )oo

Class of 972, and

follow- ps.

Table 1: Data Sets

Characteristics

J

Detailed job history and

personal characteristics

`ofkindividuajs; longitudinal file

Detailed job history..and

personal characteristics

of individuals; longitudinal file

Basic time series data

on labor foxce participation and employment

by age

Cross-section tape with

data on annual income

and standard CPS.charac7.

teri.stics oworlcers

Detailed questions on job

search'and wage rite data

as Use

Will be major source of

data for examining detailed characteristics

of unemployed and employed

youth and fo evaluating

changes in employment

over time f individuals

Will be major s urce of

data for exa ing detailed

characte tics of unemployed and employdd yOuth

and for evaluating change's

in unemployment over time

for individuals

Key input into time series

regression analyses

Will be -used to examine

'trends in teenage unemployment, related to

charactefistics,'and

family income

.Pormits'analysis of meithods

of job search by teenagers

4

iriforn6tion on Major data source for study

ooling of young persons, of school-work:transition

with lonOtudinal follow-ups

7. In erview of . Small scale project de-`

i mployed TeiA4g.ers sispell t decurMine allocanInne.KtOty-.4Y' ti orb of time cif black

!

.),.. 2 teepaters, '

A ro , . ., . .

.4. .4° '8., Cenlus of Populationt ,Data:on unemployment,

4 -4195'0, 1960,,.t1, employment arid Other

/

-; ° , 4 -*faciogg by .Age, for de- ,

.

. t 1.t.aile"d, grraph4. .5/ eas

0

9. Gross, flows: data

from CPS 0,

44

DaedVpvement in anal;

,

out ,o work 'forte by. dt

aivid a .

Provide ,insights into job,

market problems of black

inner city 'teenagers

o 14' k Prime data' source sot.cpm- , pafing teenAtie,unemplo9ment

irrates ac.1-ps,,s.S?ISA's .ani

-0.vati5 in areas with dig

::.f t .1.)v et 43.

,4 Wi'l he' tie-4. to invest'd

, charaoteKisiics f. Menage

iink° lo -TrAn tA, r

a

n u st., rIP

.71k -10'4% v . ;

it , ..,:- I. 1 .

% I. a ar*

I .., 4 A -4,6. 4.

. . 4. °A. - k

o -A ..A . - 17'1

t9;.4L:
a,

. 0

Listof Reports and Authors

,

: Report . 1 Trend Patterns Characteristics in Youth I. Ellwood ,1---/.

Employment, Unemployment, and Conflicts Between- M. Feldstein

Various Statistical'Sosurces .0- =',.'.: -: Report No. 2 ,

'Time Series Analysis of Ohangesin Youth M. Wac4er, t ' . Unemployment Rites 4, II .10s4

,1

' 0. . . ? ;

Report No. 3 'Analysis.Ofjnemployment Rates opYpung'4acks R. Freeman

J. Med0if .

e,. 4 ,

. V , r .0;

Report No. A Analysis of Young Female Unemployment Rates - M. dercoian

Report No. 5 Dead-End Jobs and Youth Unemployment

Report No. 6 Implications of diograpllic IlifierencesdnYouth Unemployment Rates for" AggregateleVels

and Changes Over Ting

-..Report gO

Report. No

7 Analysis of Effecti,of'Narious Schooling.id

Scbool-TraniitionPatterns on Youth Unemployment

C. Brown 0

R. Freemaa

'D., Wise

Summary, Di..6.11ing Majpr Results 4. ftegman

D. Wise

O

n 4

"V'

4 s

t

e Ng

0

ftcr

, 2e- - 46 4,

dd.

°

g. et 1 44,

e * .

, 6

..- ,

0 ' , I I 4 * ..), ;

e t i 74 J e '' ..,

.-Iti. , I 4'''''' 1 e '''' 441, °e . v

:° if ' '7 0 T

- L78

9.0
I

1128 125 ma= mm

111 ea it 2 2

111 ipoEra ea, 11111 2.0
.The-Vroject directors keep in close contact with each ,of the authors

and maintain an ongoing review of the progress of each study. The periodic

seminars and confetances involxing all of the participants in the project also

serve as a primary'vehfcle for mutual exchange of ideas and suggestions for

the improvement of individual studies as well as the project in general. . ,

4. 'Time Table

NoveMber 3-4, 1978

May 17-18,.1978

June, 1979

#

Wo kingeonfe'rence,to discusg preliminary

research results (papers).

p Conference to present and discuss final

research papers.

5. Research Limitations

Executive Summary of major'Tesults

r

Our initial findings suggest to us that the youth unemployment problem

goes beyond traditional economic analysis. First, the probleM is concentrated

among certain subgroups living in particular social environments: high school

dropouts, black's who do not go to college, and those from Aisadvantaged families.

SecOnd,'muchof the-problem shows up in persons out-of-the-labor force, about

. whom government statistics are. noninformative: We lack sufficient sample sizes

for the groups where the problem is concentrated; have little information on the

career progress of dropouts; and, no real knowledge,of what persons out- of-the-

: labor force are dOing. Since available data sources do not provide needed

information about the Most severely Affected ,,groups, it is difficult to see

how-these limitations can be overcome in the short-run.

1 More'broadly, we believe that the interrelations among various problems

fcing the young -1- broken homes, illegitimate children, welfare, crime, drug'

abuse -- suggests that the youth employment problem cannot be treated in isolation Mit must be viewed as part- and parcel, of a broader range of social problems..

6. ca, ?

'1 Thus far, we fee l that our finaproduct will conform in all important

revects to our initial intentions. A6

o

lInmployment rates carculated from longitudinal surveys (NLS of the

1972 High School. Class) are considerably lower than those based on

('ES data.,

.--.The October 1972 Unewiloyment rate. of Spring 1972 Nigh school

seniors (based on t;,. NLS of the 1972,High-School Class),was .102;

the rate for white .-es was .054; and for black malts was .130.

179 -

1 Qts, t;),
o. -- The hourly wage rates of blacks and whites in five successive

Octobers, beginning in 1972, were about equal (based on the 1972 NLS).

-- Post high school labor forceeexperienCe is n ot greatky affected by

high%school "vocational" training. f St

." Work experience during high school has a significant positive association with both wage rates and weeks worked in the years just

after high school. , Achievement, test scores, high schoolclass rank, and parents' Income

are all positively associated with post highkchool wage rates.

-Ihttgaasignifrdaiff-varration an youth employment /p opulation ratios,

participation rates and unemployment across geographic areas, indicating that the problem *$ less (more) severe in some areas than in

.others.

Muchof the difference in youth employment patterns:by area is

associated with labor demand conditions as reflected in aggregate

. unemployment rates,'rates of growth of income in the area, and the

distritlution.of industries.

, . The youth employment problem is intrinsically linked to other social

problems. Voilemployed young persons tend to ,come from families on

welfare; in public'housing projects, receiving food stamps, and from

families headed by women.

- There is some Indication that young persons who hacve an initially

poor employment record suffer for several years from this experience%

. , \ . lip The major group suffering frdm joblessness is young blacks. Black/

white differences in empldyment, unemployient, and labor participa,

tion cannot be explained by other factors. Much of black jobletsnesg

is associated, with low labor participation rates and high Unemployment

among 'entrants to the labor market.

-- A large part, of youth unemployment is accounted for by a relatively

small,' hard core group of young people who expe4ence long spells

of unemployment. Whire.Mogt unemployment spells are short, this

,is'due.to:the high rates of labor force withdrawal, rather than to

job finding. Among,male teenagers out of school, fdr,example, over

half'ofunemployment was due to those with morethan.sixmonths of

unemployment in'. the year.

, There'is a close linkbetween the social pathology plaguing young

, 1 blacks and joblessness. Many not emplbyed black youngsters are on - drugs, er.gage in crime', and exhibit other socially aberrant behavior.

-- Demographic patterns resulting in an i'hcrease in nurdbei-of young

persons on the lalr Larket have contributed to the increasing `rate " of unemployment, and decreased labor participation rates, and has , C'

acted to depress'employment/population ratios.

180
s 8. POlicy mplications

- - Policies should be directed toward a rather small proportion of

yopth, such as black high school dropouts.

-- Policies must be directed toward bringing these youth, or more

likely future generations of them,'into the labor force.

- - While traditional aggregate demand policies will have some affect

on overall youth unemployment, they should not be expected to solve

the problems of those that are the hardest hit -- ydung.blackS.

- -.We fear that only abroad based policy aimed at changing the whole

social environment of the most-severely affected groups will substantially rffprove,theriempTaymerft-pro pects.

9. ResearCh Priorities

- - A special detailed survey of the groups'suffering most from youth

jOblessness with-particular concern to out of the labor force young

persons.

Additional survey questions,to obtain information on what the notemployed yotng persons do, how they finance consumption, how they

4 eventually move into permanent jobs.

Researchsl;OUld focus more specifically on.the groups with the

wort employment problems, and should, deal wit"h the full specttum

of social. ills facing these groups,.not just traditional economic

concerns.

- The youth employment problem should be related to perSonnel pol-icies-

. of employers.

k

I

. .

1 CI 181 "::
t

,P

The Potential Impact.employment and

Training Programs on Youth Unemployment:

(Richard Toikka)"

Teenage unemployment has been at high 1 /els in the United States n m

recent years. Explanations for this fact vary. Some contend thatttle

..: - , measured rates are misleading because full-time students are included.1

I OtilerasmgiAt that m I I I 1

.

II . - St1,11A others point to high rates/of voluntary turnover among young people

who "job shop" early in their career:

Concern-about the high unemployment rates of teenagers, particularly

blacks in urban areas, has prompted the pasage of the Youth EmplOyment

Demonstration Projects Act of r77. Thig laW creates new youth programs

and expands funding for existing employment and training programs serving

youtlz.

This paper addresses the question of what impact, this expansion of

employment and training program for youth is likely tp have on employment

and unempIoymerit,of young people. The method used Abn the analygis will be

to specify.a model of the youth labor market which focuses on turnover flows

between employment, unemployment, and school and, then, to introduce empl':oy-

.

A

rent and training programs and determine their impac't on employment and

unemployment both in the short-run and the long-run. The turnover nodel

will th Markov model discurd in Toikka'(197 ):.

111,1 IlleoretiAral Hyde] tho,,e program cha acteristics which

are ilniant_in_affecting_amployment and otemployment,: the following

. gram characterksiics will b'e 'evaluated in the model: (1) the extent Ito which

the program is targeted at a particular group, e.g., the unemployed, 1

''income, etc.; m the placement rates'a individUals.leaving the program;

- 182 -7- . 1 uP -a.a 1,1

4
V 7

41% (3)

. the scale of the program;* (4) thetiming of the program; and (5) whether

. or not the program changes the participants longei term labor market Success.

.4

Inferences will be drawn about the impacts of previous employment and

training programs on youth unemployment. This analysis will be similar in

objective to the analysis conducted by Small (19'72); however,'the data will

be Anter reted in light of the theoretical model described above. Finally,

based on present knowledge of the parameters of the labor market model and

the program characteristics of the expandid youth iirograms,v projections will

, be Agde of the impact of these programs on youthunemploympt in 1978-9.

A

A Retn,rent Ins °

A

. , .$7 ,..i Small, Sylvia S., "Statistical Effect of Work-Training

Unemployment Rate," MonthlyIyabor Programs on the

Review, September,./972.

Tolkka, Richard-S., "A MarkOVian Model of Labor Market Decisions Ame ican Economic Rev ew, Vol. 66, by Workers,"

No. 5; December 1976, pp. 821-834. .. \

.

<v- - 183

1.'U"$
The Social and ,Economic Significance

of Teenage Unemployment

(Richard Toikka and Frank Levy).

In, recent pars, both the public at large and government policy makers

have shown increasing concern over high teenage unemployment rates. Eyed

during the -low unemployment years of 1967-69, the unemployment rate for andi-

,

\ viduals aged 16-19 never fell below 12 As the economy entered the

t .

, . downturn of 1974 -75, the 16-19 yeal-Old unemplOyment rate rose above 19 per-

'

cent and the unemployment rate for certain stiFiroups--e.g., biadk7males aged . 16-17--exceeded 38 percent. In reaction to these figures, Congress passed

the Youth Employment and Demonstration Projects Act of 1977 which created a

young adult conservation corps, a program of community improvement projectS,

and k variety of other public job.programs for both in-school and out of

,achoo youth..

D_ piteithis.concern, the feeling remains that the problem of teenage

unempro rnt is not yet in perspective. How,. for example, should one compare

a teenage unemployment rate of 17 percent to a head-of-household unemployment

=ate of 7 percent? Uncertainty over this question exists in two areas. The

Jic first concerns the extent to which teenage unemployment reflects true hard-

, , ship in the short run--the inabilit)t of a household to obtain necessary income. The second area involves a longer run concern; the extent to which a

teenager's unemploYment is the firy signal that he or she will experience

Continued Ieabor market problems over his or her career.

Witt( respect to'short-run hardship, some observers ague that teenage

unemployment rates seriously overstate the amount of harm involved. In their

view, teenage unemployment involves a, good deal of job shopping and voluntary

turnover as teenagerslower unrealistically high expectations. In many cases,

teenagers can afford this leisurely search process because the=y are nupported

- 184 -
* by their parents--theirincome 1.7not crucial. to the household. Bnt Harwobd

(1969) cites anecdotal, evidence toindicate.that high reservation cages and

# prolonged search create voluntary unemployment for teenagers from low income

households as well.

Conversely, other observers argue that teenage 'unemployment rates under- . state the short-run harm involved because they fail Sat Fount high numbers of

teenagers who need and want jobs but who become diScouraged alid drop out of

the labor force [Thurow (1977), Levitan and.Taggart (1934)] . -In the first

quarters of 1976, the Current Population Survey (CPS) tabulated an average of

1.6 million persons aged 16-19 who said they wanted 'a job but who for various

reasons were not working or looking for work.

1 This 1.6 million figure

'equaled about 20 percent of the number of 16-19 year olds who'were

. . . labor force at the time. A sco.ndunderstatement of harm may-occur because

unemployment rates fail to capture teenagers who are working part

% 7im- te butwho desire o work full time. The relative magnitudes of ttpot overstatementland understatements are unclear.

The long-run implications of a teenager's Unemployment axe equally un-

. ,clear. Somd observers argue against any long-Al impact r "'scarring" by

2

pointing to the way in whidh grOttfp unemployment rates decline with age. For

t example, in 1976 all 16 and 17 year olds in the labor force had *ea bnemploy-

. -----_ ment rate of 21 percent, all 18 and 19 year olds'had an unemployment rate of

.

. ..' g ,

17 percent, 20 to 24 year olds had an unemployment rate of 12 percent and so

V '1 ' . (But the meaning of these- declining rates is ambiguous, becausewhile.

* older cohorts hive lower unemployment rates, they also have higher labor

0,

0'

1. See Bureau of Labor Statistics, Employment end Earnings, April 1976,

Tahle A-51.

1.85 9

I \-

11
.4

. - force participation and greater absolute numbers of unemployed. In 1976,

anaverage of ,8 million 16 and 17 year olds were unemployed each month as

were .9 million 18, and ;19 year olds abd 1.7.million 20 to 24 year olds. Thus

4

_ it is, possible that many of those who were first unemployed when they were

t16 and 17 could continue to have serious labor market difficulties in liter

years .even s the unemployment rate ,for their cohorts declined.

To summarize, the current prdblems in interpreting teenage unemployment

rates center on three ,questions:

a) To what extent does teenage'u

hardship on ehe teenager's fam

ment impose an economic

b)- To what extent does the teenage u employment rate arise

from voluntary supply decisiong?

c) To what extent does unemployment as a teenager presage

unemployment roblems in subsequent years?

The research discussed in these pages focuses on short' -run labor market

decisions by,young people. For data, we are using a matcheMarch/April

Cuirent Population Survey which has information on individual and family

Characteristics and can be used to estimate movement between iabol market,

states such as employment, unemployment, ,and non-participation in the labor

force.

The research plan follows a path brolsen by a numger of researchers

Meat (1977)- Toikka (1976); Smith (1977); Marston (19761; larron andMellow

(1978)) In that the short-run (monthly) changes In labor Market state are

Mbdelled using*tranSition4robabilities. The three tate,(emploAlent/

unemployment /,not -in -the, labor force) Markov model used by matiy, analysts Will

be extended in directions which are apprprilte for udying the behavior of

yoUt*. Sii labor market states will be of interest: (1) full-time employment, (2) part-time employment, (3) uneaplofment of those who lost their

J* \

186

't
t.

, 10

, ,. limit jol), (4) unemployment of those who left thAX la'st lob, (5) unemploy-

.

.

, . . ment of entranEs or. re-entrants, and (6) non-participatioe in the labor ,. ;., . . . ,. 1 , force.

.,, , ,.:,

, . . -- ,

, . .. '1. k

The first4stage.in the empirical research invdlves 'estimating the pro-

. ,-

. rility ,54 being in each otthe six labor market sta,tds as of March 1976

,conditional on a series of variables w hich:1.'111 include ageltracei sex,

\

education, faMily income, sex of family head, presence :of Young children.

previous year's labor fOrce status, area driemploymentrate, degree' of

urbanizftion, andA imputed wage. These tabuIatiOns should provide a.

summary statement of how young people-are distributed In the labor.market.

, This distribution will be analyzed yd.th the 'following qudstionAs in mind:

# How important is age relatfve to other factors in determining labor force

. - status? What generalizations can be m.;-"de about the relation between labor

force status and family income. 'To what extent is teenage unemployment a

problem of urban areas and to what extent is it IT problem for non-urbfn

areas as well.

r The second stage of the eivirical research will entail estimating models

with transition probabilities as dependent variables. At this stage, -it will

. . beettfer-to identify the'behaviort1 patterns which produced the observed

.. . 0 , , .. , .

. . -

. , - . distribution -of persons by labor force status. 4or'ekample, 'it will be " .

.. -0.. .

.. L

., . i

. .- , , .

. , , possible.to determine, whether a high (low) rate of non-participation resultresult's

.

... 0. :e from,a high (low) labor forge exit rate or a lo4 (high) labor force Entrance

i " ,

.

.. rate. In somec;es,,it wiir be possible to.go fuitherand test interesting .

. , ......

. . ,.

....,, . hypotheses about the effects of age and family income'on labor 'force parti7

, .

. cipatiort, quits and l*pffs- - .

. - .

. .

. . .

.

. . After theprdbability functions have been estimated, it will be poS---

. ) sibie to characterize the normal. labo r market experiences of young people

'1 187 - 201
41111,..r

with certain types of characteristics.

1

0

These empirical models cab offer

at the beginning qfthAs

income as an independent

substantittl insight inter the three questions posed

10

'summary. First, because the models incl derfamily

variable* they will permit an investigation of the.extent

unemployment prOduces short -run hardship.

unemployment arising from quits, the role

i h teenage

Second, by differentiating between

of voluntary decisions in dettimining teenage unemployment can be investigated. Finally, by including the

i -v.' .. previous year's labor market history as independent variables, the extent to

which early unemploymentoexPeriences signal subsequent labor market diffi-

'culties can also be investigated.

.6>

.1F

sr

.1

&

F - 188, 2uz

4
I -References

. Barron, John M. and Wesley "UneMployment.Insurdhce: New Evidence'

_on the Redipient and Its Impact," unpublishedAanuscript, August 1978:

c % X Harwood; Edwin, "Youth Unemployment - Tale of Two Ghettos," The Public , Interest, Number .37; Fall 1969, pp. 78-87. i i.. rt-

'Holt, Charles C., "Modeling a Segmented Labor Market," in Women, Minorities

and.Employment Discrimination (Lexingtpn: Lev(ngton BookA, April 1917).

,-- ;

. Levitan, Sar.A. and Robert( Taggart, Enployment and Earnings Adequacy: A.Neg,

Social Indicator (Balltimbre: The Johns Hopkins University Press, 1974).

, / ; ) I , Marston, Stephen T., "Employment Instability and High Unemployment Rates,"

Brookings Papers on Econ6mic_Activity, 1976, Number,l, pp. 169-203.

OL,/

Smith, Ralph E., "A Simulation Model of Demographic Composition of Employment, Unemployment, andLabor Force Participation," in Ronald G.

Ehrenberg (ed.) Research in Labor Economics, Vol. 1 (1977) pp. 259-g

303.

Thurow, Lester C

Foundation,

k Toikka, Rich

he

#

A.

Youth Unemployment, report prepared for the Rockefeller

May 1977.

d S., "A, Markovian*Model of,tabor Market Decisions by Workers,"

T Americ n EconomiscReviT4, Vol.. 66, No: 5, December 1976, pp. H21-834.

I

C)

r .

-2

/

o

X

4

eJ
11, MEASUREMENTiOFWORK.OITTPUTOF YOU7 PROGRAMS - DAVID ZIMMERMAN AND ROBERT EGAN

, *- MPR

. PURPOSE AND OBJECTIVES

Federal government welfare and labor policy has recently placed

.

,. -i;orhemphasis on bot public service employment and employment and

training:opportunities-for young adults. One important objectie

, of public etnployment programs, incuding those specifically targeted

toward young-peop, is the provisioh of goods and- services. Although

providing these goods.aneservices is not the primary mason for implementing

the pr9grams, differing assessments of-the vaL of the output produced

, will undoubtedly have-andmportant role in the continuing political-arid

academic debate over them.

T1,e two 'major objectives of tke research are (1) to provide Preliminary information on the value of output of a sample of yq4th employment

it

and Work experience projects, and (2) to assess the feasibility and

. t ` relative desirarity of various strategies for, valuihg the outptit of youth

projects in the future.

To accomplish these Obfectives, a number's 4 specific quegtions must'

be answered:

- i-

.- 4

. ,

4 $ .. ' 4 (i) From a pglicy perspectiVN what Anformatiokon.

the value of outputamd,other,aspects of youth

employmeritik projects should be Provided? TAis'is

partly,a theoretiTal'questiop based on standard . evaluative criterlal,and partly a prActical :' questioff`of what information WW be most useful

to policymakers and program operators.

. -(2) In assessing value of output, what are the thods-

..for the relevant'Variables? Can t ey be

.0 measured at all? How accurately? How much'confii dence.can beplaced.in'the measure? When Imca'A

, than one strategy ists, what are the advantages,

and disadvantages of each?

, What

. (1)0- a are the best. thods for eliciting and analyzing

descriptive information on the, projedts? Do the . methods vary by type-of'project? What artho he:;t.

sources of this information?

t - 190 -

204

I

V
(4) How costly is the measurement, data collection,

and analysis likely to be? How does this vary

with the extent of inforMation available, the

strategies adopted, the characteristics of the

program, and other - factors?

These quetions are obviously interdependent, andtheli-must be

. addressed-.atcordingly. The purpose of_thljtudy is to respond to them

. in a mannpr'that best,meetboth the short-run and long-run objectives

S by conducting a set of pilot, studies of existing youth employment

,°

.2Tojects. -These studies cart provide both quantifiable and descriptive

project-infortation on a preliminary basis without haying t' wait for .

, -N future research efforts. In addition, actually undertaking a series

of studies should add significantly to our knocaedge of the benefits

and costs of°-.Orticular project definition and measurement strategy

alternatives.

PRIME'SPONSOR/PROJECT SAMPLE

In order to insure thlIke have a good representat -on of various

,types of projects, we have stratified the sample of projects along certain

dimensions. A key stratification is by type of program. The Department

of Labor has asked us to include the:following types of youth work projects

in the study:

A) Existing Title i Youth Wor Experience Programs k

1. CETA in-schooiProjects

2. CETA out-of-School projects

3. CETA summer work projects

B) Existing Work Projects of-die Title III SPEDY Summer Program

CI Programs Funded Under°YEDPA

1. Youth Adult Co5oervation Corps (YACC)

2. Youth Community Conservation and Improvement Projects (YCCIP)

3. 'Youth Employment and Training Programs (YTP)

I - 161

201-U
D) Existing Work Projects Funded Under TitleS II and VI.

'7 '0 In selecting the prime sponsors and YACC'sites to be visited, we

' . hove attempted to obtain a representativesample in terms of the type of

prime sponsor (i.e., city, county, Salome-of-State, and consortium),

geographic area, and.p4ulation of the area. Our plan has been to select'

, . h$tween 40 and 4s projects at 14 sites to be included in the study. Of

the,14 sites, 12 are CETA prime sponsors and two are YACC'sites. The

sample size assumes that we will study three randomly selected projects

f at each site. At some of the sites all three projects studied will be

;1/4outh projects, while at, other sites we will study twos youth projects . and oneadult-PSE project engaged in work similar to that being done in

one of the youth projects studies:

le

Ar :FIELD STUDY TASKS

The field study portion of the study is structured by the.majc;r

takes which welist below and which follow directly ffom the objectives

and issues discussed abOve As one reads through the list, it is clear

that,pach_task has, subsequent activities associated with it, many of which

are particular to thecircumItances of the specific prime sponsor or.

project. Nevertheless, this list of-the,majortasks does indicate the

scope-of the work and level.of effort required in the study.

. 1. Determine the state of the local prime sponsor and other operating

agency program records. Many parts, of the value of output study will require

. local program data in the form ofalocal prime sponsor or other agency records.

An assessment of the adequacly of these records will_be critical for the

accurate determination of the feasibility and costs,associhted with

=.,192 -2uu
alternatiVe measurefnent strategics. Appraisal-of the state of the -

-. . .16Cal program data will identify the constraints.which might eliminate

..,

0 ' particular measurement strategies.

-)

2. Test and Assess Supply Price Estimation Procedures

A major task of the study'will be to test various methods of

estimating the price an 'alternative supplier would charge for the work

performed by the program participants. While consideration must be

given to whether one,pkocedure,canbbe used 'universally across projects

or positions, our eXperiende indicates that the feasibility of

alternative procedures will vary by the type of work performed.

Our basic plan will be to use the pilot studies to test various

supply price estimation strategies, many of which we have used in previous

studies, itt order to assess their appropriateness for the youth projects.

We will be able to assess, On the tiasis of thtlpilot studies-, whether there

are certain types of projects which are more (or less) conducive to specific

_estimation strategies6band whether there are acceptable cost-reducing '

nodifttations of those., strategies.

3. Test and Assess Strategies for Obtaining' Information on the

Demand for the Project Work.

Deriviffg estimates of the demand for the work performed is a very

impdYtant yet particularly thorny issue. A-primary question to be addressed

in t/le,study is whAt particular type of information on the demand for the

work is desired and available. For example, will it be possible to place

monetary value of society's demand for the output? Can we obtain

descript,ve inTormation relating to the demand for the work, such as the

objectives or rationale for the project, whethej t e work was performed

(o'r planned) prior to,tbei project, whether ther are plans for tiint.inuing

Att

207

193
a

it0

the work after the program ends, whether there was opposution or barriers

which emerged in the implementation of the project, either by the

-recipient or an alternative supplier of the service or product, how

Much publicity was given to the.pioject, and various, assessments of

the, usefulness of the Project. Beyond this, the form of the inforrdation must

be addressed: Can we get quantifiable data on demand for the work, or do we

rely on descriptive information? One objective of this part ogithe study

will'be td. determine whether a single set of procedures can be used for _ all assessments of aemand, or whether the Proceduresirust'vary by type

of work or some other dimension.

4. Test and Assess Strategies for Obtaining Informatonon Project

Costs and Inputs

Estimating the costs' of the work f)erformAris'extremely important

- r

1 in interpreting the value of output results. However, estimating the costs

of the project, particularly the social costs as opposed to the budget

' expenditures,.is usually not a simple task-. In many cases the program

accounting system is not structured in sudh,a way that all the costs

associated withLthe particular project or job slots under consideration

can readily be identified. Costs of adthinistration, supervision, equipment, or materials, which may be substantial, may not be included in accounting

data, or, their treatment may vary from project to project.

'The. expenditure

. data may not contain all the costs or the correct costs needed to calculate

the social cost of the work. Our experience has shown, in fact, that in

"Many cases it is thet most time consuming of all the tasks associated with

the value-of-Output effort. In 1ght of this problem, it has been

necessary to test the cost data collection strategies on a sub-sample basis.

In addition' to assessing the availability of cost data, the ,M,).4-'

ability of daba on labor inpults wiih th- pi'ojc.vt mu.lt .11!,.)

7 - 194

9
I

4 b

, assessed. Almost any conceivable use of the results of a value of

output study would require that they be put on a"per hour", "per month",

"per year', or "per enrollee" basis. Our experience on'Other program \

evaluations suggests that this conversion is both important and

potentially error-prone. This task, therefore, will include'an examine-

, tion of -both national and local'data on participant labor inputs, and the

conversion of the value of.output estimate to a per period of service or

y per enrollee basis will be attempted for every project studied.

5. Investigate the Determinants of the Value of Output and Costs

Another important part of the project studies will be to obtain

-information on the determinants of the value of the proj9ct output" These

determinants could range from the overall motivation for the project to

suchthings as the eligibility criteria for the participants, the work

-technoloy and slow, various aspects of project supervision, disciplinary

4 _-problems,!.end even the amount of visibility or publicity given to the

poject. .WhiIe it is unlikely that the information elicited will be

O °.sufficien: to Hermit extensive project -by- project, comparisons in this

arpa, it should' be possible from the pilot studies to generate some

hypotheses concerning the. determinants of value of output and to better

de'fine what types of data oh value of output determinants are readily

available.

6. Determine theRepresentativeness of the Projects Studied'

In addition to the individual project studies, information is being

collected,to determine the representativeness of.the,projects

in terms of the type of work,performed and any other 'dimensions which may

vs be relevant in interpreting the results of the pilot studies or planning

future Vhlue of output, research efforts. While it is highly unlikely tnait

20a-

. - 195
a sophisticated

projects can be

there should be

of the projects

analysis of the representativeness Of the studied

undertAen within the limited scope of the study,

data available to make at leas:_a crude comparison',

studied to other prbjects at the site.

SCHEDULE

.The study is scheduled to'be completed in,a ten month period.,

It began in March, 1978 and is scheduled for completion,in January,

1979.' The primary, design phase was completed in March

minor revisions made after the completion of the first

with some

set of project

visits in April and May, 1978. All field studies are to be completed

I, by early October, with the summary and analysis of the data collected

to be undertaken during the months of October and November.' A draft

of the final report is scheduled for the end of November.

0

-cr p

- 196 -

Ta.
O

i

'Project Overview Statement 1/

Continuous Longitudinal Manpower Survey (CLMS)

'of YOPA Programs

by Seympur,BrdndWein

Director, Office of Program Evaluation

'Employment and Training Administration

. The Continuous' Longitudinal Manpower Survey (CLMS) has

been tracking an annual national'sample-of participants'in

CETA decentralized programs.' It is now being expanded to

include a sample of enrollees- in the ,two YEDPA formulafunded Programs: .Youth Community Conservation and Improvement P-rojects"(YCCIP) and the Youth Employment and Training

Program (YETP)'. ? ,This paper briefly reviews what the CLMS is, the basic

pur0Oses of extending it _to YCCIP and YETI), the timetable

involved, some limitations, and several issues in such

coverage..

1 1. 'Background on CLMS: 2/ The CLMS selects and obtains

detailed 41.fopmation from a national sample4of participants

in CETA,P"Ograms.rpn by State and local governments. ("prime

sponsors ")'. The sampling and data collection are conduCted

by the Census,Bureau, with technical aid, basic analyses,

and reports provided,through a contractor (Westat,

under the direction of ETA's Office of Pr.ogram..Evaluation.

The sample is'designed to provide national estimates;

it'does not provide infqrmation-on individual prime sponsors. In FY 1972, the national sample,,drawn from 147

sponsors, was 18,000 new enrollees. This included a subsample of 2,000 in youth work experience activities

(including summer youth work programs). Thd remainder

of the sample is drawn to represent the other' major CETA

activities, which are geared largely to aduats (public

see' ice employment,' classroom training, Vn:Lhe,-joh training,

and adult work programs), but it includes '(and provideb

separate data fo the youth enrolled in such activities.

1/ Prepared for "Knowledge Development Projects Cdnference"

of Office of Youth Programs', October 5-6, 19713.

2/ A technical report on. the CLMS, available on request-from

the Office of Program Evaluation, describes more fuhy its

sample..4esign, data collection procedures, and statAtical

confidence levels, and prt5rOnts its data Collection

instruments.

- 197 =

211

.1
Procedurally, at the end eeach quarter, the CLMS

draws the sample for that quarter in each:of 147 sponSor

areas, checks the sponsor records, on each enrollee in the

sample for information on the activity in which enrolled,

and,then,finds'and interviewsthe enrollees to get detailed)

information.from themon personal and family otaracteristics, on-employment, earning, and school exper nce in the

year preceding entry to the program, and'on their attitudes

to the program.

About 9 months later; the CLMS follows up both to . check sponsor records on the services provided to the sampled participants and on termination dates of those who've

left the program by then, and to intermiew-the participants

on their'continuing labor market experience. This has been

the final interview forthose in the youth ,programs, but

those in theoadult-oriented programs are followed up again

at 18 'months and at 36 months after enrollment

...CLMS Coverage of YCCIP and YETP: The same design and 4 procedures are now being used to develop information,on a .

sample of FY 1978 YCCIP and YETP participants,: for FY 1,9,79,

additional questions on youth attitudes are to be added to

thesurvey on an experimenall basis.-

The sample will be 3,000 a year'in the YETP activities

and',1,00T in YCCIP; the 3,00.0 in YETP is asshied to be s.uf- . ficd'ently large to enable sizable subsamples for in-school,

as against out-of-school youth and for work ekperience , activities as against class or On-job-trainingt

. Purposes of YCCIP - YETP Survey The principal initial

purpose i% of course to determine, in detail not available'

from the sponsb? summary reporting system, just who is being

served by the two YEDPAprograMs. The initial interview

will provide extensive information on the demog aphic and .

economic characte sties of those enrolled. al So, will

permit4comOhriso of the maketip of the youth .enrollees

other CETA acti ties as against those. in thetwo YERPA

programs:

A second "purpose is to get the views of participants

on the .prograii. The CLMS asks, while they and still in or

shortly they have left the pregram,'ada'again upon

followup, about theiromsatisfaction" with theopprogram, whdt

they most liked and liked, and views as to possible

improvement.

- 198
A

°,1

al

t2

.A third purpose, from the followup tradking, is to

determine how long they stay in the 'program, whether there

are notable distinctions .in who ,stays longer versus leaving

early, Teasons,tor leaving, whether they remain ,in or re-. 'i

turn to school, andwhat their early postprogram employment and earnings experiences. . , * ,The data will alto enable identification'of major

patterns of differences between the two YEDPA progra4s, the

major YETP 'components, and other ETA activities with large , .,4, youth enrollments.

Timetable: The initial focus.is on data to be available

for the March 19.79 report to Cdngress on YEDPA.

. . " 1 The data collected from sponsor records should be avail-'

'- able for the*full tample of FY 1978 enrollees. ro The data from

the the personal interviews will be available for those enrolled . through 3rd quarter, FY 1'978. (Since interviews of those

enrolled in the 4Lb quarter of ,FY 1978 will not be completed

until ,December 1978, it is unlikely that processing and_

analYseifof that data can be compteted in ime for a, March

1979report). Still, the data that will be available should

permia comprehensive report on just who was being reached

and served by the YCCIPand YETP prfOgrams,in their initial

year. , . - , 0

,. illir 0., .. . Thereafber,,in;49rmation from the first (baseline)

interviews, conducted with enrolleeS after the quarter-in

whibh they enrolled, 'should be available pproxir*tely 9

mohthsafter their entry quarter (about °5 -6 Months from 'the

end-of_the interviewing period). Information from the

followyp intervies.should similarly: be available' some 6'.

. monthsiafter end of the interv(ieuprter (although probleMs

of constructiv Mergedpre7post lonq?..tudinal files and of

.developing suitable analytic formats;probably will require , a longer interval .for .the first follOwup analyses) .

Limitations: One 'limitation of-the CL.MS data, as . 'indicated in tht notes above on "timetablet" is its timing:. . there is an inheV.nt timalpg before the basic data can be' , -provided', and mofe time, is necessary fOr any%complex.

_____ analyses. .. . .... ...-

.

, - 4i. Specifically, since the survey activity does not start

bill after End of a quarter,'andtthen requires- sampling,

. checks of the sponsor records, finding and'interWrewing of ,

-...2 , .

\

199 :21:-;

II

4
V

'

, ;

'4 i v-- ,

_ . .

.

.

. .. .

. . . the enrollees, processing.,of the questionnaire forms for . computer handling, development-Tif 4ta tapes, dfiNelopment

of tables', and then,analyses, it iS Tt.0 9 months:after

the end "of:'the enrollment qUal-ter befdre basic analyses

ordiriarily..0n bd .available% .'.- . ,, 0

,

. .

... 4

4

' .

'. ,.. There are various 'technical limitations..oe'reliability ..-

. and,consistency On some data items because of wide diversity

and some.etraticness in sporisor recordkeeging practiceS (on

v- recordinti of termination dates, for example, and on "identi-

_ fication of activitices or services provided)butihese'"

should fr,seriously limit the utility of the bagic CLMS

data. . . t

2. - ett

: 1

- Issues: Three major design issdes have arisen which

warrantcomment. . .7,.

, -

. , . , (1) ,The Office of .Youth Programs, believes'it'would !Ye

desirable to develop a "psychological " profi4 of YEDPA

participants and also to try to:gauge whether certain` attii

1 4 tudes change after program particiliation. It developed

questions about,perceptions ofiwork,-,6ccupatiOnAl aspirations,

perceptionsof future oppbrtunities, self-est:40m, and other

"i. attitudes The4qUestionsare similar to those in the Parnes

.

'' Longitudinal Surveys or in several other youth AA:lies. , . 4 . 4

. The Office of Program Evaluation and the CenieuS

Bureau were dubious about adding these clilestions, a bit

. because of.Concerns about utility, but more .so heca6se Of

'concern about ,their effects' oh answers to other,questions

and on,'-response rates, as well as concern about bias where

. 'parents Or others are present during the intSview 1,-,, The compromise deisiOnhas been to add such questions

, on an experimental basis for.,the'iY 1979 sample. The ques-'

1), Lons (assuming OMB approval) 04.11 be asked of .half the 4 enrollee sample, ws a means-of testing the effects of such,

questions, getting the information if it proves:praCtical (

to.do,so reliably withOut adverse effects, and redpcing

the risk of adverse effett on the entire sample:if i't turns

out that the questions do significantly,alter responses and

cut -the response rate. 4 , - v:

.

. , .

. ,

... , (2)-A second issue is the timing of the followup interview

and the number of followup interviews.' Atpreserit,.the .. first CLMS followup interview, intended.to provide early

information on initial postprogram e'ilperience, is approximately'9 months after enrollment (about 6 months after .the

initial interview). This 'limited period between the initial

-,----\, - 200 -

9 1 0

G

4
416

4

. and tkie second interview has probably helped maintain

response rates and eased recollection problems. But on - the negative side, the, limited, time before the followup

intervieW has meant that many ormost enrollees are still

. .in the pogramWhen reinterviewed,:so that initial postprogram data isjust for a reduced sample, acid only for

relatively earl leavers: .

--;-

1 7

Consideration, is therefore being given, to deferring

-.the fitst.foll,dwup until 12 to 15 months after enrollment,

to get at least 'Some postprogram informatioll on virtually

YEDPA enrollees._ This presents logistic proiilems

'for the Cerisus,'hQwever, which will have field scheduling

problems if different CLMS'fallowupaintervals are used for'

YEDPA as against other CETA enrollees. A. likely answer is

that the followu interval 'may be changed to 12-15 -60nths

Lr all of the LMS. v ,.As to the number of followup interviews, the YEDPA

enrollees are tentati .vly scheduled 'tb be followed up

twce (compared to%just once for other,CET4youth tiro rams i

and three times for,CETA adult-oriented Programs). Th

-second followup would be 18%months after enrdllment; if

the first followup .is tie'ferred from the print` 9 months

to &new 12 or 15 months, it will make sense to defer the

Second followup to atleast 24 months after enrollment.

10. W helzher to retain the_planned second fdflowup or add

-a third followup will be decided at a later point in light'

of experience 'end findings of the,first followup.

, - (3) Finally, th'ere is 'the bas&cissue of how to estimate

net impact of.the'YCCIP and YETP programs upon participants.

The.CLMS .followup data will present:gross information.on

their ,postprogram experience, 'but will .n'o't be. able to detertine how much of any level,, or any pre-eo-post change, is

ateributable-to.program participation rather than other

,factors.

For the, adult- oriented CETA pro(4rams, the CLMS sdeveloping d comparison group from the Current Population

,Survey, and will cdMPitre its Sdcial- Security earnings with

those of-the CETA participants. Tf-ie differences in nature

and extent of earnings changes for the two groups will,

indicate the net impact of program participation on part-

. 'c1pant earnings. -

4

4
(

0

A similar effort is not being planned specifically.

.for.youth, program patticOants beqause the unsettled movement of youth in an out 6f ,school and in and,obt of the labor

force severely limits the meaningfulness of any earnings

: comparison with a constructed comparison group and requires

'a longitudinal comparison at 14,stlsothe years into adulthood.

At the moment, general. plans are to explore whether

youth comparison groups developed for other yopth clogram

evaluationi'most nOtably that of the Jobt Corps progtam

and of the Supported Work program,7as well as the CLMS CPS ,... youth group,-might serte in combination as sensible q-tfer-s

ice points" for estimating net effectsiofYEDPA on its

participants.

I

Ow.

21G

- 202 -
A PATHWAYS INTO THE WORLD OF WORK: EXPERIENCES'OF YOUTH

THE NATIONAL LONGITUDINAL SURVEY OF YOUTH , The Ohio State University

Principal Investigators - Michael E. Borus and Herbert S. Parnes

, Overall Purpose

7

\.

The Office-sof Youth Programs, ETA, United States Department of Labor, has

Thnded an extension of the Natiorialiongitudinal Surveys of Labor MTrket

Behavior to cower a new sample oe ybith. Annual interviews Will be conducted

between 1979 and 1985 with 12,000 young men and women, ages 14 to A21. T/i4

-longitudinal examination of young eople, which will pay particular attention

to those groups with above average labor market diTriculties, seeks to provide

\- thqOata to answer many of the urgent policy6questions dealing with the movement

ofVouth from the classroom into the work place. Among the ieveg'to be

studied are the effectiveness of Department of Labor employment-`and training

programs, the factors affecting a youth's educational experience-and educational

attainment the det inants of the sop or jobs into which the youth will move,

and the facttpr marital, child-bearing, and .familial decision-,

making. S This,new study i s an extension orthe National'Longitudinal Surveys whiChhave been conducted since 1966 by The Ohio State University Center for Human

Resource Research under U.S, Department of Labor contracts and which.have resulted

in the publication of 18 research monographs by the Employment and Training

Administration and over 200 articles, theses, monographs and special reports.

Fot the past decade the National Longitudinal Surveys have been one of the basic

data sources for social scientists - economists, sociologists, psychologists and - educators - who have sought to tackle employment and training .issues. NLS

data tapes and dociamentation are available at over 150 institutions in the

U.S. It is expected that data generated by the new study will allow researchers

throughout, the nation to develop solutions to many of the problems which face

youth in bridging the gap between formal educatioq and training and the

establishment of relatively stable attachments to given types of work.

Basic Research' Questionl.to be Exknineu,

The vast amount. 0 t. 'kJ L: that wil,' u tim ,t .. ononfied by W. Ill'W

I . ilt, . 1415.tiona] Lonv:itud.inaL (.1.r1.roy or You,,h w,H o'r?,i I'Leraj, n,.:fdrecis of

questions to be answer 0d. i(:i.ow ;l 0 i .1oLirTtl,11 To't, or ,t,v0:1 r' :,et.irch tozicl;

that we propo., to Jtady. i . Evalrtatjon of Denartmern-: of Laver .:11L l'rogn,ms. The

Youth Employment and DemoLstration 7rojects Act ad(,ed several new programs

to those designed td upgrade the saki Lis of ul.cop vcd, wIderemployed, and

economically isadvantaged youth.- It also funding; for youth programs

provided by ETA prime spolisons. In all,. app.ixima,,e_ly 2 mill ion youth are.

,likely to be enrolled, annigally. In ishe we estimate that 1.,500.youth

will participate in one or more employment an tNlining programs, while almost

2,500 yOtAng persons w41 be eligible but will not participate. We propose to

use these groups to conduct two type's of evaluationt, of these programs.

2 3 2 1
r

.

. . Aft C1) ''he first ty4 of evaluation will study bile proaess of the programs.

Participants will be asked their reactions to the program in which they parti4-

pated, ho w it helped them in the labor market, and what facets of the,prOgram

could be improved. Responses will be_related to the individuals' characteristics

and backgrounds and_to the specific services they received in an effort to discover,

, why peolale choose to participate ,in employment and training programs what

`-s*rvices are bei,q provided-l'-how well they are received, who drops'out of

programs, and wieeigght be dope. to improve the programs...

. ..". 0

. ,

. (2) The second type of evaluation will study the impact of the programs 1

by comparing the post-program labor market eptperiences Of participants With

those of a comparable group of youth who were eligible.to participate but who

did not: While the time period since completiOn of the Progivam May b.e very

brief at the time of the initial survey in 1979, the longitudinal nature,of ' of the study will permit estimates of the gains from participation over time. A

,4

wide variety of0ependent variables will be examined, including earnings,.

welfare and other ti-an4fer payment receipts, weeksof employment ,a,tkl unemplOy- ..,, Ment, aspirations, job aelcisfaction, quality iof working

i ng conditiovs, length*- %

.... of schooling, on-the-i.job training received, and job-search activities..

- i . . .,

.

.. Factors in 'Educational Programs. The proposed study will yialikdetailed,

, information on the progress-61Nyouth in high school and college which can be

used to provide answers to a number of policy-related luestions"concerning

both the causescand consequences of dropping out of school:.

S":1 (1) Why do some school students complete their eduCation(qhile

others (withdraw without.Waining diploma? What is the relative iliPortance

of such factors as diffences in ability, differences in mOtivatibn, and

differences in the eeonoMic status of the youth and their families?

I t- (2) Are high school dropouts at a disadvantage compared wIth-tigh school

graduates in terms 'of earnings and occupational status In thei hrst jobs?

If so, do these differences narrow and/or disappear over time, or do they

-1, persist? 00 .

. . \

(3) Is there any

, eridence-rthat'private and puOl:c employe use the high

school diploma as a screening measure to classify app.licants' pential produc-

.qvi-Cy onlhe j

, ab? - Transition from School to Work. Fm a.policy pen3pective, the high

unemployment dates among youth during and",4,netne sever 71 years following the

termination of schooling and the high rates of mobility during the early postschool years suggest the need for a better understanding df the factors associ-

,,ated w4,,th relatiVely smooth and rapid entry into satisfactory and satisfying

employment situations. For example, among the questitns that may be addressed

A

are the following: . s (1)What relation ip is there, if an;i, betwe,:l .1,11,J high schoo l experience,.

, , , cif

inci:Cing.wOrk activity, and the postschool'labor : .....-:,, activity of no

col ege bound youth? Do youngster ith work exp.:f',,,ce do better, othu

thin equal, after they leav nool? lAes.the ir.-wer to thts questittii-.

. depen on the extent and/or 'of worls. experience? , .; :4,, 4i

- *:

2 1

2'04 -
(2) Is there evidence of a "dual 'labor market" for noncollege bound youth,

such that individuals with certain socioeconomic characteristics are likely

. to enter and to be trapped in low paying and demeaning jobs in which traditional

human capital variables appear to be irrelevaat; or does the labor market

operate so as to sort individuals out among jabs pretty equitably in terms of

their productive capabilities?

(3) In the early postschool years, what are the processes of mobility,

postschool training, work experience, and modification of goals that result

in youth settling into career jobs?

(4) Does unsatisfactory experience (e.g., extensive'unemployment) in

the immediate postschool years'leave "scars" that affect later market

behavior and experience, or are these problems essentially transitory, with

no lasting effects?

(5) Are there various avenues of advancement for youth during their

early exposure to the labor market?'

(6) What is the character of the extensive job changes made by youth?

Does "job hopping" result in progressively better jobs along all of the

dimensions described above, or does it simply represent a string of equally

poor employment opportunities? .,How does the answer to this question vary

depending on sex, race, ethnicity, and the socioeconomic and psychological

characteristics of the youth? What is the effect of variation in the

economic environment?

(7) What kinds of jobs do youth consider desirable, and at what wage

- rates? What differences are there in this regard between those still in

school and those who have ended their formal education? What is the extent

of variation in this regard by sex, race, ethnicity, and socioeconomic status?

Racial, Sex, and Ethnic Differences in Employment and Earnings. One

Of the principal purposes of the NLS study of youth will be to examine racial,

sex and ethnic differences in employment and earnings. At a descriptive level,

gross differences in employment arid earnings among various race, sex, and ethnic

groups will be identified. In addition,, multiVariate analysis.will be used

to ascertain the underlying factors responsible for these gross differences.

Differences among various race, sex, and ethnic groups in .termA of their human

capital attributes are one source of the gross differences in earnings. A

second source of gross. earnings differences is labor market discrimination.

Using the technique of wagegap analysis, it is possible to decompose the gross

difference in earnings between two groups (e.g., white males and black males.)

into a portion attributable to -diT,ferential*endowments of those characteristics

presumed to determine earnings and a portion whiCh reflects labor market

discrimination. ,

. . Causes and Consequences of Marriage, Divorce and Fertility Among Youth.

In recent years-nearly 20,perceht of-all births were to teenage mothers.",

. Nearly 40 percent of these births were illegitimate and nearly twothirds of

the mothers had not completed high school. Past research suggests that these

women have much poorer prospects than do those who have children later--teenage

mothers receive less education; have more children, and have both a higher risk

'Of divorce and of becomi4g dependent On public assistance. For young men as

1 ( - 205 r "9

o
well as young women, early parenthood may curtail the amount of education they

receive and reduce their earnings potential. The proposed extension of the NLS

will make it possible to study a variety of related issues including:

(1) What are the cultural, familial, attitudinal, and economic factors

that increase the chances Of early childbearing, early marriage, and separation

or divorce? How have these causal relationships changed over time?

4 (2) What are the social and,conamic consequences of early childbearing,

marriage and divorce? How do these effects vary according to sex, race,

ethnicity, and socioeconomic status?

(3) For youth who assume the responsibilities of marriage and childbearing at early ages, and for young mothers whose marriages dissolve, what

kinds of public interventions are most likely to be effective in promOting

economic independence? What are the potential roles of (a) provision of child /

care, (b) counseling, (c) 'access to continuing education,, and (d) job4training? . The Mobility of Youth. The NLS provides a unique opportUnity to examine

in detail the associations betweeh geographic mobility, local and national levels

of economic activity, and social, economic, and demographic characteristics

of youthand their families. The longitUdinal surveydesigh, in conjunction

with the plan to follow respondents regardless where they move (within the

United States), will enable researchers to model the determinantg and consequences of geographic movement. The proposed study also'will permit

quantification of many other gross changes in aspects of the labor force

status of youth, including movement into and out of the labor force and between

.employment and unemployment, movements between jobs, and between full and

part-time employment. Moreover, the relationship between these changes and

changes in school enrollment status, demographic events, and work attitudes can

be analyzed. Eiamination of changes in labor force and employment status in

relationship to changing levels of national and local unemploymeht will permit

the testing of the "discourages worker" and the "additional worker': hypotheses

and an analysis of a variety of dimensions of frictional and "disguised"

unemployment'.

Project Design ,

. The tample include 6,00b young men.and 6,000 ypung women between

the ages of 14 and.21 as of January 1, 1979. .There Will be 2,000 Hispanics,

3,000 blacks, 2,000 non-Hispanic, nanblack youth from poor families, and

'5,000 non-Hispanic, nonblack,-youth from a cross-section of families. thus'

there will be heavy offer - representation o1 youth who'experience labOr market

difficulties and who participate in the Department of Labor's employment and

traininfi.prpgramS. These youth will be interviewed' annually beginning in

January 1979 and continuing through the beginning of 1984, fin- a" total of siX

interviews., Each interview will include approximately one hour-of questioning

and will usually 'occur in the youth's home. Overall responsibility for

designing the survgys, analyzing the results, and making the data available

to the research 'Community rests with the Center for HUMan Resource Research

at The Ohio State University. TheNational Opinion Research Center at the

University of Chicago has'been subcontracted to design the sample, condfict,the

field work, and.preloarethe initial datg7tapes and documentation.

Interviewing will occur in 200 primary sampling units throughout the

country selected so that the entire sample and each of its four subgroups will

yield nationally representative ltatistics. In all, interviewing will-bccur

206 -
in 39 states and, the District of Columbia. Screening interviews in a representative sample of approximately 90,000 households aresurrently being

conducted,in order to locate' he individuals who meet the age, race, ethnic

background, and income standards which have been prescribed for inclusion

in the sample.

Time Table

The proposed time table for the 1979 interview and analysis follows.

Approximately the same schedule will occur annually for each .of the succeeding

years, with the submisAon of the sixth final report being due on approximately September 1, 1985.

First Year Activity

Sampling design complete

Screening pretest

Screening data collection

Respondents selected for full survey

Annuel ACtivities

`IA Date

April 1, 1978

June 1 June 30, 1978

September -5 - December 5, 1978 . January 2, 1979

Date

Interview, pretest August 5 - September 7, 1978

InterViews January 31 - May 15 1979 . Data reduction February 1 - August 31, 1979

Data analysis begins September 1, 1979

Initial report on survey (tabular analysis) December 31, 1979

Final report on survey. (multivariate September 1, 1980

analysis)

Potential Limitations '

Any large scale.data collection effort faces a number of problems. _ Perhaps the most difficult of these is. imposed by the limi tation of funds and

time, giiren the large number of hypotheses that would be desirable, to test and

the large-emount of information that_one ideally:like to collect. When

the budget permits only an hour interview, the researcher is faced with the

problem of whether to touch a number of issues briefly or_to concentrate on a . fewt main topics and gather intensive infOrmatiop. The longitudinal nature of

the five-year study will.definitely provide the opportunity to ask more, questions

than can be used in the usual government study.. At, the same time, the longitudinal

nature of the study requires that some information be collected year after*

year in order to make intertemporal-coMparlsons,,and this reduces the potential

for introducing new topics each yd r. We are hopful that approximately half

of'each interview will contain new inforMation. However, as we have learned

from our recent efforts to reduce our preliminary questionngre.by one- half,,much

-important'data must be foregone.

Another limitation is that our sample will grow older over' the period of

study and by the end'of the six-gear period will be 20-27 years Old.; .they will

no longer be "youth." SiXch an aging of the sample is obviously useful if one

is to attempt to measure the longer-run consequences of employment and training

program participation, education; training, etc.' However, it does havethe

limitation that. what is true for 16 year-olds in 1979 may not be true in 1984.

- 207 2 -L.
-This liMitation mty be very important because recent estimates by the

Bureau of'Labor Statistics indicate that the proportion of youth in the labor

force will fall substantially during the early 1980's and this *drop in absolute

numbers of youth may reduce the 1abOr market problems faced by this group. The

present study design will not shed any light on this issue however. This

shortcoming in the study could be met by adding younger persons to the ssmple

on a_regular basis, For imstanc-Ga new group of 1J4 and 15 year-olds might be

added in 1981 and 1983, and their experience compared with the 14 and 15 year

olds in the 1975 group.. The regular addition of youth to the survey should

be given consideration by the,Department of Labor.

`.\

ga,

a

1

b208

e ;_

c

2

a

a
NATIONAL COUNCI.L.ON EMPLOYMENT POLICY YOUTH EVALUATION PROJECT

An evaluation of prime sponsor formula funded activities

under the Youth Community-Conservatton and-ImprovementProjects

and the YoUth Employment and Training Program111 The National Council-on Employment Policy's Youth Evaluation Project

is being done to provide the Office of Youth Programs with (1) a running

account of the first year experience that prime sponsors are having in

planning for, designing, implem ntirig and operating YCCIP and YETP, (2)an

analysis of the factors imped g lbcal implementation of the new youth policy,

and (3) recommendations for i proving future operations.

Q estions for Examination

The.evalualion is geared to assess prime sponsor experience with ' respect to certain issues embodied in the ten principles that are outlined

in `the Office of Youth Programs' August 1977 Planning Charter.

YEDPA was established largely as an extensive learning -- "knowledge

development" -- experience. The `Council's study is attempting to determine

what "knowledge development" translates into at the local level: How do

prime sponsdrs interpret the mandate, and what are they doing under the

rubric of knowledge development? Has the mandate encouraged them to undertake Innovative approaches they Might not have takes] otherwise? How are the

lessons gained from knowledge development experience used?

An issue that historically has been at the heart of all youth programs,

is:the quality of work experience. We-are trying to determine whether local

sponsors are taking special steps to adapt newly created jobs to the special

needs of youths, to provide effective superion, and to produce tangible

output...

The law end regulation.&yequire youth partiOpation in the planning

process; the Office of Youth Programs. vigorously encourages' youth participation

in the impleMentation and:management process: The evaluation is examining

the role that youths are actually taking under the new-policy, and assessing

the consequencesof that role. The evaluation is also identifying promising' d't alternative -roles of involvement.

Although each program of YEDPA has its own target group and eligibility

requirements, there is considerable leeway for local operators. They can - either "cream" -those youths most responsive and likely to succeed or they can.

attempt to servethose youths less likely to be aided by more traditionalmanpower and education programs. The evaluation is assessing who sinparticular

is selected for YCCIP and YETP and how local targeting policies are implemented.

The evaluation is also examining the practical implications and apparent

effects of the variations in eligibility and targeting requirements and - practices among.thdOfferent youth programs.

209

WM'
s with any new program that overldps existing programs, the apparent.

effects o YEDPA may be diminished if local decisionmakers subttitute new resources for resources that would otherwise have been committed for prOgrams

helping uths. Priftsponsans may do_thisby using _YETPmoney in -Lieu of

CETA Titl I money to provide services to youth. Local education agencies

nay do by substituting YETP money for school resources that would otherwise

be ailable to provide services to which youths, by law, may be entitled.

The evaluation is assessing in qualitative terms whether the more obvious

kinds of substitution are.occurring. We. are also identifying some of the

less obvious kinds of substitution that are occurring and diminishing the

'effects 'of YEDPA. . / Besides presenting some.opportunities'for programmatic change,

Y DPA is presenting opportunities, for institutional change, both in administrative

structures for service delivery and in the relationship between the education'

establiOment and employment programs. The evaluation is gauging the impact

of YETP and YCCIP'in stimulating institutional change at the local level, and

analyzing the implications of those changes.

For prime sponsors irliplementing new prograMmatic approaches, there

are conflicting pressures to try entirely new program organizations on one

hand, onto stick With.proven performers on the.other. The choice can have

implications for how well narrow program objectives are achieved, and for how

well broad institutional objectives' "are achieved. The evaluation is studying

how prime sponsors chdose their delivery agents, and how the performance of

different.agents can be differentiated.

Finally, although YEDPA is part of a decentralized - decategorized

approach to formulating and implementing employment 'and training policy, it

reflects certain federal priorities. As such, there.is an interest ip how

f4ithfully the federal emphases are transmitted to local decisionMakers and

how well they are implemented., Responding-to this interest, the Colinc-il's

evaluation is also examining certain systemic. features of YETP and YCCIP

`implementation, and.is identifying the barriers that prevent or impede the

realizationof national policies. Jn the same vein, we are assessing local

experience with respect to reasonable expectations about what YEDPA,could

accomplish,(based on unique local conditions), as well as with respect to

broad national objectives of ,YEDPA.

Project Design

'A case study approach has been adopted for the evaluation because

it was felt that such an approach is best adapted to analyzing a new program

that is both decategorized in its. prescriptions and-decentralizedin its

decisionmaking patterns. The case study.approach is sensitive.enciugh to

detect important, variations in local strategi8 and tactics, and because it

provides the flexibility to document and analyze issues that emerge over time.

Ten case studies are being prepared by field associates, each

covering 3-5 prime sponsors. ,A total Of 37 prim

r

sponsors are covered by

the case studies.

- 210 2, -
The case studies actually consist of three interim studies and a

final study. A summary analysis accompanies each set of 10 interim studies.

Two interim reportiOaveteen completed; one in February and one in August. , -Athird will be submitted around'the first. of the year, and a final report

willbe submitted in June of 1979.

The evaluation, so'far, is essentially exploratory in nature. We

are emphasizing hypothesis formulation more than hypothesis testing. Because

of that emphasis -- and the qualitatNe nature associated with it -- little

syftematic 'data has.beeh developed so far. Patterns in local programs and

management styles make certain kinds of..tabulations.impossible or impractical.

Some of the similarities are so Self:eyident as to not warranuchanalysi.

The variations defy systematic quantitative analysis. ,Given tTiese civuTstances,

it would be pointless tcLatiempt any elaborate quantitative analysis,

Aside from the limitations inherent in our research design, thereis

a developing problem with respect to paining access to knowledgeable program

personnel. In some plages, evaluators are encountering increasingly stiff

resistance'to their questions, and are being increasingly shunted to rigid';

bureaucratic channels for inforMation. Some evaluators are finding4the weldome

mat pulled in despite painstaking' efforts to corroborate findings with

different sources and despite the practice of soliciting the views of local

administrators on case study drafts.

Findings to Date and Policy Recommendations

Based on those Findings

The experience of prime sponsors so far suggests, on balance a

signs scant record of success in implementing YEDPA and achieving its goals

Loca.experiences also suggest some measures that might be pursued at the

federal and local levels in order to:improve performance even more.

(1) Planning for the fiscal year 1978 YETP and YCCIP activities jagged,

. . behind schedule; it was rushed, confused, and marked by a disturbing dualism.

Given the newness and complexity-of the youth programs, the reOirements

for review of local plans and the enormity,of the implementation task, it is

not surprising that the local.efforts fell behind the Office of Youth Programs'

ambitious timetable. 'With experience, the difficulties encountered in the

first year can be eliminated for most prime sponsors. Very large prime

sponsors, with many bureaucratic layers for reviewr, are certain to encounter

chronic difficulties, however,, in meeting even the loosest timetables,

The planning process itself consisted, almost invariably, of two,

independent procedures. There was,a pro forma process required. by the

Department of Labor's 0"ant application package, and an ad hoc process'that

was centered in the real' local.,decisionmaking arenas, and was based on data

and other.factors distinct from those required by DOL. The DOL-mandated data

were'the iind of data associated with popular planning paradigms, but were not

used s a basis for.local deciAionmaking. They were deemed unreliable and

too restrictiJe;-other variable entered into the process. The grant

'application paokage-Zlso required. data to be standardized in a format that

was frequently notuseful to local'decisionmakers.

214'1
The YCCIP/YETP grant application process should be simplified to

p rovide only information DOL needs to support an up or down decision on funding.

If the Department of Labor wants to know more about local plans, it might ask

for it on a selective basis, or require sponsors to document whatever planning

process they use.

(2) Knowledge Development. YEDPA marks a new emphasis o n price

sponsor involvement in evaluation, research, and demonstratiOn activities.

The stress on creating a new prime sponsor role in these areas naturally

brought with it a need for a,great deal of guidance. It also brought.With

it the realization that in the first year accomplishments' wOuld be limited.

On the positive side, it appears that many orime sponsors did increase

evaluation, activities. A minority tried out new approaches, directly 'in

response to the call for knowledge development. A handftrl attempted structured

local experiments. It is almost certain that these types of activities were

far more prevalent than they would have been in the absence of the emphasit on local knoWledge development. These developments may hel,peepare a base for

absorbing the lessons of nationally directed discretionary' knowledge development

activiti4s. They may also lead to improvements locally.

It is doubtful, however, that local knowledgedevelopment'activities

will substantially improve overall understanding of youth problems and programs.

There is uneven 'capacity among prime sponSors to undertake this function.

There continues -to be a good deal of confusion about the substance and form

that knowledge development should take., The Department of Labor.provided too

little guidance about how prime sponsors might approach knowledge development.

In the face of the ambiguous, open-ended knowledge development requirements, . many prime sponsors did> very little, or tried only to respond to what local

officials thought that DOI. offidals v,anted. Others attempted,to set 0

excessively sophisticated research' projects that, by attempting.to do too

much, probably will accomplish very little.

4

The Department could remedy this situation by providing more technical

assistance, directly to prime sponsors, or indirecti-b by facilitating

communication among prime sponsors. The Department shouTd 'also assure'that

there is some minimum degree of agreerent among the regional offices, with

respect to what knowledge development is, and what is allowable in the name

of knowledge development. Further_guidance pn knowledge development should

encourage a linktbetween what is being 1Wed as "knowledge development,"

and other evaluation/monitoring activities.

(3) The site reports suggest that supervisionand job enrichment

have been given heavy emphasis by CETA prime sponsors; to the extent that,

these factcks affect the impact of work experience, the new youth programs

have produced major improvements. YCCIP, which was designed as a well- - supervised work experience with limited enrichment has been linked' with

other programs t6 provide more than work experience alone.

. Under YCCIP and YETP, prime sponsors are giving much attention to

career exploration for youth through actual work experience and through more

structured occupational information channels. The accent on placing youth

in career-related jobs'is proving to be impractical, Some youths have not

212 ,4L,2
o.

thought about career plans. Others have either, wh'at might be.conidered

unrealistic plans, or plans fok jobs that'db nclt fit in with the youth jobs

that are available. The emphasis on career-related jobs is also premature-.

for many youth, in the sense that. they have only the vaguest notions about'

the world ofewOrki Before they can appreciate the differentiating details

of different kinds of jobs, they need to understaki the common requirements

of all jobs.

f N* ''' ;

. o a e , e On the other hand, vocationai exploration classes and counseling ..... seem to be more effective mechanisms, for providing-exposure toAi.fferent -- career options. To the' extent that career exploration is emphasized, prime

sponsors should be encouraged to provide it through counseling and classroom ''

.experience. The 1:)iAtest,priorily in any joh development,campaigp should be

on work experience that teaches at a miriimum, basic work habits, and the ° importance of those habits. , . , - .'. . .

,

. While prime

. SponsOrs have increased eiliPlisis on private settor . linkages, apd while their efforts have had positive effects in few cases,

contihued emphasis isneeded. Prime sponsors should be encoura.ged,4frmoiew

than they are already, to attempt to establish a strong :bond with private

sector,interests. ,More practically, they should be encouraged to evaluate

job slot develbpment plans with reference tp how much of what is learned

in a certain job-is transferrable to private sector employment.

(I) Prime sponsors have folldwed the regulations requiring youth

'membershipibn youth advisory councils. However, the advisory council

appears to be a, poor vehicle for incPeasing youth participation in prime

sponsor* affai The councils tbemselves'are'often mere figureheads, and

youths onthe councils are not participating very much. Some sponsorsare

__euerimenting with alternative modes pf parLiciOation, and. are enjoying

,,O Me degree of success.

Although DepartmentalMance On increasialouth_partictpation isprime sponsors experimenting_withother forms,orpareictpation._

not limjted,toiouth advisory councils, there should be more emphasis on

(5) The YEA legislation provides prim ,loonsors with a degree of °' latitude in choosing who ,to enroll in youth progrNws. However, the BMployment

and Training Administration has gone, beyond the. legislation in emphasizing ° targeting on youth most in, need. Prime spomsors haveresponddct positively,

by consistently.aiming servicesfor economically disachtantSged youths.- There

'-- -ay be some force of-habit in effect, but local pressures for serving'those

,. - mast in need are alsalpensuasive. Enrollments should ,be-monitored to see',

. whedier_prime sponsors,continue to concentrate' on seriingeconomiciM.

A o 4 disathiantagedjoufh. If/when there is ,3 shift in targeting, further monitoring'

shouJd be undertaken to determine whether there is a reasonable ba---f6F

.__. .slianelfs. .

(1., .. Prime sponsors are seriouslylunderonrolling females in YCCIP -, .:

.

. projects. It is not clear whether this is due to' relatively feWfemales in , . the YCCIP applicant pool or to discrimination in job placements. In either

casellprime sponsors should be encouraged to undertake more outreach activityL

---7.

so that female,enrolfnentsjin YCCP'-qan456 f raised:

:-.;-- 7 , ,iC

-. .. - 213 29"
r. 0

(6) %Substftutibn of-YEDPA resources for Title.I resources is not

prdving to be a.terious problem. In-fact, there has been a remarkab1 degree , .0 .utilization of-other services Ad-programs to eriricb'theyouthjnitiatives.

Nowever,substitution of YEDPA resources for non-QM resources could prove

to seriously diminish the impactof the new funds. There is ittle.that-can.

- be done; legallysto control 'that kind of.substitution. But Where legal

recourse is limited, moral suasion might be:more effective.

0

One way to increase the interests of local sponsor's and ndn-CETA

institutions in minimizing substitution is to give them a stake in the action.

This, unfortunately, is easier said than-done: Sponsors ard.alTready being

encouraged to build-links with- other 'agencies and private sector interests..

But this_shoUld he pushed-even more, on the grounds that a broader local

base of-participdtion.will better assure a broader.constituency-interested

instretcbing YEDPA resources to the maximum. It would be naive to assume

that a coherent sense of purpose among man players could be developed, or

that this-alone will prevent Substitution. But since the substitution

phenomenon is so nebulous. and difficult, to control at times, more effective

solutions may not be feasible.

_ . -JilCommunity based organizations are thoroughly involved in the s40- youth program delivery matrix. Although performance differential's are ,hard

to,detett, the GBOs do-broaden-the ba,se of local participation. Tlie current

provisions giving them special .consideration in the selection of program

delivery agents, and incl ding them in the planning process should be retained.

, (8) 1EDPA_s s promise'fdr altering institutional roles and,

relationships 'at the local leVel,. The role of local-schools is central to

this effort. In spite of the rushed implementation:CETA-LEA agreements ,have

been negotiated and programs are in place. Even in areas-with little 6-i!'s-tory

ofCETA-LEA cooperation, the separate CETA and LEA communities are talking

withone,another, and joint efforts are underway. Nevertheless; there are ' some obvious limitations oryhowmuch change the,Department of Labor can.

leverage in LEAs,.bY using prime sponsors. LEAs areLnot accustomed fa--

receiving guidance from gtTA prime sponscirs. Under the best,.4f conditidgs,

prime sponsors can-hope/to affect local education policy only marginally.

In son of the worst eases, LEAs.and sponsors do not speak at all.

/ ' Fo these .reasons, the Department of Labor should explor:;:alternate

cHanfiels for affe9fing education decisionmaking at the,locOlevel. Until

'those otEer channels for communicationare identified and uti-rized:-maoyschools will fayl to meet the needs of,youths who could otherwise be' served

under coopera:pe arrangemOits with LEAs and prime sponsors, or even through

the LEAs alon,.

/

One7of the significant patterns that seems to be emerging

Presence of'LEAs where CETA-private sector links exist. It is posstPle that

LEAs may provide the missing piece necessary to complete productive relationships

between public sector CETA agencies and kiVate sector employers. The Department

of Labor. Should examine CETA activities that involve4rivate sector tiles, to see

LEAs ply do integral role. The-Department thOuld7so study some of

its exemplary CETA -LEA projects to evaluate their private sector links where

they4exist, and to estimate' the feasibility of such ltnks where they do not

exist. -.

-S
Project Overview Statement 1/

ECotomic Impact of Job Corps

by Seymour Brandwein

Director,`Office of Program Evaluation

Employment and Trainin4 Administration

liplike the YEDPA programs focused on by,most of this

conference, the Job Corps has been in optration for some

time. It has survived over 14 years of fluctuating attitudes towards sizakle investments to aid disadvantaged

youth.

PtoW turbulent start-up years, it has evolved into

a relatively,stabld peogiam with a good notion of what

it!4,doing,toward its general statutory purpose (first

.stated in th- 1964 Economic Opportunity Act and since

carried foriaard in CETA) "to assist young persons who,

need and.can benefit.from arT,unusually intensive program,

operated in a group'setting, tb become more responsive,

employable, and produCtive Citizens..."

Becausets intensive nature and Sizable investment

pee participant should yield reasonably measurable results, because little was known` reliably about 'ti-46 magnitude or timing of economic effects of such investments,

and because the evaluation art had advanced somewhat

since the Job Corps wasp the subject of controversial

evaluations, in'the alte 1960s, we decided in 1976 td

'undertake evaluation of the J4D-Corpsas the principal

impact evaluation effort in the youth field:

. A competitive procurement ended up selecting

Mathematica Policy Research ,to conduct'the evaluation,

The preliminary findings on the short-term economic,

effects of the 1977 Job Corps program arc now in and

being' reviewed, with a report scheduleqJto be issued in

November.

1/ Prepared for "Knlowledge Development Projects

ConfeienCe" of Office of Youth Programs,

October 5 -6, )1078.

215 22) ,
V

Objectives'of the Evaluation: The principal Purposes

-Of the evaluation area to determine:

1. To what extent do Job...Corps participants (as

compared to comparable nonparticipants):

-(a) increase 'their employment and earnings.

.(b) return to school, enter military service,

or enter other training or work experience

ti programs.

4 V

(c) rely less onwelfare transfer-payment

programs.

. (d) reduce antisocial behavior' particularly

criminal activities and,:drug abuse.

2.' To what extgnt do such effects vary by type of

participant, duration of participation in the program,

and by type of Job Corps center?

fioN.A> do the Job Corps participants ?ate their satis--'

faction with the program? What is their assessment of

its strengths and weaknesses?

4. Do. the program benefits-to participants and to

sociey, where measurable in dollarterms, outweigh

the program costs?

Design: In brief, the program impact is measured

primarily by comparing the experience of Job Corps parti-.

..ciPants over time to that of a group of similar youth not

enrolled in the Job Corps.

The comparison-group was drawn from.school dropout

fists and Employment Service applicant files in geographic

areas which had relatively low Job Corps participation

rates (in large pat because they were not near Job Corps

centers), but which `in other respects were similar to

areas from which most Job Corps enrollees have come.

The baseline interviews of participants...were conducted at Job Corps centers in Spring 197.7 to obtain information on demographic characteristics, socioeconomic

background, employment and earnings, education and

training, antisocial behavior, and participants.assess=.

rp9nt of the program:

- 216 -

1/4
".,

-$

v 1 Thesample, some 5,100, was a random selection of

'about one-third of participants. Nine months later,

those who had terminated and been out at least 5 months

were interAewed again to obtain information on their

earl 'Post'Pr`ogram experience; the followup sample was

about 2,400, who had been out an average of.7"mOnths.

The comparison group baseline sample was 1,500.

TheJollowup interviews were obtained for some 1,300 of

. them.

A baseline data analysis of the participael and.

.'comparison groups' characteristics and prior history indicates that. the two groups were quite similar in these

respects.

I In additibn to the interview information,'the study

drew cost data from Job Corps operationalrecords and

developed estimates of the value,of output'produced by

6arti&ipanls as part of their training and work while in

the program.

Preliminary'Findings An early report on the baseline

anterviev( data, "An Examination of Job Corps Participation,".

was published in June 1978, and is available on request to

; the Officg' of Program Evaluation.

, It describes the nature of the Job Corps program in

1977,,,-docdmenbs the "disadvantaged" naptre of thosd it"

is serving (enrollee derlographics, family background and

incOme, prior educationil he Ith, and work historsand

it "antisocial behavior"), reV ws why theychose to enroll',

and Tresents their ratings o the program while they were

still in it (alme-g% two-thirds were "fai-rly" or "very"

satisfied, over 90 percent thought their ei.aiaing pas

"good" or "OK," bUt half were,critioal of.the food and

size oftheir pay allOwances). . ,. The followup data has now bee developed and analyzed . in a preliminaryf.'report whichi eing.given tedhnical . , review by an outside expert advisory group and by Labor

'Department staff: . .. -Its tentative findings on short -terms impac-tl/o-n --. labor market experience are: , 1. Program completere (about 30 perCent of all,

enrollees) did very well orrb employment-and earni450.

217 31
as compared to the comparison group, for both

menoand women, butt "partial ompletersr

petcea0 and early dropouts (40 percent) did

only about as well as the comparison group.

2. In the initial months put of the Job Corps,

its participants lagged behind the comparison

group, many of who were employed at the start of

the period whileftit took time for the.Corpsmembers

.to resettle and get jobs.

3. After the first two months, the positive

, economic impacts began to predominate for the

completers. By.the week before the followup

survey,, the completers (both male and female) had

gained about $23 more in weekly earnings than.the

comparison group. The partial completers were

slightly ahead and the early dropout's were behind,

1,though the differenceswere generally:statistically

insignificant.

.wr*

On\other measures of impact too, the completers had

positive, large, and statistically significant benefits:"

more were'in the labor force, more weroe employed, they

averaged more hours at work, more of the men were in

military "service (a goal of some Job Corps participants

more had gone on to a high, school degreee college or more

training, and they had less welfare dependence and reduced arrests and abuse of drugs. For those who partially _ completed the program, such net impacts were small, and

cigDpouts did not do as well as the comparison group._

Asito enrollee ratings of the program, 7 months

after leaving it, 77 percent.expressed satisfaction

(1;tersus 67 percent giving, such ratings while they Were,

still in the program).

Some 90 perceit or more gave good or OK xatin to

the training and education services they received. In

retrospect, they rated the food and pay better than they

did while in the prograti (but with a third-to nOarly-ha4

still critical of those two aspects).

, . .Threy fe they did not get enough placement.,asSistance. Nearly 60 percent lid that they elid*not get

placement a from the Job orps or agencies thd Job

Corps sent them to -- andthree-gba-rters of these said

they, could have used .more help in finding a. job.

eg . ty

- 218 2"
On the relation of benefits to costs, the study

draft report develops some comprehensive estimates of the

dollar value of various measurable benefits to participants

and to society as a whole.

These tentative estimates depend-very substantially

.on various assumptions, most notably about continuation

rates of gains in subsequent years, about appropriate

rates of discoUnting future dollars to present values, and

about thereliability and relevance of data on societal

savings from reductions in criminal activity.

The initial estimate is that economic "benefits" to

society in dollar terms for,eah.Job COrps enrollee'are

larger than'the economic "costs":'of the program; societal

benefits per Corpsmember are'estimated at $5,270, almost

- $200 more than the costs..

There patently,are considerable issues on the cpnceptual

and' technical judgmentsUnderlying these estimate's (as well

as issues of their significance for policy on scale or mix

of youth program approaches}, but these are beyond the scope

of this brief paper.

Plans: Issuance of a reportis planned for November

1978, after.the.technical review now being made of analytic

assumptions and methodS.. No doubt it will generate further

technical comments which would be taken into adcountin

planned continuation of this evaluation.'

Thekey limitation of the study is that its findings

are based on-just the first 7 months of ippstprogram ex,

perience. It is by no means clea*whether the participants'

ppsitive gains. inthis.early period are maintained, deteriorate (as the benefit-cc 7t calculations-assume), or'

-4,. -1 increase further, °'

. The evaluation is therefoi*e to be carried forward

(as was originally planned) to ..follow up on the partici-:

pant and comparison groups in-similar interviews a yea, -

after the first forlowup, by which time tlie first follo0-

up sample will have been out of the program for an average

-of some 19 months. - We plan, for budgetary reasons and because of indications that it may be as efficient, for repeated interviewing, td conduct the second followup on a telephone

qt, 233 . - 219 -

4
IN

t

. ra.the than'facerto-fkpe basis,y,a,lthOug h'persbnal'inter-,

views Will besought where the individual is not locatable

or responsive by iihorie.".

(Data will Alsope collected on the baseline interviewees who had not yet left the program or had been'out

less than 5,.monhs at the,tiTe of.the-frrst followup.

This will enlarge the sample of terminees and enable Som

check on'whether the findings hold'up.for partiCipants

who-left the program at a 'later point_in"time).

0

Findings froh this second fol,lowup should be ready

by late 1979. 'stip: another'followup at a later, point is

a possibility,.but.that,ii-contingentin major part on

likely adequacy.of the sample in light orresponse rate".

expetience-,;in the second followup,

C4

4

4- 220 - 0

, :
U.S.. DEPARTMENT OF LABCNR.

EMPLOYMENT AND TRAINING ADMINISTRATION

WASHINGTON, D.C. 20213

II OFFICE.0 YOUTH PROGRAMS

'KNOWLEDGE DEVELOPMENT PROJECTS CONFERENCE

Prog.ram'Agenda

Sheraton Conference Center

.Reston, Virginia

.THURSDAY,:OCTOBER 5, 1978

:00-10:00 AM

. .

INTRODUCTIONS: -I John Drew and Andrew Hahn.

Brandeis University,

. Center for Public Service

CONFERENCE GOALS:, Robert Taggart, Executive

Administrator, Office of

,. Youth Programs, 'Employment

and Training-Administration

D:00:12:00 Noon tPANEt:

01.'

0

FACILITATOR:

. YOUTH INCENTIVE ENTITLEMENT

PILOT PROJECTS (YIEPP) --

DEMONSTRATION AND RESEARCH

4Adbert Evans, Brandeis

University

PRESENTERS: Overview of National YIEPP

Effort -- Judy Gueron, Man- - power Demonstration Research'

Corporation (MDRC)

YIEPP,Ithpact

. Analysis --

Ernst Stromsdorfer and

Robert Jarrett,' Abt

ates

YIEPP- Implementation Analysis --.Josetih (3a11 and

William Diaz, MDRC

DISCUSSANT: Robert Lerman, D9L/AsPER

2 3'5 :

".
. . Knowledge Development Projects Conference

Program Agenda

es 1:00-2:30 PM PANEL: YOU_COMMURITY CONSERVATION,AiD THIMPROVEMENT PROJECTS (YCCIP)

DEMONSTRATION AND RESEARGH

. FACILITATOR: Martin.Leviri,.arandei5 University

-4

PRESENTERS: Ventures in Community Improvement

,Demonstration. Project ---John

Kelly, Corporation for Public/

,Private Ventures (CPPV)

7.1 DISCUSSANTS:'

2":45:E:60 PM PANEL,.

. . 1

. FACILITATOR:

PRESENTERS:

..CommunitY 'Conversation and Improvement Demonstration Project . Bill ScheCter, HUD; jay .0strowef%

and Roy Feldman, Boston University

. .Robert Shrank ,Ford Foundation , Burt-BarnoW,IpOL/ASPER

. YOUTH EMPLOYMENT AND .TRAINING PROGR4MS.(YETP) -- DEMONSTRATION AND

,'RESEARCH

Wyatt Jorrs; Brandeis Uni'versity

Exemplary In-School Youth Program

Demonstr'atjon Project, Harvey

Pressman, Y4thwork, nc. Career Intern .Program DemonStration Project, Howard Leznick,,

HEW/NIE; G.K. Talmadge, RMC

Research Corp.

PrivateSktors Initiatives

Demonstration Project, Richard

' DeLotie and David Leary, cPPV , NOuth Comtunity Service Demonstrati.on Project, David Muchnick, ACTION

, Service. Mix Altecitivet Demonstra-,

.tion Project, Jos'eph Seiler, DOLYR.

DISCUSSANTS:. John'Palmer, Brookings Institute.

Donald 'Nichols, DK/ASPER

6:30-7:30 PM SPEAKER:- 'ArtOld H. Packer, Assistant Secretary

-for Policy;\Evaluation and Research,

Department of Labor.

- 222 - r 'Jr) 4

U
Knowledge Development Projects.. Conference

Program Agenda

7:30 -9:00 PM' PANEL: BASIC RESEARCH

FACILITATOR: Barry Friedman, Brandeis

University

Introductioilio.Research on "'

Youth Unemployment, Robert

Lerman, DOL/ASPER

Trends in Youth Employment,

Richard Freeman aria)David Wise,

National Bureau of Economic

'Research

PRESENTERS:

Youth UnemplOyinent: ImpAct of

. Employment and Training Programs/ -.

. Social and, Economic Significance,

' Richard Toikka, Urban Institute

Measurement of Work.Oufput of , Youth Pr'ograffis, David Zimmerman

Mathematica Policy Research

DISCUSSATS:' Avril Adams, National Commission

.for.Unemployment and'EmployMent

Statistics

Howard'Rosen,-Direcior, Office.of

Research-and.Development, DOL'

FRIDAY., OCTOBER 6, 1978

4 9:30-11L80 AM PANEL:

FACILITATOR:

DATA COLLECTION AND EVALUATION

Randall'Filer, Brandeis University

PRESENTERS: Continuous Lopgitudinal Manpower

Survey of YEDPA Programs, . Seymour Brandwein, DOL/OPER

.

. % '. Nafthnal Longitudinal Survey,

Mike Borus; Ohio State University ' Research roundatiOn . ,...

.- . Process Evaluation of Implementation . of YEDPA Programs: Greg. Wurzburg , National Council on Employment ... ,,

"4.... Policy; Gary Lacey, MDC, Inc.;

Joseph Seiler, DOL/OYP

.,:., . Economic Impact of Job Corps,

. Seymour Brandwein, DOL/OPER

A

DISCUSSANT: Andrew Sum, Northeastern University

223.7

23 7'

A..
0

r

1

Knowledge Development Projects.Conferen6e

Program Agenda -

1.:00-3:00 PM PANEL: CONFERENCE SUMMARY AND A.LOOK FORWARD'

, a

FACILITATOR: LeOna,rd nausman,,BrendeisUni,versity.,:-

`CONFERENCE PANELISTS:

. . . _Robert Taggart DOL /OYP

Donald Nichols DOL/ASPER' . . -..Patrick O'Keefe National Commission

for Manpower Policy

Vernon Briggs Cornell University

Joan Wills Nattonal Governors

- Conference

Othello Poyfard, Center_for Com-

. 'Munitg Change

Al'frda-V.:*Love. AFL/CIO. Human

.1

Resources Develop--;

ment Institute ...

"-,.1

o

A

a

I t

o

r

-.224 -.- 2')"

. I .

. :

,
.40

.%)

CONFERENCE PARTICIPANT§

Avri 1 Adams

National Commission for. Unemployment

& Employment Statistics

Room 550 . 2000 K Street

Washington, DC 2G006

Joseph Ball

ManpoWer-4emonstration Researc Corp.

3 Park Avenue

. NeYork, NY 10016

Burt Barnow

.USDOL/ASPER- . Room S 2231

200 Constitution'Avenue

Washington, DC 20210

Michael Borus

_ . Ohio State University.

Research Foundation

Columbus, OH .43210

Seymour Brandwein, Director

Dennis Derryck

ACTION

806 Connecticut Avenue

Washington, DC 20525.

William Diaz

'Manpower Demonstration Research

Corporation

28th Floor a6

3 Park- Avenue

New York, NY 10016

Robert'Ddhaldson

Secretary's-Office

Department of Labor

200 Constitution Av'enue

Washington, DC- 20210

George Fatkas

Abt Associates

55 Wheeler Street

Cambridge, KA., 02139

OffiCe of Program Evaluation Research

601 D Street

.Washington, DC 20213

Vernon Briggs- , New York State School of Libor

& industrial Relations

Cornell Urriversity

Ithaca, NY. 14853

-46

Pam Clark

Office of:Youth Programs

601 D Street

Washington, bC .20213.

Richard elone

Corporation for Public/

PriVate Venture's

W26 Cherry Street

Rhiladelphia, PA 19103

Roy Feldman

Regional Institute on Employment

Policy

Boston Urtiversity

270 Bay State Road

Boston, MA' 02115

Thomas P.*Glynn., Director

Planning

ACTION - Room 606

806 Connecticut Avenue

Washington, DC 20525

Judy (Lieron

Manpower Demodstration Research

Corporation

28th Floor

3 Park Avenue

New Ybrk, NY 10016 '

Kitty Higgins

Room-220 ' - Old Executive Office Building

-Washington, DC

225 - 2r3',)

.
0

t

Conference Participants

Robert Jarrett

Abt Associates

55 Wheeler St.

Cambridge,' MA 02138

John Kelly

Corp. for Public/ Private Ventures

1726 Cherry Street

Philadelphia, PA 19103 '

Gary LaCey

MDC Inc.

137 East Rosemary St.

Chap41 Hill, NC 27514

Robert Lerman

USDOL/ASPER

200 Constitution Ave.

Washington, DC 20210

David Leury

Corp. .for Public/Private Ventures

046 Cherry St.

Philadelphia, PA 19103 , Howard Leznick

Senior Associate

National :Institute-of Education

Room 816

1200 19th St., Ni.

Wpshington, DC 20208

yarry Lieberman

OPER/DOL

601 D Street

Washington, DC 20213

Alfred P. love, Assistant

AFL-CIO, . Human Resources Development InStitute

Room 405 = 815 1.6th Street,-N.W.

Washington, DC 20006

Cha rles Mallar

Mathematica Policy Research

O Princeton, NJ 08540

Director

N. Ernest Martin

Office of Youth Programs

ETA/DOL

601 D Strut

Washington, DC 20213

David Muchnick

ACTION

Evaluation Department

Room M 207

$06 Connecticut .Ave., N.W.

Washington, DC 2.0525 \

Donald Nichols

USDOL /ASPER' % Room S 2233

200 Constitution Ave., N.W.

Washington, DC 20410

Patrick O'Keefe

Natiogal Commission for Manpower

Policy

1522 K Street, N.W.

Washington, DC 40005

Jay Ostrower

Regional Institute on ,kmployment

Policy

Boston University

270 Bay State Road

Boston, MA 02115 ' 11

Arnold Packer, Assistant Secretary

PolicyZyluation Research

U.S. Department of Labor

ROOM S 252,4

200 Constitution N.W.

Washington, DC 20210

John Palmer

Brookings Inst4ute1775 Massachusetts Ave.

Washington, DC 20036

226

O ,.*

2,;

b

I

j.
`')

Conference Participants

Cheryl Perry

REW/NIE

Room 816,

1200 19th St., N:W.

:Washington, DC 2008

Mr. Othello Poulard, Director

CETA Resource Center

1000 Wisconsin Ave., N.W.

Washington, DC 20007

-Harvey Pressman

183(Lake Avenue

Newton, tIA 02159

Tom Rhddenbaugh

National-Economic Development

Law Project

1523 0 Stre t, N.W.

Washington DC 20001

ToM Ros e General Accounting Office

-441 G Street, N.W.

Washih§ton, DC 20548

Howard Rose , Director

Office of R earch and Development

U.S. Departtent of Labor

RoOm 9100

601 D Street

Washington, DC 20213

Janet Rosenberg

Office of Youth Programs

ETA/DOL.

'601 D Street

Washington, DC 20213

Bill Ruttenberg . . AFL-CIO - Hudan Resources

Development Institute

Room 405 /.

815 16th St., N.W.

Washington, DC 20006

4.1

4

4

Williad Schecter, DeputrDireCtOr

Youth Employment- Demonstration ' Project

Office of Neighborhood & Consumer Affairs, Room 4230

451 7th St., S.W.

;Washington, DC' 20024y .

**, Herman Scott III

Office of Youth Programs.

ETA/DOL

601 D Street

Washington, DC 20213

Joseph Seiler,

2.U.SDOL/Office'of '40outh Programs

601 D Street

Washingfori, DC 20213

Robert Shrank

The Ford Foundation

320-East 43ta,St.

NeW York, NY 10017 . William Spring

The White House

. .1700 Pennsylvania Ave., N.W.

Whington, DC 20006

Ernst Stromsdorfer.

Abt Associates

55 Wheeler St.

'Cambridge, MA 02139.

"WilbettSolomah

Office of YouthYrograms

ETA/DDL . , 601 D Street

,Washington, DC 20213 \

- Andrew/turn I-- Depattment,of Eco omics

.Northeastern Univ rsity

Boston, MA 62115

Robert Taggart, Administrator

Office of'Youth Programs

601 D Street

Washington, DC. 20213

- 227

1 .

.

1

I :

1 i

i
Conference Participants

G. Kasten Talmadge

RMC Research Corp.

2570 West El tamino Real

Mountain View, CA 94040'

Richard Toikka

The Urban Institute

2100 M Street

Washington, DC 20037

Joan Will s

National Governors ',Conference

Su'ite 2p2 ? 444N. Capjtol St., N.W.

Washington, DC 2,0001' , : : - a . David Wise

J:F. Kennedy School

.79 Boylston St. ;

Cambridge, MA. _02138:

Maryanne Wol fez Director

Women's Bureau.

ROom 3313-,S

200 ConstituttN'Avenue

Washington, DC - 20210

Gregory Wurzburg . National ColinCi 1 on Empl o*,ent . . Policy a 2000 K Street, - linhington, DC-20006

David Zimmerman'

,MAthematicePolCy Research

905,Unixersity Avenue

Madison, WI 3715

4

I

.

228
CONFERENCE PARTICIPANTS"

, FROM THE CENTER FOR PUBLIC SERVICE

BRANDEIS UNIVERSItY

PRINCIPLE INVESTIGATORS:

John Drew

Apdrew Hahn

Leonard Hausman

Research Review Panel:

Robert Evans

Randal Filer

Barry Friedman

Wyatt Jones

Martiq Levin

4

Center-for Public Service

Brandeis University . Sacpar International ,Building

Waltham, MassadhuSetts

02154

'Telephone: 617-647-2108

' John Drew, Director

- .229 --

ito