DOCUMENT RESUME

10 206 862

01LE. . The Standardized Assi'sgment.Systea for Youth

DemonStration Rrojects., KnOiled4e Developtent

Frateiork: Youth Knowledge.DevelopaentReport 1.6.

INSTITUTION Educational Testing Service, Princeton, N.J.

-`SP_ oils AGENCY Eaployaeht-andTraiting AdiiniStration (DOW,

.. Washington, D.C. Office of Youth Prograas.

ANIR DATE . May 80

!CIE 201p.f For related docuaents see note of CE 029

96B.

AVAILABLE FROM Superintendent of Documents, U.S. Goyeriaent Printing

Office, Washington,DC 20402 (Stock No.

029-014-00121-0, S6.00).

EDRS PRICE ,MF01/pC09 Olus'Postage..

'DESCRIPTORS Attitude Measures; Career Awareness; Dec sion Making

Skills;-Detoi*fiatk#Otograis;.Education 1 . AsseStment EdudatioditkiResearch; Eaploy = _Attitudes;

*EiplOildit;prOgraas; Evaluation Methods; ollow0p

Studies; JOb-Se4ch MeihOds;67Ob Training;

Participait:CharaCteristics;-Patticipant

Satisfaction; Perionality Measures; *Program

Developient; *Prograa Evaluation; Psycihologic 1

Testing; Self concept'Reasures; Sex Stereotype

Student Attitudes; *Validated Prograas; Work

AttitUdes;-Youth Empioyaeut; *Youth Programs

IDENTIFIERS Individual Participant, Profile; *Standardized

Assetsment.Systea; Vocational Attitude Scale; Work

Relevant AttitudesScale; *Youth Eaployaent and

I . Demonstration Projects Act

ABSTRACT

This report describes the background, developaent,

alidation, and sajor coaponents of the Standardized AisesSment

(SAO. (The SAS is a.multi-component Systea dengue(' for use

it:eiraluatinga: wide variety of ezperiaental and deaOnstration youth

11;00ass initiated the:II:11:th E#ploysent and Demonstration projects

Adt4IEDPA-,;of 1977..) Followii4a brief brief desCription of the

qvirkils of the Systeri, the major Components of the systea (including

initru ents to measure-participant characteristics and a psychoaetric

:battery \designed to-aSeess.career decision making, awareness, and

.:00041#0,. self.riaiget.wOrk attitudes; job search capacity; and

OtcuPatiOnal sex Steieotleing) and their rationale are discussed. The

yalidationeof SAS.ihstruments-under current demonstration piojects is

eiaainedl,'Ouilined next is the application of the measures, f011owed

by ,an ezOlanatiOn:of the basic analysis gall. Appended to.the4teport.

at the.follOiting SAS instruments: the IndifOual Participan*'

ytOfilef theVocatiOnalAttitude Scald; a Series of measures to

,asSeim-job.knOwled4e, job holding skilli, job seeking skills, sex

stereotypes Of adult occupations, and self- esteem; the Work Releyant

Attitudes Inirentory; a program coapletion survey; and a prograa

followup survey.. Instruction4 for administering the instruments are

:iuCludek (Related youth knowledge and development teports are

aiailablemeparately through EPIC- -see note.)-

CE 029 970
YOUTH KNOWLEDGE DEVELOPMENT REPORT 1.6

THE STANDARDIZED ASSESSMENT SYSTEM

FOR YOUTH DEMONSTRATION PROJECTS

U.S. DEPARTMENT OF EDUCATION NATIONAL INSTITUTE OF EDUCATION

EDUCATIONAL RESOURCES INFORMATION

CENTER (ERIC) /This document has been reproduced as

received from the person or organization

originating it. L I Minor changes have been made to improve

reproduction quality.

Points of view or opinions stated m this docu

ment do not necessarily represent office; I NIE

position or policy.

Educational Testing Service

March 1980

For gale by the Superintendent of,Document3, U.S. Government Printing Office

Waghington, 1).C. 20492
U.S. Department of Labor

. Ray Marshall, Secretary

.Ertiploymeht and Training Administration

Ernest G. Green, Assistant Secretary for

Employment and Training Administration

Office Youth Programs

Material co tained in ttfis publication is

in the public omain and may be

reproduEed, f y or partially, without

permission of th Federal Government.

Source credit is uested but not

required. Permiss on is required only

to reproduce any opyrighted material

contained herein.
TABLE OF CONTENTS

PAGE

OVERVIEW

INTRODUCTION 1

Sr

THE STANDARDIZED ASSESSMENT SYSTEM 3

THE COMPONENT MEASURES AND MHEIR

RATIONALE -6

VALIDATION OF SAS INSTRUMENTS

UNDERCURRENT DEMONSTRATION PROJECTS 27

MEASURES APPLICATION 40

BASIC ANALYSIS PLAN 44

APPENDIX 1, INSTRUMENTS

APPENDIX 2,, SAS PROJECTS .LIST

o' s.
OVERVIEW

The Youth Employment and Demonstration Projects Act .of 1977

authorized a range of research, evaluation and demonstration

activities to increase-understanding of the employment problems of youth and to help deterdine the most effective

policies and programs to address these, problems. In the

preceding fifteen years, a massive array of research and.demonstration activity had been undertaken for this same purpose.

Under the War on Poverty and continuing under CETA, there had

been thousands of innovative projects and tests of alternative

approaches. While much had been learned from these efforts,

it was the Congressional perception that there was no firm

informational basis for public policy.

There are several reasons why more had not been ]earned.

First, past demonstrations wero rarely implemented in mul-

- tiple sites with the varying conditions and large sample sizes

needed to reliably capture the impacts of short-term in4-.erventions or to predict success in alternative settings. S cond,

the assessment procedures' varied from one demonstration to the

next in terms of measures of success, control or comparison

group selection and the like.. Third, the evaluations were too

frequently limited in scope. Rarely was process'and cost

information integrated with rigorous statistical impact inalysis. The projects funded by the Department of Labor put

.__perhaps too much emphasis. on economic outcomes and too little

on the psychOlo-gidal-and-15-ehavioral changes so important in

understanding the youthexperience. Studies by other agencies

frequently ignored employmeLt issues. .Fourth, there was no

system for tracking participants over the long run. Use of

Social Security data to assess the impacts of youth programs,

foundered because of the lack of adequate mechanisms for

ideritifying comparison groups for,,program participants and

controlling for the many individual differences which prove

important over time.
In view of the need for developing more precise knowledge

on the effectiveness and impact ok alternative youth program

approaches, the Youth 'Employment and Demonstration Projects

Act ( YEDPA), was designed as a limited duration, experimental

effort to.enable the,development of a knowledge base which

would faibilitate the design of new and improved national

youth policies and programs.

YEDPA,provided the Secretary of Labor with substantial

discretionary flexibility in order to design .the needed

knowledge base as well as the resources to overcome some

of the earlier shortcomings. Using these resources and . flexibility, the Department initiated an array of demonstration projects testing alternative approaches\and exploring

key policy questioni. Most of the demonstrations consisted

-of -relatively-standard42ed-aet-icit4esimplemented r pi

sites. Insofar as possible; random assignment control group

methodologies were employed in the research designs. Process

and impact evaluations were built in from the outset for each

of these demonstrations.

To realize the full potential of this structured array of

demonstratims, it was necessary to develop abStandardized

Assessment System so thFtt the findings could be reasonably

compared ana contrasted across projects within and between

the various demonstrations. The SAS which was developed

aita is analyzed in this volume was geared to:

x, (1) Provide a uniform data base across a wide variety

of programs: Although the YEDPA demonstration projects are quite diverse in intents, most gather

similar information in,a standardized format and

report the data on an individual rather than

aggregate basis. The common data base is essential if ijinerafizations dre7-to-b-e-made-about-impact

across sites and delivery agents, and to differentiate4the impacts of specific elements in the

service mix.

O (2) Provide comprehensive information: The SAS was

designed to provide comprehensive information

about participant characteristics including

psychometric variables to track in-program

changes in attitudes and awarenesses, to

describe the nature of services delivered and

the setting, and to report on a range of possible

outcomes in- a consistent fashion,

(3) Furtherassess the usefulness of instruments:

The SAS consisted of instruments that had been

demonstrated to be- both -valid and reliable for

disadvantaged populations similar to those

served,by YEDPA. It is expected that this

common assessment battery will undergo some

ii

6
revision as a result of the data to be collected

during programevaluation. The.determination of

the measurement properties should thus yield a

battery of Appraisal and- survey instruments that

are (a) appropriate forproyiding descriptive

information-tUdhasgroUp norms; (b) sufficiently

sensitive to'lheiSbre'the change resulting from

program. intervention; and (c) predictive of relevant

outcomes, regardless of program interventions or any

other dbmpeting causes for changet-in scores over

time.

_thiLAddressaffAriety o'critidal-questions and issues

regardina program effectiveness` : frtivenesb-maybe.defined. as-improvement along many work-related

dimensiOns,jnCludin4 development of world -of -work

Competencies and attitudes; job attainment, job

retention, and j*performance. The :effectiveness

of. each program sponSor will'be assessed -on these

'dimensions by comparing' enrollees who experience

a partictlar program approach to a comparable.

(control) group of nonenrollees, or-to; a group

enrollees who experience a 'different program approach.

(5) Provide a foundation for longer term assessment of

impacts: The SAS provides detailed standardiied

information about indiViduals, permitting refined.

statistical 'analysis necessary for construction of

control groups as a baseline for long-term tracking.

The longer the follow -up, the more important

that-the-cohorts-being-compared are similar_and

that the interventions be closely defined. The

data base dreated'by SAS can be integrated with

Social Security records over the long-term to

determine if there are impacts on future employability

----beyond - the short-term follow-upg. built into the demostration designs.

The formal evaluation paradigm assumes a common goal for all

prime sponsors; namely, "to provide a broad range of coordinated

employment and training programs for eligible youth in order

to provide effectively for comprehensive employment and training

services to improve their future employability and to explore.

.% and experiment with alternative methods for accomplishing such

programs." Regardless of the delivery mode, services help to

"bu31A job-related sxills and job-related attitudes. The

iii MI
listing below presents the (pre- and post-program) measures

of specific world-of-work competencies which are relevant

to the various YEDPA programs.

Work-Related Skills

In-Progiam

Evaluation Measures

(Job-Knowledge Scale)

(Job-Holding Skill Scale)

(Job-Seeking Skill Scale)

Work-Related Attitudes _ (Vocational Attitude Scale)

(Worh-Relevant Attitudes Scale)

(Self-Este-dm Scale)

(Sex Stereotypes of Adult Occupaticint

Scale)

The administration of these pre- and post-program measures to

enrollees and-comparable non - enrollees, or enrollees in alternative

prOgrams, provideata means for estimating short-term program

effectiveness, or impact; in the relevant Competency areas. The

cotprehensive common data base also permits ,the examination of

program-related gains of designated subpopulations within, as

well as across, programs.

Because such measurement is only a short-term assessment-tend a

substitute, or "proxy", for more relevant distal or longer-term

criteria, additional mandated information will be collected at

program completion and, also at three and eight month periods

following program completion. A program completion survey will

solicit feedback on the enrollee's perceptionsof the program

as well as his or her self-estimate with respect to variables

that form the following dimensions: (a) _work motivation, (b) social andcoiimunity adjustment, Cci 'training Program adjustment, and Coll-vocational planning competency.

The above areas have been demonstrated empirically to be related

to post-program on -the -job measuresof satisfaction

and-performance for disadvantaged youth in work-training programs.

The short-term indicators of gain (in.the case of the standardized

pre- and post-program measures battery), and the status assessment

(as measured by the program - completion survey instrument and by

work supervisor and counselor ratings) become relevant and

important, if they, in turn, predict the more pertinent post-program

job attainment/job adjustment/job earnings expert ces.

The behavioral outcomes following program participatibn are

relevant to the:goals of YEDPA, and instruments for' measurement

of these criteria have-been demonstrated to be both internally.,

con's'istent and predictive from the earlier shori-teiiii criteria.

These post-program criterion dimensions consist, of: (a) on the lob

success and satisfaction, (b) general social and Vocational

adjusttent, (c) job-search motivation, (d) job-Planning

competency, and short range job success,.

iv 8
To evaluate fairly the effects of specific program activities

or curricula on short-term and long-term criteria, both within

and across programs. it is necessary to,gather additional

information on various program implementation processes, since

programs differ in their objectives, budgets, duration of

/employment and training activities provided and in their cur-

/ riculum content. Thus, for example, there is little exRectation of finding a positive change in. the perception of

occupational sex stereotypes at a site which allocated neither

time nor resources to this particular goal. Process surveys will

supply the information necessary with regard to *staffing, modes

of delivery, service time and budgets allotted to the various

service categories. The. statistical analysis will make

use of such information in attempting to determine why gains

or program effects were or were not. observed; Similarly, a

fair program evaluation involving measures of job attainment

and other criteria external to the,program, must take into

consideration relevant external conditions (e.g., the 'availaoility, in both kind and guality,of nonsubsidized jobs).

Indices based on the local labor market conditions will,'

therefore, be used as both'explaratory and control variables

in the various evaluation models.

Because of the massive array of demonstration projects, it is

also important that there be some standardized groundrules

for the timing of the apaication of measures and follow-ups.

Likewise, there is.a need for a standardized analytical

approach-so'that.findings of disparate analyses are compar-

, able; While each demonstration may have-its own knowledge

development objectives and special evaluation components to

assess these objectives, there are certain analytical approaches

which are a necessary baseline for all assessments. These

standardized - analytical approaches must be built in from the

beginning of demonstration projects.

This volume and its appendices describes the elements of the

Standardized Assessment System including the impact and process measurement batteries, the standard analytical approaches,

as well as the procedures for'application. It analyses the

background and accumulating evidence on the validity of these

elements in this system. It presents the measures themselves

(Appendix 1) as well as referencing the demonstration projects

in which they are applied. (Appendix 2)

This volume is one of the products'of the '"knowledge develop-

- ment" effort implemented under the mandated of te Youth Employment and Demonstration Projects Act of 1977. The knowledge

development effort consists of hundreds of separate research,

evaluation and demonstration activities which will result in

literally thousands of written products. The activities have

been structured.. from the outset so that each is self standing

but also interrelated with a host of other activities. The

v
framework is presented in A Knowledge Development Plan for

the.Youth,EM lo ent and Demdnsttation Pro'ects Act of 1977.

A Enowle ge Development P an- or e_Youth 'Initiatives Fiscal

1979 and-Completing theYouth.Agenda: A Plan for Knowledge

Development, Disseminatimand'Application in Fiscal 1980.

Information is available_ or will be coming available from the

various knowledge.-development activities to help resolve an

alMoSt limitiess.arrayof issues, but answers to .policy

'questions will usually require integration and Synthesis

irom*.mumbet-of separate prodUcts; which, in turn, will depend

on knOwledge and.availability of these products. A major

shortcoming of past research, evaluation and demOnStration

activity: has been ,the failute%fii,organize and disseminate

the products adequately to,assurefhe full exploitation Hof

the findings. The magnitude and structure Of the youth knowledge development effort puts a premium on organization and

dissemination.

As part of its knoWledge development mandate/ therefore, the

Office .of Youth Programs of the DepaitMent of Laborwill

organize, publish and disseminate the written products of

all major. research, evaluation and demonstration activities

supported directly-1v or mounted in conjunction with the

knowledge development,effort. Some of the same products

may also be published and disseminated through other channels,

but they will be included in the structured-series of Youth

Knowledge Development Reports in order to facilitate access

And integration.

The ___,____,.edeDeveYouthKnos Development Reports, of which this is one,

are divided into twelve broad categories:

1. KnowledgeDevelopment framework: The -products inthis category areconcerned with 'the structure of'

knowledge development activities, the assessment

methodologies which are employed, validation of

measurement instruments, the translation of

knowledge into policy, and the strategy for

disseminating findings.

2. Research on Youth._ Employment and Employability Develop-,

ment: The products in this category represent analyses

of., existing data, presentation of findings from new

data sources, special studies of dimensions of youth

labor market problems and policy analyses.

3. Program Evaluations: The products in this category

include impact, process and benefit-Cost evaluations . of youth programs- including the Summer Youth Employment

Program, Job Corps, the Young Adult Conservation Corps,

Youth Employment and Training.Programs, Youth ComMunity

Conservation and Improvement.Projects, and the ,Targeted

Jobs Tax Credit.

vi

z 0
4. Service and, Participant Mix: The evaluations and

demonstrations sununarized\in this category concern

the matching of different typet of youth with different

service combinations. This\invoives experiments with

pork vs. work plus remediation vs.. straight

remediation as treatment options. It also

Includes .attempts to mix disadvantaged and more

affluent participatts, as well as -youth with older

workers.

5. Education and Training ApproaChes:\The products in

this categdry present the findings of\structured

. experciments to test the' impact and effectiveness of

various education and vocational traihirig.approaches

including specific:education methodologies-for the

disadva aged, alternative edudation and advahced

caree trainin4.

6. e-EMployment and.Trahsition.Services: The products

ITI'TEls category present the flOrtn-Of structUred

experiments to test the impact and effectiveness of.

school-to-work transition activities, vdbational

exploration, job-search assistance and other efforts

to better prepare youth for labor market success.

'7. Youth Work Experience: The products in thiscategory

address' the .orpnization of work activities, their

output,, productive. roles for youth and the impacts.

of various employment approaches.

8. Implementation Issues: Atis category includes cross-.

cutting analyses of the practical lestont concerning

"how-to-do-it." Issues such as learning curves,

replication processes and programmatic "batting

averages" will be addressed under this category, as

well as the comparative advantages of alternative

delivery agents.

9. Design and Organizational Alternatives: The products

in this category_represent assessments of demonstrationsof

alternative programand delivery arrangements such as

consolidation, year 7round-preparation for summer

programming, the use oZ incentives and multi-year

tracking of ,individuals.

10. -Special Needs Groups: .The products in this category

present findings on the special problems of and adaptations

needed for.significant -segments including minorities,

young mothers, troubled youth, Indochinese refugees

and the handicaped.

vii / t
..- _ ,

.-

.__

.,...____---- 1 1. Innovative Approaches ) the products in, 'this

present the fipdings/oChose activities designed to'

explore new apiibiChesitiV subjects covered include

the Y,Outh'indenti'Ve".trititlement/Pilot Projects, private

Sector. initiativei/the.,hational:youth service.

experiment;' and ener4ir-ImiXiati4es in. weatherization,

lok..-head hydroeieettit.'deti'restoration, windpower

and the *ice.' -':, ::.:. ''.:. -'I. '' :

I.

.12.- Institutional: Linkages 'The producks.ln this.category,,,-

. 'will include studits"df inatit4io4a1.,ariangeMents

and linkageS as well .as,assigiMbnig'of deMOnstratiOn

activities to.enCourage-iiich linkageS:wiih/edUCa tion;

._ -- volunteer iroupth, drug abuse /agencies andrOther :,'

youth serving agencies. c, '4 , , . . --`,'f

. t., , ,,,:-- .; :. in each of these knowledge development categories :there will

be a.rahge of disCrete demOnStrition, research and evaluatiOn.

activities, focused on different policyf program and analytical

issues. For instance, all experithental demonStration projects

hay.e both process andftpact evaluations, frequently'undertaken

by differentevaluation,agents.Finding will be published as

they become avaliable'so that thete will usually be a series,

of reports aevaence accumulates. To organize these products, each publication is Classified in One of the twelve

broad knowledge development described in terms of

the more specific issUe.,activity.or cluster of activities towhich it is addressed, With-an identifier bf the' product and

what it repreienth relative, to.other products in the demon-

, stration., Hence, the multiple products under, a- knowledge

development -activity are closely interrelated and the activities in each, boad cluster have significant interconnections.

This volnme*on the Standardized Assessment System haS implications for most of the demonstration projects describdd in

the other categories. All the products in the nknowledgedevelopment frameworktrcategOry.are closely related. In

partiCular however, The Knoled e DeVelo Ment A enda provides

the conceptual underpinnings Q the YEDPA research, evaluation

and demonstration activities, suggdsting the policy issues

which will be addressed, through the use of the' SAS. Likdwise,

Knowledge Development Under the Youth.Initiatives discusses the

needs for such a standardized appraFiTin order to avoid past

problems.

, The SAS is an ambitiou6 Undertaking. It was developed'at a

breakneck pace while implementing the massive array:of

.demonstration projects ender YEDPA. There will inevitably

be problems in design andiapplication,, Yet standardized

comprehensive information-about hundreds of local projects . and tens of thousands -of :-participants and controls

12
clearly offers unparalleledvopportunity to improve our

knowledge of youth employment problems and how they can be

overbome. .

Many, individuals, particularly the staff of the Educational

Testing'Service, played a role in the development of this

system. However, the individual who conceived, formulated

and then implemented the SAS was Joseph Seiler, head of

research and deionstration activity in the Office of Youth

Programs dllring the first 2h years of YEDPA. Bureaucrats

tend to remain "faceless,-" but his accomplishment is clearly

outstanding and deserving of recognition.

"

ROBERT TAGGART

Administrator

Office of Youth Programs
INTRODUCTION

The Office of Youth Programs "Knowledge Development Plans'

for 1978,and 1979 included.structured sets of questions

toward which all research? demonstration and evaluation

activities were to be.directed. major end-purpose of that

ambitioUs undertaking was to generate systematic and valid

empirical evidence which would' provide policy related answers'

to the complex and ,perplexing problems of, continuous youth

unemployment. Ai noted' in the Overview statement of the

report, "Completing ._the Youth Agenda: -'AePlan for Knowledge

Development,-Dissemination and Application for Fiscal 1980."

"In fiscal .1978 and 1979, the major' emphasis was, placed

on 'knowledge development'. through a structured:array

of multisite demonstration projects, large scale

evaluations and coordinated research efforts. The

first ,year's goal 'was to assess alternative interventions

and delivery approaches, focusing on the broad issues

needed to-legislate more effective youth' programs. In

the second-year, emphasis shifted to more specific

"issues which needed to be-addressed in the administration

of improvement of youth programs, exploring the problems

of significant segments of the youth population, assessing

service components to determine how each could be improved,

testing longer duration interventions, and promoting

the integration of youth programs."-=

With these activities in place, the focus can appropriately

shift to the critical tasks of evaluation synthesis, dissemination and diffusion of knowledge, and bridging of the gap

between knowledge and policy/programmatic application..

One4aajor evaluativeun&rtaking is represented by the Standardized,

Assessment System (SAS) direated at-determining-the-relative

effectiveness of alternative program approadhes. Through a

variety of standardized pre-and post-instruments, process,

social-psychological, and demographic data, SAS represents

the potential for identifying that combination of individual,

programmatic, and social structural variables which act to

either enhance or block occupational, educational, and social

mobility.

The SAS data base isnational in character, representing

approximately 40,000 young persons at 190 project sites. The

size of the sample, the range of instrumentation, and the

variations in program structure are unique and unmatched by

previous efforts directed at evaluation of intervention

strategies and the work/education dynamics of low income

youth.
In the pages which follow, the-reader is proVided with an

account of the origin and development of the instruments

which are part of the Standardized Assessment System.-

Further, a detailed desdription of the experimental design

and data analysis piacedures to be utilized in this major

evaluative undertaking is included. Finally, a listing

of research and evaluative products are identified as

example& of the kinds of decision tools which can be

generated from SAS.

15

_ _
3 -

THE STANDARDIZED ASSESSMENT SYSTEM

In order to fulfill the requirement for eVaiiiating a wide

Varietyof experiMentai,And deMonstration youth programs

initiated under,YEDPA, a "CoMMOn core" of, .measures was

considered Ito, 40 an essentialleatUre of the knowledge

deVeldpment process. only in that AriTty-cOulduseful systematic

coMpaiisons be made between the ^individual project sites,

withizia given program, it well zt):ietween operationally

diffetentwOgraMs.of'differing_study design. Constraints

in the-=choice:-of-measuring-,instrurgents_and _the_ implementation

Of a, Standardized Attetsmeni'..Piem CSAS1:were necessarily

imposed by program 4i#4enOes andby the charabterittics

of, the.youth.'poPulatiOn served under YEDPA. ,Thete constraints

stem,,primarily, frOM,tbteer.SOurcei:

ik-diVerSityoigoalt atleast, differing program

priorities assigned to tpedifid training objectives.

Some,programt emphasize work.experience in public. or

' private sector jobs, some incorporate training for

specific job related skills and academic abilities,

while others stress general knowledge about available

careers or employment prospects (i.e.., "career

development") and may focus their instructional priorities

or -job search methods, Proper'behaviot in a job setting

and improVed trainee self confidence, Many programs

incorporate some aspects of all these objectives. The

bctiavioral areas (or constructs) to be measured must,

therefore, be broad-enough-to-cover a range of

intended outcomes and ,still be as common to all programs

as feasible.

, Technical problems - in the availability of appropriate

evaluation - instruments that nct only tap the defined

,behavioral constructs but are also deirigd-for--adol-escents and young adultt who are from economically disadvantaged backgrounds CETA qualified), are

largely minority group members.and also trend to . possess lower levels of verbal skill. Ground rules

for the choice of such test materials rests with their

adequacy in format, content, mode of administratidn,

'reading level and demonstrated measurement properties

for the youth population being evaluated.

Operational limitations which dictate'the need 'for

evaluation measures that can be administered by program

professionals with reasonable levels of`training and

withinreasonable periods of testing time (i.e., highly

complex work samples or observational assessment systems

extending over many tours or days would be considered

impractical). The measures should also possess a degree
4 - of acceptability (face validity) for those program

,professionals who must administer, them as well as the

enrollees who are iske&-to respOnd to them. That is,

the item content should be seen as "sensible" in terms

.of_ program_expectationvand objectives.

Major components of the Standardi,zed Assessment System (SAS) that

are currently being applied to a number of YEDPA programs

are-Contained in:

(1) Meaiures of-the participants' characteristics

'essential for describing- the sample composition of

a given program as well as for use in data analyses,of

program effects to serve as controlling

variables that allow for statistical:equating when

contrasts are made betWeendifferent subgroups of

AndiViduals). That inforMation-it.obtained from a form

designated as the Individual Participant.Profile (IPP)

that provides background and demographic data regarding

age, sex, race, economic status, marital status, etc.,

and from a short reading test (STEP Reading) used to

measure the student's level of verbal skill.

(2) Tn order to assess attitudes, awarenesses, as well,

as chahges during the course ofparticipation, there is

a psychometric battery Oomposed of seven measures

subsumed under 5 behavioral/Categories that are intendedtO assess characteristics of participants and controls

at the entrance point-as-well as-changes-between the

time.of participant entry and completion.

(3) In order.to compare and contrast program approaches

and tworrelate.these with outcomes, standardized

process information is gathered on each project,

describing its qualitative dimension inputs and costs.

(4)-7-A7performance --outcome-questionnaire-measures

the participant's status at the completion 'of the

program (i.e.; a Program_ Completion Survey), his or her

assessment of the prokam experiende; as well as staff

assessments of the participant.

(5) A follow-up program survey provides performance

indiceS regarding the nature and extent of "success"

achieved by the individual at the defined time periods

typically standardized at 3 and 8 months after

t rmination.

As tools that are integral to the overriding demonstration and

knowledge development role of YEDPA, the instruments chosen for

evaluation should themselves be scrutinized for their

effectiv_eness. t Some of the variables and scales applied

here will lirkely be found more suitable than others. Thus,

on the basis of their psychometric properties, particular

instruments may be found more applicable to certain types°of

youth programs or tO particular subgroups of trainees.
.)

- 5 -

Results of an examination of the quality of the measures could

provide the evidence for suggested modifications and improvement

in measures application and/or design.

It should also be understood, regarding the Standardized Assestment System, that no single-set of measures could be expected

to serve all of 'the'evaluative needs of all YEDPA programs.

Unique features in evaluation design, curriculum content

and_required objectives .of many.programs_are certain to remain

so that core data from the SAS are not intended to be the

exclusive form of measurement information obtained. Hence

projects have been encouraged to utilize their own instruments

for local evalaltive needs. Such' information, based on

measures of rSasopable quality, can be,expected to serve as

a useful supplement to-the overall yEDPA7SAS data base.

s.
l'HE.COMPONENT MEASURES, AND THEIR RATIONALE

The, instruments and each of the contructs in the SAS were

carefullyselected-based° upon the best available evidence

-of-validity; reliability-and-applicabilityto,CETA youth

populations. More comprehensive information would have been

desirable but was not-adthinistratively feasible. More

careful testing of the instruments would have been preferable but it was necessary to develop these measures while

implementing certain programs. The instrument's, which are

inclUded for illustration in. Appendix 1 to, this report (some

of thetest elements are vendor owned and may not he used

without the permission of the vendor) represent the best

possible compromise between the many constraints at the time

tiw System was implemented:

(1) Measures of Participant Characteristics

The measures of the SAS that define participant*

characteristics (age, sex, race, education, etc.)

are intended exclusively for individual demographic

or status information purposes. They provide the

information needed for describing the samples of

youth whohave participated in the YEDPA programs

and also serve as controlling variables that are

essential for .those-data analyses in which groups

are to be equated, or made equivalent, by

statistical means (i.e., since the 4reilipt could nr,t,

in any practical way, have been perfectly matched

or randomly assigned for- study purposes). The

two instruments used to 'obtain those forms of

information are the Individua/ Participant Profile

(IPP) and the STEP Reading Test.

o The-Individual Participant Profile

This dOcument is used to-record information for

49 items dealing with the participant's

characteristics as well as status in the program

and at termination, These,data essentially

duplicatethe standard information gathered for

each participant-in all CE TA: programs:, The first

29 of these items are largely .demographic and'

cover such information as the indiVidual's age,

sex, race, ecoRomic, educational and labor forct

status - all at time of entry into the youth

program.* The-remaining 20 items are, designated

*These first 29 items are also applicable to control group

sample members for those YEDPA studies using a control

group in theix--eValuation design,

iff
- 7.= -as the "Progi*Statudr,' items ithiCh indicate the

stains, of the4iaitio:4,aiii- at the time ,of,!program

cOMtiletiOn or teirAinatiOn. -These include such

inioiMatiOw.as 'tepnination dates, total

hOUit j:)0*.t4.0.4.0-oitiiig in the .PrOgraM,. ;whether

or not ,the-:'0i040,0*-::pi-OVided: the .participant with

idadeMid:Credit,,andiliPedifiC forms otiterMination

'statue--,Undei:,-*pOeitiv.elzand 4nonpOeitiVe"' Categoria:z.

_lietrof,,definitiOn04hich..accOMpanietj.fie

form definei;.eiCh.-iteM detail and'- how3t is

to be OOMPleted 15-y pieierik*iiiojoat

personnel fkoM -their` -:PiOject,-reCOrds-.-

Although: .he- item*, r?f,' _the

_ cannot, generally, COnsideked',46:,;PerfoirrianCe outcome' (Criterion) some, .few, of the iteta, in the Profile

aiei:obViOUSlY Uirabie:i* that waY. ; Poi example,

Cher "e are .items .dealing_.ith-,eMplOyMent status

WhiCh-..tO 'fOrms of '-rOUtcoMe

inforMatiOn -found' :in the Progiai, doMpletiOn EurVey. These- and. several Cither;IPP items can.. serve as a

limited check on the reliability of the data

obta-iiied in the ,survey by contrasting program - provided. IPP-AnforMation with participant-provided, surVeY:informai:ion._

o The STEP Reading Scale

This status measure was specifically compiled`'

for purposes of the YEDPA evaluation studies.

It is intended to fill-th-e--need-for-a-very-short- (ID to 15 minute) easily administered measure

of reading skill that would also cover a fairly wide range of the reading levels 'likely to be - found in -the 'YEDPA enrollee population (i. e.., an

estimated range from Ath to 9th grade" reading

level).

None Of the conventional (published) measures of reading rability would. appear to meet these particular requireMents since they are usually

lengthy ,,_,reqUire, .different forms of the measure

for ability levels and are

intended' either to define the Stud-int-el 'reading grade level, with some precisiOn-; or identify

speCifie Skill- deficiencies for diagnostic purposes.

20
+ - + ;

Thesole,application,:of:the score from this 20

A.tordWiding,'COMPrehenaiOn,measUre is to serve,

isa_kek-Viriable..f0k,analYtical uses in

grOup equiting:pn Verbal:("academic") skill level.

--- 1116-#itemSChosen for this short readin-§ measure

were selected from the SaErlocator tests, (ETS,

1970) ,-6Overing,4th,;t0-9tii'grade reading levels.

Those locator tetts:Are-Shortreading comprehension

- =thia-s441us e&itS4i#04*iniLT4quick_screpriing)____

devices for deciding which 14'01:710-t-ihe-completed_

sTEp'*hioVe0Orit,-teSts-isuitahIe- for

adminiStratiOntO a particular student. Appropriate

item aniltota300cale analyses were undertaken by

the'pUbliShOrt:during_:the'deveioPment of the

measures, to assure the .accuracy 'of the items

forreading,gradeleVel identifidatioh.

HOWeVerOrerification-of the suitability of this

SpeCifi0:40.-40.cOMpilation't for purposes of

YEDPA evaluation, can most readily ka determined

by the level of relationship between its scores

and:#00res. on some widely used, (pUblished)

reading measures), If a high degree of positive

relationship is found', this 20 item widerange az readingscale.can be considered applicable for

its Purpoie-in the SAS. StCh data are, expected'

to be available from at least one samPle-of youth

Program -participants during the course Of_the

eValOatiOnAita

(2)---ThePtychoMetri-c-tattery

140*res:chosen for incorporation in the Psychometric

Batterli-reflect YEDPA program objectives while still

'bein4,Compatible-with'.the characteristics of.:the

traini,e'pOpUlation and the 'operational constraints

iOf thelOtth-proj*40. As the starting point, five

behavioral categories in which trainee performance

cbangeswre,to.be effected, as a result of YEDPA

'NPic.),4t404a#1.04540.001,yere definedby

Department of _Labor'S'..Offiee of Yo"th -Programs.,

Were'aohiiderect-reaionabl , , common to and believed

to7e:oMpa4S-, the objectives of a vast majority, if

not-a1-64:-Ue4E5PAl*Ojects,and-Were:deti4nated as

(I) caree.dedition-making, awareness and :Capability

(2) self-1 e (3) Work attitudes (4) ?,tib search

capability an (5) occupationai-Stx-steteofirPing.

In order to. be compatikile with the YEDPA participant

population the measures chosen to represent those
- 9

behavioral categories should possess features that

make them suitable for low-verbal skill adolescents

from economically and educationally disadvantaged

backgrounds. CriticisM that has been leveled at

the design and admiWistration of conventional

paper and. pencil' tests used with such groups (Karp

and Sigel, 1965, Lennon, 1964, Potthoff, 1966),

would lead-, ideally, to a choice of measures that

'are relatively .short, pr'esented orally, contain

pictorial information (to supplement the verbal),

use a16Yel and styld:Of language appropriate for

adolescents and young adults of low reading skill,

allow for marking of item-reiponses directly in the

test booklet-ancl-permitcareful examiner attellion

'to individual respondent difficulties in the testing situation (i.e., small group administration)'.

"Off-the-shelf" .(published) instruments: suitable

for assessing required youth ,program behavioral

Constructs-that also meet such deSign'standards,

--are-relatively rare.. Some that might claim

applicability to "disadvantaged" Iiopulationi are

often of highly questionable measurement quality -

particularly with regard to demonstrated predictive

validity against meaningful criteria of.trained

performance following program participation. After

a review of the literature the paper and pencil

measures that were selected to comprise the

._psychometric battery are considered the most

suitable ones available, or known, for representing

the desired behavioral categories. Thosp measures

are deperibefil below, under each of the five overall

areas of their content and a brief summary of the

available research evidence that had been drawn

upon to justify their incorporation in the psychometric_

battery.

Two measures were chosen to assess the career decision

'making, awareness and capability performance domain.'

One deals with vocational attitudes shown to be related

to "vocational maturity"'of adolescents in making

appropriate career decisions. The second is a measure

of vocational knowledge dealing, with the youth's

knowledge of what is required for carrying out

different job funOtions.

o Vocational Attitude Scale - This scale,is derived from the

Career Maturity Inventory developed by John Crites

(CM/McGraw Hill 1978). The measure contains

30 verbal items found'in a longer 7S:itemAttitude Scale (Counseling Form B-1) that are

scorable as 3 ten-item subscales. Thoie scales

are designated'as "Decisiveness" in Career Decision

Making (CDM) "Involvement" in CDM and "Independence"

22
M. - 10,1'

in'Cp$: The respondent indidates'his or her

:agreement or disagreement -with each of 30

itatementi_aboutvocati.onal careers and

employment.

0 Early versions of the AttitudecScale (containing

50 items) had been utilized in an extensive research

project in order to study. vocational development

among students. Approximately 100 studies cited

by Crites (1971)i deal with the construction,

standakdization and.applieation of the scale..

In essence the Attitude' Scale ,has been shown:'

(a) tp.differentiate reasonably well between

studenti at different grade !levels, (reflecting

increases in vocational' maturity with increasing

`grade level), he,appicable across -a vide

range of educational, Curriculat and-demographic

gimps (Crites; 1971) including studies based on

disadvaniaged-Whites, Blacks, Mexican Americans

and American Indians, and.(c)- to be content valid,

with the scoring key, matching independent

experts' judgments of ".proper" item retpdonse

at a high level of agreeMent (74%).

The reliability. of (the measure (its internal

consistency) for the earlier 50 item version

differs' somewhat as a function of .grade level

with a range of correlations from .72 to\.77

found in grades 114:11 and 12. Test-retest

reliability (i.e., stabilitY.)'Aot the scale for

students tested and retested over a one year

interval was found to be .71 (Crites, 1977i.

These values far scale reliability were cited

by Crites' as repreaenting adequate levels fcir

an attitudinal scale of ,complex structure that

deals with.a developmental (i.e., changing)

construct.

Validity checks with various forms of criterion

performance involving vocational aspiration,

vocational: choice and vocational maturity, (i.e.,

relationships between'the Attitude scale

scores- and scores on thOse criterion measures)

resulted in correlations-ranging from the mid

.20's to the high .30!s: These included a

correlation of .38 between the Attitude scale

and the Readiness Fot Vocational Planning Scale

(RVP)":by,Giibbons and Lohnes (1968) for minority

group students in the 9th grade

00 23
- 11 -

o',Job Knowledgp.Test (Educational TeSting Service,

1978)- This component is_a43-item-sdalekcontaining

pictorial and verbal material dealing with various job

qualifications, requirements and tasks. The

itemS, in multiple choice format, require the

respondent to indicate the correct response to

questions ,about the specific occupations depicted.

Studies with the'measure have utilized samples

of enrollees in the Neighborhood. Youth Corps

(N.Y. C')' work experience program for adoletcents

and the "Opportunities Industrialization Center

(OIC) vocational itaininig.PrograncfcaOut-ofschool yOung,adultil: Score, distributions based

on-.neans,.virianOdS_And,kingeS haVe-been,fOiind

highly_siMilar:OVer-a number of=enr011ee:PaMples

for nalei and,fendIeiaS has the pattern- Of

inter-relationships between the Job Knowledge

Test 'and 'other Cognitive end:attitudinal-measures

(Freeberg, 1970; Freeberg, 1974 Freeberg and

Reilly, 1971; Freeberg and,Shimberg, 1974).

Reliability internal consistency of the measure

has 'been found to be in the mid - .70's averaging

.75. for CETA and Vocational School samples

(Freeberg and:Vitella 1979), with levels of .75

for a sample of-N.Y.C. males and .72 for N.Y.C.

females. As a scale designed to measure a

composite. of differing forms of job knowledge.

(salary, educational requirements, nature of the

work performed, etc.) these reliabilities are

consonant with what would be expected---,.baspd

on similar scales that have been attempted for

other studies reported in theliterature--and

at levels that justify a variety of test uses.

Validity was determined relative to a number

of criterion using N.Y.C. anaDIC-samples. Some

of the criterion information., obtained at about

the same time as the tests were administered, is

used to_cOmpute 'concurrent validity; other forms

of criterion information were based on longitudinal

data obtained at periods of 3 to 12 months following test administration and are4.1sed to .determine

predictive validitY'. The extent Of the relationships

representing those forms-of validity, using the

Job Knowledge Test and various outcome (criterion)

measures, are summarized 3.n the table below for

statistically significant correlations based on.

OIC ancI.NYC samples.

24

4.

N
e

lob Knowledge Test

.Validity Sumer,

Concurrent(C)

Criteriin Validity Criterion Sample or Predictive(P)

Me. asure Coefficient(r) Sample Size(N) Relationshit_;_

1

. .: Training ,Prolii*m.,,

.,: Adjustment,Factor. .28 OIC 00 3

115 P

'Positive- Job Orientation

tutor .-

.. .28 OIC (P) 189 P

PersonalSocial . djusrawst,Factor 01' OIC (M) 115 P

a Job Search Motivation Factor .28 NYC 01,11 120 P

-Training PrograkAdjusiment

Factor. '.25 OIC (F) 189 P.

Training Program Adjustment

Factor .17 NYC IF) 215.

: P

, work. Superviscir Rating .32 NYC.(M,F) 1C9 P

Counselor Rating .25 NYC (4,F) 109 4.1

1 6 . , Counselor Rating .19 OIC (1,F). 220 P

Vocational Instructor's ,.20- ,OIC (4,F) 261 'P

Rating Posttraining Employment

(No/Yes) . .22 NYC 00. 104 1

Counselor Rating

. .22 NYC (M) 112 C

,Counselor Rating .19 NYC (F) 102 C

Work Supervisor Rating .24 NYC (M) , 112 C

p

1Criterion measures, designated as factors, were derived from independent

:clusters of .outcome- variables obtained by factor analyses. .The

clusteri were, injurn, used- to compute factor scores that served as

thic.- dependent (criterion) measures. ,A listing of the variables, that

make up the factor pattern' is-found in Appendix

2All validities shown are statistically significant (i.e., P=.05

confidence level or better)

3

M R Male Sample

0 Female Sample

Mil = Male and Female trainees combined in a single sample

2
13

The JOb-,,.j(noWledge--ifeasUre'i:exhibgtlit-.1tt, ( validity (prey ctive'Ly) for- work "site;.,°supervisor ritingi: general . fifa i411.4%,;Prq40.3* Aiict Job =Adjustitteiit. criterion °

faOtors crtr,SNin40 to 20:10-4: The . -measure,-446:lholie-!,a.'10;w44t,s40;ificant :level Of validity for ,he,114,:ghly,fiMPOttil*,Criteripn

,.of Whether or. no:E::..the:: trainee- obtained_

ernplormnt f011OWing;:triOling- tr. ce :for N.Y.C. , lnales);.- . o Self Esteem Sdale (EdUcational :Testing . TM) - It the -often --eXpresied intention_ of

youth program prok efe2Si§na*.,S,, -'enhitrice, participant's iSerifOrfai Value,. or self-4orth, -ekpe-O4tiOn:that':iitiproveil self -perceptiOnSi'StitaUlat*-0Ore-,:-SUOOetii,,, oriented socia.' and .vocational' ,adjustment

behaviorS.: One rrieascire vae-choSen't6-aetine. the

level at. whiOh, the program partiogkini rates his or her peksOnal Tht,, self-esteerit;s0aIe is

a! 15 item"- SOal,e' containing .pictorial and Verbal material used to assess perceived self -worth in

terms of- expectation for acceptance ,Cir - . achieveMent,.in various social, vocational' and . educational settings. The resoondent:_kindicates-, on.:',a three point scale, the degree -to` :Which he

would successful :or receive acceptance in the

specific situation portrayed. . Studies to support the, value of the Self

Scale' are based. n the same set of dtita4Obtained

for development and research. use with the Jab

1(nowledge Test, discussed above, using N.3,C.,- and OICstudy samples.

'The or, internal' consistency of this' 'saale.

has been shown- to a.irerage .64 for CETA and vocational

student .samples and in, the mid .50's for N.Y.C.

,,samples. These relatively_ modest levels of useful: reliability represent the lowest ones obtained .;

:or any ,of the measures 'of the. psychcmetric battery.- That result stems, in part, .fkom the design of the-""` scale_ as. relatively short (15 items), as well, as the

toihrile.lity of its item composition dealing with

estimate of different 'perceptiOns of self worth in

a_, varlet of specific job and interpersOnal , sitUati s,.*

*Measures that tend to be more- complex in. their content, by

_ .

. a- sainPling- a broader range of the construct under cosii cation,

generally stand a -chance of -sbow'ing better validity against --_trtioe .001,14_ criteria (eig. , fob. adjUstment).

r C

'4'
- - ^- , 4

- The: validity-4 the self esteem scale using a

8tatiety..Of ''performance 'outcome (criterion)

measures= is .summarized in, the . folloviing.

'table; WhiCh'`fiaS, thi.-Saide- format as the one

presented' above for. -the fob Knowledge Test and

also presents Statistically iiignifibant correlations

only.

0

a

0

..."

.7

I
".Z, 1, 44 NIA

-; 15 -

Self-Esteem Test

Validity iuggua

. - Criterion Validity Criterion

Coefficient(r) Sample

Sample

Size(N)

Concurrent (C)'.

or Predictive(P)

Relationship

positive Vocational, and

Social Attitudes Factor' .34 NYC (M) 182 P

:Traiping Program justment

.20 OIC (M) 115

Training_Program.Adjuitment_

Factor .16 OIC (F) 189

C

Overali.Social and

Vocational Adjustment Factor .26 NYC (M,F) 120

Counselor Rating .34 NYC (M,F), 111

Work Supervisor Rating .24 NYC 0(,F)

r 111

Remedial-Skills Instructor

Rating .18 QIC (4,F) 134 P

Work- Supervisov Rating .21 NYC (F) 128 C

Moderate levels of predictive validity for the Self Esteem scale

are seen for a.criterion factor that measures vocational and social

attitudes-(r-is-z.-34-fot-L-MTinalciiereales). Another notable

validity efficient of reasonable magnitude can be seen in the

tendency for trainees who have higher levels of self esteem to be

the ones who are rated as being better adjusfid'(i.e.,.more

proficient) by their guidance counselor Cr .34 between the scale

and the'cotinselorratings for an N.Y.C. sample).

28
o Work Relevant Attitudes Inventory - Measures of

work attitudes are ntended to deal with the

youths' views about jobs, 'the importance

of working, appropriate ways of behaving in

job settings anclgeneral feelings about one's

capabilities foesucceeding in a work situation.

Of the two available measures chosen as most

Suitable, one deals primarily with personal

feelings about work, relations with people and

life adjustment in general, while the other

deals with judgments regarding how .one would

behave in,specific situations that might arise

in a job setting. The Work Relevant Attitudes

Inventory (Walther, 1975), contains 15 items

in a short.form,that had been developed.by the

author from a 'longer 26 item measure. The. 16

items provide not only'a single total scale

score, but can be scored on the basis of three

-factored-subscales-definedLas,-!OptiMisW4

"Self Cobfidence" and."Unsocialized Attitudes."

Responses to each of the attitudinal statements

are based on a 4-point scale ,of degree of agreement with, or applicablity of, the statement.

0 Studies carried out for development of the'WRAI r. were based largely on samples,of Neighl,orhood

Youth Corps enrollees and students in experimental

.eduCation programs, (e.g.,, youth correction centers,

MDTA, vocational high school). Studies were

based, primarily on longitudinal samples with the

measure administered and readministered over.

several time'periods for a'number of the samples.

Measurement'charebteristics of the .WRAI indicate

that it is a somewhat'mpre effective instrument

for use with females than with male adolescents.

The reliability of-the total scale as assessed by testretest stability over periods of one year and more

is repotted by Walther (1975), for N.Y.C. enrollees,

to range from lows in the .40's to a high of .67,

with. consistently higher reliabilities found for

females. 'or an attitudinal scale with multiple

underlying factors, the measure can be considered

to possess useful levels of score reliability for

evaluation. purposes.

20
- 17 -

Using various longer (;34 item) and shorter (10.item)

versonions'of the'WAI,validities.were found

to be significant W-the'biail oft (a) the

ability of WRAI scores to differentiate between

black male highschoOi.,dropOuta. who made "good"

,andlpOor" idius*enti to work, and (b) pre7 dictive validitY-dfrthe scores for l criterion

J7-16f-vork.adjuatMent,r*tingS Obtained frors-several

Monttili7-tO- as longl'AS,2-4/2_yeari after administration

Of the.*RAII,4igrakic4pt.correlatiorii obtained

ranged frOm.atigh-of .44. to a low of` .26, for

females; with thi,Ughest_Iyalue OZ males%,being

an,r of 31.. With:a.total.sampre.ok Males'and

Ifemaies coMbine4 the-predictive-validity was

highest-far a. second adMinistration'of the WRAI

against the Eaork.adjustment -rating criterion

= .38).

st . 0 Jo) Holding: Skills (EducationalTeating-Service,

1978) - Thia'psychometric-battecomponent deals

. tfiith respOndett-awareness of.appropriate'on-thejob\behaviors in settings that depict interaction

with supervisors and coworkers. Thia 11 item scale,

containing pictorial and verbal material, requires the,

trainee to indicate which one of three alternatives best

define what his or her response would be in the

situation described. (Response alternatives.

have been scaled to represent "most",to "least"

acceptable behaviors for maintaining employment.) . EVidence-for the suitability of the measure is

bases on longitudinal samples of NYC and OIC,

from \which item and scale characteristics:were

determined, and is derived from 'the same data

set used for the Job Kndwlejage and Self-Esteem

Scales. 'I

1 Thg reliability internal consistency of the

'measures \has been shown to be at moderate

levels of\ .74. for a sample of-N.Y.C. tales and

.72 for N.Y.C. fethales. Samples of CETA and

vocational high school-students produce reliabilities

of airoxiMately AO.' These levels can be

consi erect sufficient for an attitude scale that

. is to be used foi YEDPA evaluation purposes -

especiallione of such relatively short length.

1 ,. . 'Validity as found in studies with N.Y.C. and

OIC simples is summarized in the table below.

F.

%V'
-

Job -1 dlding Skills Test

Validit Summary

.

. Concurrent (C) .:

,Cilterion Validity Criterion: Sample or Predictive(P)

Measure Coefficient(r) Sample Size(N), Relationship:

Social-Adjustment Factor-- --- .21 ------:------ -----7.. ------ V'ationalOriintitiOn,Factor .16

t

, Training Program Adjustment

Factor .34

Personal -Soci Adjustment

Factor .1*9 Training Program Adjustment

Factor .36

Positive Job-Orientation Factor. .21

Monetary-Expectation Feztor .25

Overall ,,3 anei Vocational

Adjustment Factor .29

Counselor Rating .31

',. Work Supervisor Rating .34

Vocational Skills' Instructor

Rating .15

Remedial Skills Instructor

Rating .18

Counselor Rating .22

f Counselor Rating .26

NYC (M)- 182

NYC (F) 215

-- OIC (M) 115

OIC (10 115

, - OIC (F)- 189

OIC (F) 189

OIC (F) 189

NYC (M,F) 120

NYC (M,F) 111

NYC (F) 111

OIC (M,F) /60 P

OIC (M,F) 134-

NYC (M) 112

NYC.(F) 129 C.

PP

PP

P

P , C 1
a

-

Awareness:of proper behaviors in a job setting

.is.seen to-be pOpitiVely related at significant

leve10-- to,a-vari4y Of'adiuttments achieved'

by the trainee. Thus,.reasonably good predictive

validity.:for _this <measure.lies in ;its relationships

'to'suCh,,,Criteria-at::tiainingiprogrzu,n adjustment

(r-= .34 /or-plIC-.Male*fand :36'for'OIC females).

-proficiency ,ratiUsisrbyleork site superviScrs and

guidance-- counselors-of N.Y.C; progr4ms (r's =. .34

.and- .31 respectively). and fOr overall social and

vocational adjustment following:training (r = .29

for an .24:TX. sample).

o Job Seeking.. Skills Test - To define the capability

of the youths to, exhibit a set. of'elementdry skills

essential for Undertaking, an employment search,

theJob Seeking,,Skills Test (Educational Testing

Service, 1978) ^hai beep adopted 'which is a 17

item "measure-,of job search. capability that.

samples some of the,skills needed to' ,initiate

an employment Search; interlpref information

about prospective jobs (in newspaper want ads)

and understand the information'requirements for

filling out a job application. Items in a

multiple-choice format require selection of the

one correct response to each question.

Evidence to justify the choice of this measure,

as most suitable for assessing the job search

construct .is again based on data from the samples

of NYC and OIC.

The-reliability-(internal-consistency) of the

measure, has ranged from the mid .60's to the

low .70's in study samples drawn from N.Y.C.,

CETA and vocational school samples. These

coefficients represent useful levels for the

evaluation purposes intended with the

psychometric battery.'

Valldity as found in studies with NYC and OIC

longitudinal samples is reported in the table

below: ,

IN
- .

Job-Seeking, Skills Test

Validity unu&±j

oncutreni CC)

Criteri Op Validity :Criterion _Sample or Predictive(*)

:Meisniet Coefficient(r) Saapla :Size(N) Relationship , Training Program .- ,.

't'rlt 'AA

Adjustment Factor .18 . INC (F) 215, P

toll) Success and ,. : Satisfaction Factor .26 NYC. (M,F) 120 P

Positive Job Orientation Factor .28 OIC :(F) 189 ., P

Tr "lining. Program.

AdjuStmeni..Fictor '...24 'OIC (F),- 189 . P .: Effective Job 'planning Factor .20 ,OIC-(i) 115 P

... .?

,ietsonal-SOcitil ,. t. . 'Adjustment Factor :21 ; NYC (M). 182

_ P

c, Counselor Rating .22( . NYC (11,F) 111 P

Work. Supervisor Rating --.7.31 N NYC (M,F) 111

, .. P

,'Posttreiming. Employment (No/Tel)' .36.,,', NYC (14) 104 P

*ostcraining4mployment (No/Yes) .21,: OIC <Ma) 157 P

Counselor Rating .32 , NYC (M) 112 C

Counselor Rating .Z61 NYC (I?). 129 C

Work .Supervisor Rating .35 NYC (14) 102 C

Work Supervisor Rating .22 NYC (F)' 128 C

In terms of the size ,of its significant Validity

coefficients over a: nuMber of _criteria-, the Job Seeking- Skills teit-co:,40-Consideied, one f the.

most vo.pit, of thel,-AS psychometric ' battery Measures.

.,. It 4gio--intider4i*, levels r cOnCUrrent, validity with

traj.ning' program counselor and, work, site supervisor proficiency ratings (r,*2.-32 and .35 for an NYC sample) , as well as predictive validity for .0680- :ratings--

though ,-at.,lower. for work sjte supervisors and for counselors). - It apparently possesses the best predictiVe validity of the various- measures for the employment -criterion (i.e., whether Or,,-,'-not 'the -former trainee found - .-emplstimerit):, with an 'r .36 for an N.Y.C. sample of nines and .21 for a sample of OIC males and females-

, dombined. _ o Occupational Sex Stereotyping

'Attempti to- meaeure:attitudinal perceptions

toward sex role .stereotypes in occupational choice have been relatively recent and .rare. The available

nStrüMts that deal with suCh-a.'constrUct

have either ,i*04_(a):404n104., zit, onngr-chi-ldren (for--
.5-

- 21 -

dOrelopmental study purposes);, (b) designed for and'.de,yeloped :With:eamples .of yOung adults who

possess reatonably,Aigh.Verbal (usually Majority..#oup college, students)''or

(c)-embedded. in-broader measures dealing with geneial; aspects of ,sex=role-.stereotyping:.beyond

the purely -vocational. Few scaleSdealing with

occupational, sek",StekeOtyping have ,Undergone

systenlatid..s.tUdy of :their measurement CharacteristiCi.and there is virtually no direct evidence for the validity of such Scales based

on'vocational PerfOimance ,criteria .(e.g job

entry, or. vocational adjuitinent criterion ,measures) .

One .available measure was chosen for use in...the

psychometric ",battery -to deal vith this ,behavioral

,category, or -construct. Howelier;..any measure Of

.vocational :Sex stereotyping- developeci-ihna far must be seen as than a 'sensible ,00mprOmiSe and of tenuous Value pending evidence. frOm youth program eValUation stUdits regarding its suit- ability for a YEDIiik poiiulation

The Sex Stereotyping of Adult Occupations Scale

was 'developed by Garrett, Ein and Tremaine, (19.77) and used with. elementary school children from

1st to 5th grade. This relatively short (21 item) verbal- scale presents job titles along with a one

sentence desdription of each job and requires the

respondent to indicate "who should be a " '"

(job title as giVen),. A five-point response scale

ranges from "Only Women" to "Only Men".

The 21 items are categorized and scorable under

three gender. designated _groups of "male ",

"female". and "neutral" jobs, with seven jobs

(items) assigned to each of those thiee categories. Although previously applied only to grade school students, the measure, baged. on its format andcontent, was Considered the most readily adaptable one for low verbal skill, economically disadvantaged CETA qualified) youth.

cs)

The choice of the Sex Steieotyping of Adult Occupations Scale is based on a coherent researchsupported rationale in the development of its item

content and scoring and'on4its internal measurement

34
C

r, 22

prOperties,(Garkett, Em and TreMaine,. 1977).

Thud, fOr*0:040.6.'the,chOice of an item pool was

based:on.achieVing:rePresentation of -jobs with

difierent.14veliof:educatiOnal, requirements, while

the-aiSignmentOfjtehtt0-gender categories was

baSed,A*Oentius,Eurea# information regarding

'-distribUtionS%bysekfov_the jobs described.

Items were also pretested in terms of the effectivenesa-Ofthe 5-point responte alternative format

and- selection Of iteMS- for the final-scaleAwas

based On item measiirethent chatacteristiCs along

with analytes:of seianclgrade level effects.

Reliability internal consistency of the scale, as

is reported as ranging frOM levels of .85 to .90

for a samPle,of-eleMentar;y school children. An

examination of test-retest, stability by Garrett,

Em and Tremaine,-(1977 Oheicied reasonably Stable

Scale means.Over:a-bne month time period in a

grade school sample.

validity in the form of direct criterionrelated evidence has not.yet been demonstrated

for this (or any other) measure of occupational

sex stereotyping. Such validation may be

considered. part of the YEDPA knowledge development

effort and will be undertaken as performance

criterion information becomes available over the

course of the evaluation studies.

.Each of 7 measures of.the psychometric battery

discussed above were chosen, not only for their

measurement quality, but also because

administration requires a relatively short period

of time. The total psydhometric battery can be

'administered-within-A-reasonable time and, ,where

required, in a single' sitting (e.g., especially

important in attempting to test control grou,,

students). On the average the total time for

administration has been found to be about 80

minutes.

An Administrator's Manual is provided along with

directions for oral presentation of all .test items

to small groups of respondents. Item responses

are marked by the respondent directly. on the

page .where the item appears. The psychometric

battery is available, in English as well as in two . Spanich language translations - one form representing a colloquial language style and the other a

more formal style.

35
123'

Project. and Process Information

To :gain a-bettersunderStanding of the factors that

md4a6dOunt-fo-differential program effects on ,

Articipants, it-it.neceOary-to-knoW the specific

_r Imrvice:.tiX;,_the,inten#44y of services, the types

of expenditures, : the level and qualifications

of.ttaffithe stabilitfpf.operationsvand the

project cOpleXityas:JUdged by linkages: bile

tbe.various-demonStratiOns'have,a variety Of - organizational.arringpments äid fUnding routes,

it is at the local,p:roject level where effectiveness

is determined. .Even though the demonstration'

designs_tend'to hold constant certain key elements

of service hiiclh-d- client-Iroupsl-there_are___

significant variations among-projectS which

explain7manyof-the-differences-in7-outcomess A

battery 'containing' six sets Of questions -seeks to

translate -key site-specifid variables

quantitative terms. First7-bagid-InfOrmation

is gathered,,about the sixe and sponsors of the

project as well as the setting. Second, the project

is described in-terms of component services and

activities as well 'as principle goals. Third,

the linkages involved in the project are described.

Fourth, there is a profile of the staff involved

in the project.. Fifth, project stability is

assessed as well as the .position of the project

on the learning curve. Finally, the project costs

are measured in a standardized fashion.

These descriptive measures were developed by a panel

of practitioners and, process evaluators based

upon judgments about the key dimensions of local

----pTogram effectiveness and the ways these-could "

best be described. There has been very limited

previous researCh.linking.such proCesa measures to

outcomes and the use of the specific descriptors which

havejoeen developed is clearly exploratory.

4: The Performance Outcome Measures

The outcomes of programs are determined by a series

of indices applied at completion and at two

periods following program departures. These indices

define the nature and extent of "success" or outcome

achievements realized, Loth in the "short term (at

program completion) and over the longer term (postprogram time periods). Two-questionnaires contain

the items developed for this purpose. 'One is

designated the "Program Completion Survey" and the

other is the "Program Follow-up Survey."
-zz:"-

.

'

. - I -2,4-

-

g -Proqratil ,Coripletion..,SUrirey

ar ' l + thiti7-queStiontiaire--COritaihil:44.--iteras most of

-Whioh_arel.hraSed'aueitiOnls to be presented

to-the-,i,00*Prograittpartidipant at the time he or she has 600010ted. ,60Sileaving_,the training

prOg*: 1they-coVer:;thil*rtiaipant!s.activities,

in the ,;program, ::attitUdet, toward the program,

job,and_ed00AtiOnal*piiitiontOund.-!exPectationsand:ipbial ,,,Ccii*Unity'AdjUstMenti. 'these

questions are iritendedfOt_Oral presentation,

on an! indiiidüal basis, by an inter-Viewer, with

the enrollee's item reapOnses-tO:be:reborded-by,

the interviewer. ;(TWO items- obtained,aby the--

inier;Viewer,trOm-00,iirbgra-ii106 are number

of-days_abSent ,from ct the program ancl,number a

wOrkIiite4ie-anyi-i6-WhiCh the trainee was

7 liWgilid-dukingtraining).- Other program

_coMple_tion;:pUtpOrt*MeaSUrea4obtained:at the same . timeare-tWo_p#ofiCiericy-rating scales. one is

an 11 itercidounseiOriX,Rating Form, to be completed

by' the yoOth,prograin4rofeasional who had served

in the gdidande'and counseling role for the

participant. The .Other scale, containing 10

items, is the Work Supervisor's Rating FOrm to

be-completed by the participant!S work-site

supervisor (for those participants enrolled in

a work. experience-program).

A.number of the questionnaire items can be

grouped to yield rheaningful subscales that

define outdome.dirnensions of (1) Work Motivation,

(e.g,,,,feelings-of.vocitional adequacy, saving

of money, willingness to accept further training,

motivation ttiwork, awareness of job characteristics),

(2)i Social COihMunitY AdjuitMent, (e.g., family

adjustments, getting along with people in the

community,, avoiding',PrOblescwith law enforcement

agencies), (3):Training"Program,Adjustent and

(e.g.,'Oountelor",proficiency'ratings,absences

from the training program,. wd±k-site supervisor

pro-idienCY rating#,. working on or having obtained

the GED, knoWledge-ok,desited job), (4) Vocational

Planning doMpetenCY,, (e.g., level of long-term

and short-term job plans, ability to accomplish

plans, knowledge of what to ask a job interviewer,

awareneit, of job characteristics, quality of first

job tought.)

I

"-

37
e "Peri Ref" _gueltionnaire 'containing similar

itisa,material tor use with control

group-meMbere'."anCie -ciesig4tied the Olintrol.

GrOUpl'Statiis.-SUr'veY0°- PrOgranui whose, evalUttion,design' calis. for

-%-inCorPOratialii of asoontrOl. group.

:o -Folloi-up. :Prograini-.SurVey

This '.50. iteia4WelitiOnnitire- is als0 to be

?idMinistered; orally, Yong aM individual basis, by

the..iritervieWer who is to reCOrd the partiCipant's

.responses,. The :Sun* sis! presentli :intended for 3; .months after the ,participant -haiLiegt the training; .PraVign and again:, at -8 Months fallowing-Prbgrani- participation. _

items -deal. ft-rat' -1:Y participant's part=

training -:eXperiencel iir,arear-,cf employment,

,education; and` social 'adji;stments and futUre plans.. In.iidditibn,- there is -a 5 item -Employer rtatiiig Forst which iisto,helcoMplated by the. present feir-labiti recent) emPloyer. (Permission

to interview the-employer Muet ,be granted by the youth4

Performante diteneions isubscales) that 'can be

formed for scbring-responses, by grouping items

--of the Survey, appropriately,-consist of: (1)

General' . Social. and Vocational Adjustment', g.

emploYer. proficiency 'rating, .:family adjustment, community adjustmenty amount of trouble with

police, employed 'VI'. not !asuplciyfed at the time

Of interview,. ,saving of 'Money, perceived impOrtance

of keeping, Out of trouble),. 12) On-the-Job SUccess

and tatisfactibn, .employment and earnings experience since.leaying, program, starting

salary in jOb, .tc0, -pastry expectations,. hours

worked per*eek, level of lob latisfactiOn, extent

to :which job :expectations were That, amount of , .,echopl.attendance)4.- -(-3)' Job Search Motivation,

(eg4,1 "number,;,:of:14.aces: 'interviewed, number of application' filed, ,fiumber of sources used to

find gitat job, .in.gaber -Of' visits to State Employment Service,.--t*e .splayt on 'Present or

last job)-,- 1.4). Job: Planning Competency, (e.g., quality of' next, job' sought, level .of short- and '10:V-range career Plans; quality- of .present or 'last 'amount .Of echool attendance), (5)

Short-Ranger-Job- SuCcespi a (e.'go tune to find . first job, number of places interviewed, saving of money, .time oh present or last jobi, obtaining of salary,raile, -knowledge of next job sought)
1

- 26 -

A parallel version of this.survey is also'

available for use with control group members

and is designated as the-"Control Group Followup Survey." For all versions of the Program.

Completion and Follow-up.Surveys, detailed

interviewer instruction forms are provided that

deal with guidelines for item presentation.

The choice of items to be incorporated in the

performance outcome instruments was based, in part,

on previous research designed to show their

applicability as relevant performance dimensions

for youth work training program enrollees

(Freeberg and Reilly, 1971;'Freeberg 1976). That

research had included firstp.the.rational choice

of variables that encompassed stated objectives

found in enabling manpower program legislation,

-information obtained by examination of worktraining program practices and curricula, and

interviews with program professionals. Second,

these variables, representing desired performance

outcomes, were administered to cross-sectional

and longitudinal study samples of youth-work

training program participants and analyzed to

determine which of them formed uoherent'and

interpretable clusters. In addition, the valueof the various short-term outcome variables (i.e.,

at program- completion) was examined by determining

their relationships to the longer term (postprogram) variables..

Resulting clusters of outcome variables, as

summarized above, were found 'to be similar in

studies with enrollee samples from NYC and

OIC programs. Major variables that comprised

the performance outcome dimensions were, therefore, chosen for incorporation in the outcome

surveys fdr the SAS. This provides not only a

justification for choosing the items of

demonstrated value (relevance) but a way of

obtaining meaningful grouping of items for scoring

purposes.

O
-.

27 .g

VALIDATION OF SAS INSTRUMENTS UNDER CURRENT DEMONSTRATION PROJECTS

InfOrmatioA pre.,.ented. previously 'dealt with' ihebackground,

,rationale:and aailable'evidence used to support the choice

of measures incorporated in the.SAS.. These instruments and their

. applicability will. be further refined and validated as data be-

. come available from the'deMonsttation prO3ecii. in which they . are-applied, One of the first-demonstrations to be launched

utilizing the.SAS is the Youth Career Development-or YCD

deMonetration which provides counseling, career information,

lobseardi assistance, placement, motivational training and

other transitionaliiiervices to high school students. The

psychoMetric'bettery of pre - and post-tests are available for

.3000 participants and control group students. Outcome

1Criterion) infOrMation".hat also been obtained for this in-,

ichOOl-aimple from the Program Completion SUrvey, administered_

to thoSe high schoolsenior class, participants who had remained

enrolled in YCD for 60 hOurs or longer. Finally, three months

-folld0-up-date are available for nearly 800 participants and'

controls who were served' during the course of treatment.

71.1-L, there is an opportunity to utilize-these short-term

criterion data, from one of the earliest YEDPA programS, for an

initiaiexaminatioit,of: (a) the internal characteristics of

the SAS measprei-When applied to participant and control group

samples (i.e.., means, standard. deviations,, score ranges', reliabilitiei),

00 some limited forms of construct and criterion-related

validity for-each.of the seven measures, and (c) possible

contrasts with-findings from prior studies --presented earlier

in this section - -as an indication of the stability.of the

measures and their suitability .for evaluatioifpurposes with an

in-school YEDPA sample Of high school students.

The following Table present's the scald characteristics of each of the

seven sets of measures in the psychometric battery. From these

results it can be concluded.that:

(1) Mean scale scores, standard deviations and score ranges

represent reasonable values for the distributions of

*these measures when used with a disadvantaged,

CETA gualified, largely minority group student

population--particularly so, since-this is a sample

that is relatively homogeneous by virtue of its'

composition.

From the means and variances, it can also be seen

that the measures generally possess sufficient

spread between-their mean scores,and the highest

possible score on the scale, to minimize any ceiling

effects ",topping out" on the scale).. A simple

index of the degree to .which a ceiling. effect is

likely to be.experienced with eacb measure is-shown

in column 4, as. ,a ratio based. on the Cdifference

3etween the realiAnd the higheit possible

score;diNiided by the standard deviation. Customarily

a mean that is at' least one standard deviation'from
a

2

Mean

,Cluireateriiititi- of the Psychos Natteiy Mealur:ett . . . . Pretest Administration: YCD- Seniors' ,

1' (N 1666, Participants; N 1590 Controls)

Test

SAW Measures Items Part. Controls

,locational,AttitUde 30 I 21.5 21.6

AA).

4ObAnowledge 1JK)

,-Job Holding Skills

lurk Related

Attitudes Inventory

144.1)

.Job Seeking Skills

(JSS)

lex Stereotyping

'(SS)

*Self Esteem (SE)

30 22.5- 22.8

11 30.7 30.5

16 -44.4 49.7

- 17 12.5 12.8

21 p45.8 45.8

15 36.7 36.6

4C:Omplete data cases only - , a Non significant MeaWrakfference between

,±,

**Mean difference significant at P <..01

41_

.C4

3

Standard Hey

Part. Controls

4.3 4.1

3.6 3,4

2.2 2.3

6:7 6.7

2.5 2.5

8.3 8.3

2.9 3.0

4 5

Ceiling

Effect In

Standardized

Units -Mean Diff

(Perfect Score-- Part.

Mean/SD) vi.

Part. Controls Controls

-1.99' 2.06

2. . 2.12

1.02 1.07

2.16 2.14

1.70 1.66

2.06 2.07

2.83 2.76

4

KS

NS

.01**

NS

Ns

; NS

NS

Participant and Control group students

6

If

11 Reliability

(Alpha)

Pirt. Controls

.76 .73

.67 6S

.78 .77

.66 .66

.90- .90

.58 .61

7

Score Range

Part. Controls

7i-30

1-30 6-30

/ N)

28-64 28-64

1-li 1-17

21-63 23-63

-22-44 22-45

42
-29r

the highest possible score would be desirable,

where 'the evaluation,measute is to be used in order

to show change,over time (i.e., a pretest; posttest gain-scote analysis). Although all psychometric

,measures meet that approximate criterion, the one

measure that is obviously borderline in its

sensitivity td,a ceiling effect, is the Job Holding

Skills scale. ,However, eilidenckof its validity

shown in prior- studies and some.of the validation

evidence to be presented:below ihdicatkthat the.

measure possesses a .sufficient degree'of useful

variance to qualify for the evaluative purposes of

this battery. Attempts to increase the difficulty

level (i.e.; reduce-the ceiling effect) of a Jobl , Holding Skills type of construct have tended tot

increase the complexity of the:job situations

portrayed to a point where it.becomesr one of cognitiveverbal comprehensioh,.- when used with a low velibal

skill population., -r rather than a measure of awareness

od how to behave in a. job setting.

(2) Reliabili* (internal consistency) of the 7 scales as

,(column 6)' `ranges from r's in the mid .50's for

the Job Holding Skills measure to ,90 for the

Sex Sterecityping of. Adult OcOupations scale.

Overall, the reliabilities are comparable to those

found in prior studies and represent sufficient

levels of internal consistency for evaluative uses.

The generally lower reliability of the 'Jim scale

is commensurdBeywith its having.the fewestitems.of

any)neasure In the battery (i.e., 11 items) and can

be considered a reasonable magnitude for such a short

scale with complex content.

(3) There is. ,a fairly high degree of similarity between.

the YCD participants and the control group members

with regard to their psychometric battery peiformance.

Diffvences between mean scores for the two groups

,are not statistically significant for 6 of the 7

measures. The one exception is the.Job Holding, Skills

measure which, although statistically significant in

its mean difference for the Participant and Control

groups (at the Pc 01 level), is negligible in terms

of "practical" significance with a mean of 30.7 for

participants and 30.5 for controls (barely one tenth

of a standard deviation).

4 lJ
- 30 -

This eocaMination of the validity of the measures, of the

psychometric battery is based on two types,oUanalyses from

available dete.of the YCDtOidgrem,- One is a brief and general

forM of construct velidationifor',Which the intercorrelations

of.th424eVen' measures can7(bet:used-to seOghether the broad

:Pattern:dt relitionehipi.is sensible or interpretable (i.e.,

whether the 0§taine4 pattern, of relationships is consistent

with,,the'donstructS intended! in the desigm.of the measures).

The mieCondMrthe.mo4e bre' itaht end.heaningful criterion -

related. : 0401 with e

av Validationfp0 a variety of performancelimited for.thian lysis to a !elect group of shortterm (program completion)-driteria of trainee capability and

Adjustment, \ ,,

. . "

. - 1 (1) Construct Validation

, The coilowing table presents.the 7)0 intercorrelation matr ces,

fdr the piychometric battery measures for the merged

participant and control group samples; The patterns

of relationships in' either sample are highly_ similar -

as might have-beenexpected from the considerable

'similarity already shown for their score distribution

indices and reliabilities. Other points to be highlighted are: .. . (a) The correlations, which are all statistically

significant at the .01 confidence level, show

sensible and logical patterns of relationships

that tend to support broad aspects.of construct

.validity. First, these largely vocationallyoriented measures are all fOund to have a'

positive relationship to one another. Second,

And more important, thi two major categories of

measures that can be reasonably defied, as

attitudinal' and cognitive, have their\appropriate

interrelationships within their appropriate

category. Thui,*the measures intended to

represent-cognitive ability constructs-4.e.,

Job Knowledge and Job Seeking Skills--dliplay

their highest correlations .with one anothtkr

(r sic .51 for the participant,group and .52\for

the oontrol%group). Similarly,' those measures . designed to assest clearly attitudinal perceptions

are most highly interrelated---e.g., the Vocational

Attitude Scale and the Work Related, Attitudes

Inventory show their highest relationships with

one another (r se .54 for Participants and .48

for Contras)'"in contrast to their ldwer

correlations with the cognitive.ability measures.

44
3].

Psychometric .11Sttery: Ihtercorrelation _Matrix*

YCD Senior Sam le

.(Participant N * 1666; Control N * 1590)

- VA JR JES WRAI JS'S SS

VA - 38 . 27 54 37 32

JR -.33 , - 30 38 51 24

J'fiS 29 31 - 39 32 21

WRAI 48 39 40 - 41 34 43

JSS '37 52 33 42 - 24. 29

SS 24 22 18 33 22 - 16

SE 25 24, 28 ' 43 31 18

* PArticipants above. diagonal; control group below. All r's are

significant at the .01 level. Decimal points are omitted.
^4.

O

AdditiOnal evidOce4*.support,of construct validity

can: be: °found in the relationships between the STEP.

7',Reading_T6StS-ind'thepSYChoMetric battery measures.

if-th-IFWWftiitUdinal constructs under-

:IyingthOleitiUreee lOgi0Wit Would' be.ekPedied

that the primarily cognitive: psychometric;- measures (JK

timild:JS5):woUI44emoTe 4441Yi:COrrelated 'with reading

AikiI1 than thOCttitUdinal scalese This is pre-

, cisOly what it -Skin to-oCur for the relationships

shovinl,elo*, between each psychometric battery

measure and the STEP 'Reading Test:.

Psychometric Battery

Measure

Correlation With

STEP Reading

VA .34

JK .41 ' JIM .28

WRAI .38

JSS .50

SS .25

SE ..24

(b) Among the four measures (1K, JES, JSS.

and Se), for which there is prior evidence regard-

- ing their intercorrelations, the relative size

Of these relationships-in the present YCD sample

is fairly similar to, previous findings with

youth-work training program samples of N.Y.C.

and OIC enrollees.

Of the psyChometric battery measures, the WRAI

is found to have the highest correlations, overall, with the other measures in thebaitery, In

part, this is because its attitudinal item

content overlaps to' some extent With the types

of items used in the Vocational Attitude scale

and the Self Esteem Scale. But, the measure.

also displays the strongest relationships of any

of theattitude scales with the two cognitive

measures of JK"and'JSS, as well as with the

STEP Reading Test. Thus; it is the one measure

of the epsychom6tric battery that most clearly

cuts across both cognitive and attitudinal

aspects of test performance. By contrast the

measure that has the lowest overall level of

relationship to the other SAS' psychometric measures

for either participant or control groups (i,e.,

the most independent-or-unigue measure of the

seven) is Sex Stereotyping of Adult Occupations.

The finding is a logical addition to evidence , for construct validity, in view of the relatively

distinct intent that seems to underlie the rationale

or purpose of the SS scale and the subject

matter that defines its item content.
3 -- 4:2) :'Criterion- Related- Validation,

0

;

74.1tteemliariablesl'66ritaihedm the- Program Completion

-Purvey, COstiiiite--.theSfiOrtterin trainee performance

',O0tocimeilCriteridnA0asUreWOlat. serve:* the basis

:thiti: initial examination.-0-the .predictive.

4'401W-4:each' battery

Theie 15 criterion variables ,are;:identified

the_f011OW##tabIO ighiCh,,presents thettatiatically

lid4nifidant..corrOiationS' (i4s,, validity Coefficients)

,Obtaihed between eaChofc the briteria and the

vsychoMetriC,:battery,pleasUrei In a 7 X:15 rectangular

matrix. ':141aSs.clita-are4reSentedfor the ycp,-

partiCipant.seni6-014601A(ibeem enrolled-in the )rogram

during the 1076-19,40ademic:10,4ri 'remained in.YCL

at ,prOgram servicea*!and

voluntari 41#00*)CoMPlete the'PrograM CoMpletion

Survey.

, . Three of the criterion variables from the Survey

represent multi-item scales -- i.e., the Feelings

of Vocational Adequacy Scale (6 items), the Awareness

of Job Characteristics Scale (6-items) and the Counselor

Rating Form (11 items): The respective reliabilities

.of the 3*scales are .56, .56- and .89 (as internal

consistency frOM Alpha coefficients). The shOrter

6 item scales are near the lower boundary of

reliability desirable for criterion mearlres (though

still. useful), while the counselor proficiency rating

scale possesses a substantial level of reliability.

)

For ease of interpretation, as well as more meaningful

and practical value in any discussion of results,

the validities shown in the following table are those that are

at r = .10 or greater and significant at the .01

-...,.(pqm .01) or better. From that table, all measures

of-thepsychometric battery are seen to possess some degree

of statistical/y.significant validity against a number

of thb,criterion measures at program completion. Of

importance ih this pattern of significant relationships is that all'are in the expected or logical,

direction, micHFhat a higher (more favorable) score

on an SAS measure is associated with favorable

criterion performance. It can be noted, that its is

also.true'for the three .r = - .10 values shown .on

the "Lowest Hourly Pay Acceptable" criterion in

that those participants with more favorable attitude

scores on the JHS WRAI and SS at program entry-tend

to be the ones who, at program completion, would be

willing to accept the lowest hourly pay in order to

obtain a desired.job right now (i.e., more motivated).

4,4
psychon!etric :Battery Validities Using 15 Prograi Completion Criteria'

4

PrograM'Completion Performance

, '0iiidole- (Crite11on) _; Measure

1 .Family.

. Feelings-'nbout Program -(40)2

2. Change in Getting Along with

Family (39)

3 0 People Giving Hard Time (44)

4 1 Days Absent From Program

5 Satisfaction with Program (12)

6 Trouble with Police (45)

7 Imp. Keeping out of Trouble

8 Job would Look for after Program

(34)

9 Job would Like Best Now (35)

10 Lowest Hourly Pay Acceptable (36)

11 Knowledge of Desired Job (37)

12 Willingness tEnter Training (38)

13 Feelings of Voc Adequacy (22-27)

'14 Awareness of Job Char. (28-33)

'15 Counselor Rating

Vocational

Attitude

Work Sex

*Job Related Job Stereotyping . Job Holding AttitUdes. Seeking ,Of,Adult, Self

Knowledge Skills Inventory Skills Occupations Estee*

(JK) (JHS) (WRAI) (JSS). .. 1SS) (W

V

.14

:13

.w

C

.11 .14 .17 .14 .18

t

.12

.10 .12 .10

.12 .12 .10 . .14 .18 .15

-.10 -.10 -.10

.12

.22' .13 .12 .26 .16 .15 .14

.16 .13 .23 .16 .27

.10 .15 .11 .1\ .16 .18

.17 .16 .20 .18 . .11

lAll r's shown are significant at the .01 confidence lever or better

Numbers in parentheses.refer to item numbers in the questionnaire.
..)

- 35 -

Overall, in terms.of number and magnitude of significant

relationthipi, the. Work Relit-ea Attitudes Inventory

possessesthe beSt predictive validity against these

short -term outcome,_ 4ith'Sts highest validities for

"the-trainee's expression of willingness to enter a

training ptogra*in-order' to obtain a. desired job

(r m ..26),.Feilings'of Vocational Adequacy .(r = .23)

and-Counselor Rating (r = .20). 'The WRAI is followed

closely in levels of validity:by the Self Esteem

measure and the Vocational'Attitude.Scale. The

patterns for those three attitudinal scales are

,fairly similar,,although the Self Esteem Scale seems

to evidence' better validity with sevaral social

adjustment variablei. than do-the other two-scales

Family Feelings'AbdutThe Program, Importance

ofKeeRing out-of Trouble and. the highest relationship-of all (r Luir .18) for Amount of Trouble 'With

Police.

Of the psychometric battery cognitive skill measures,

the Job Seeking Skills Test displays somewhat higher

levels of validity with the job-oriented criteria

than does Cob Enowledge.

Generally, criterion variables for which psychometric ' battery measures demondtrate their-best validity are

the'job-orient3d ones and the' counselor proficiency

rating; .alt le the. personal and social adjustment set

'Of crite.:4,-, family-community adjustments,

satisfact with the program,,absences'from the

program) are mmpaiatively negligible, There is

only one social adjustment criterion for Ilhich the

psychometric battery measures are almost uniformly

significant predictors :i.e., valid) and that is the

"Amount of Trouble With Police" only the SS measure

fails to correlate significantly with that outcome.

Those trainees who obtain high scores on the

psychometric battery measures at program entry

tend to 'report, at program completion, that they had ,

less trouble with police during the course of their

enrollment.

Any comparisons of these results-for an in-school

YCD sample with those of prior validity studies Using

similar criteria must be made cautiously (especially

since prior result's were based largely on unemployed

school dropouts in N.Y.C. and OIC work-training

programs). Not only .are there differences to be

expected between in-school a.xl out-of-school samples

from different types of youth programs hut', in addition,

many important criteria that were found valuable at

program completion in prior studies are not logically

applicable to an in-school program that does not

provide a stipend (e.g., work-site supervisor proficiency
t.t

--36-

ratings, trainee financial management of, income

derived from the. program, Whether or not the

trainee is working toward, ,or obtainedl'a GED).

Furthermore, for three of the seven measures of

the psychometric battery (i.e., VA, WRAI And SS)

c there has been little or-,no previous study'of their

validity with the assortment of criteria. utilized

here. ,Nevertheless, there are some general similarities-with,prior results for the four measures for

which such direct contrasts are possible. Thus,

for example, it can bemeen in the preient validity,

results that'the JICs MS and JSS measures each tend

to be significantly related to the counselor rating

criterion while Self Esteem is not. This is a

result obtained previously with an N.Y.C. sample

(Freeberg, 1970). In all previous validity studies

the four ETS' measures had also tended. to produce

significant validity-against the social adjustment

variable of AmOunt of Trouble With 'Police and with

\ the set of criterion' variables involving job-Oriented

\ perceptions of the trainee--as was evident in the

\\present study. The only area of discrepency, however,

exists in the valhe of the social adjustment criteria

(e.g., variables involving family.and community

adjustments, program absences, importance of keeping

out Of trouble) for which these 'four measures had.

tended to show a pattern of more extensive significant

validities for an out -.of- school sample than are

found here for high school seniors.

Although\the.magnitude'of the significant-predictive

validitieS obtained in thie.study are relatively

low, with most of the correlations in the teens or

low .20A, the patterns are indicative of the potential

value of these Psychometric-battery measures as

program assessment tools. Distortions that serve

to minimize the extent and degree of validity obtainable

are often a funCtion of (1) the quality of the criterion

data collected (for this study all data were collected

by the YCD project personnel) and (2) possible sample

biases from.restribtions'in_the range of performance

for the sample,from\which the criterion data could be

obtained. Thus, for\.the present study, validity would

be expected to be weakened by_ the fact that there is

a reduced sample of aboat 970 YCD senior participants

ivailable,at program completion ftom some 1,660

originally pretested at'program entry. Since these

students could be assumed\to represent those who

were motivated/ encjgh to rem!tin enrolled in the

programs it might also be assumed that they represent

a sample who are more restricted in the range of

their program completion criterion performance

'clustered, hopefully, at the high or more "favorable"

end of the performance distribttion) - a result that

would unavoidably -serve to minimize the magnitude of

the validity coefficients. obtainable.

51
- 37 -

In the following table, the long -term predictive validities are shown_

for:the 74syChometrio measureS using criterion variables from the

Preqri0,Followu04urVey administered to the iOD-partiCipints

three,,MotthS-aftei the end -of; senior academic year. The

liorrelitiOnt,10aliditiesrigeiehtea in rectangUIer matrix

repreeent ;those thatAre'significant.it the R = .05vOnfidence

orAdetteri,,iridorAi.megOitUdecif is - .10 or larger.

-The. .45:14VelNeslconsidered-tmore sensible standerChere .

Iratherthan,the,*(il leVel SOPlied-to the ihort,term validities in

ths4recediwtable,ibedauseof.the.corisiderPblY'reduced, sample size

4,4SilabIe SUrvey. In addition,

for-siiMplicity in.,suMmarisinjthe. patterns of validity, the

4SluisYakoS,Ohawn-onlY for thoimPost-ProgramoutcOrne,vaeiables

,fOr--Which-etjeast:One4iychOMetriomeSsiure-dii0laYed

si7nificant:PredictiVe'vaIiditY. Thui, there are 13 criterion

vat Cables: shown; in the -table out of a total -of 29 criterion

variables from the4011ow4U0,ftrvey that had been examined

in an .original 7 'x-29,mitrix. (Obviously 'the number of

Signifier:fit r's presented here 44-- indicate that

-*hers war _a greater'than chance number of significant r's

present in,that original matrix of 203 possible corrleations.)

As -With *1w:short-term (program completion) criteria all 7

measures of the psychometric battery possess significant

validity for least some .of the post-program criterion

variables. The one outcome variable for which all psychometric

Votary Measure:D.6re predictively valid, is the status level

of. the job held 3 months after training. Those who score

higher on-the -psychometric _measures et program entry (especially_

on -the Job/Knowledge, Job-Holding Skills and Job .:leeking Skills'

'Scales) .are.more likely -to occupy higher level jobs some 3 to

(months following high school conpletion.

The 7 measures as a gtouvare seen as most consistent in their

Predictive-validity for: (1) post-program performance outcomes

. of Job .Satisfaction (with 5 of the 7 having positive significant

relationships)'; whether- or not the former Ypliarticipant,

goes on. for further formal school or enters some form of train-

' ins lfor which 6 of the 7 measures show predictive validity)

and (3) status level of the job that the individual would seek if

he or she. left their present job (for 6 of the 7 psychometric

batter" measures).

- Of 41cial interest, among the predictive validities shown/

with the remaining job-oriented performance outcomes, are:

(1) the poiitiVe relationships betwIten former participants

expressions of how well they know the job they would like to

have,(Job Knowledge) and their scores at program entry on the

three attitude scales of VO0ational Attitudes, Work Related

Attitude* Inventory and Sex Stereotyping, (2) the tendency for

-those with .more favorable .attitude (WRAI, SS and SE) to find

their first post-program joiii more quickly and C3) a similar /-
;`, :0. -.38-

finding of predictive relationships, between. attitudinal scales

(VA and WRAI) and engaging in some farm of activity. with

"vocational promise" versus doing nothing (i.e. Current

ActiVity Status);

For the ol#comes bearing'on social adjustment, the two scales

predictivejof the importance of keeping out of trouble are. the

attktudinal.measures'of Job Hold" kills (i.e. the degree

of compliance with socially acce behaviors on a job)

and .Self Esteem'. However, there s a egativit relationship

found between those whoscored'hi JA, WRAI, JSS and SS

and contributions to family finances dilating that a

favorable-score on those measures is ciated with a lesser

financial contribution.

Similar to the findings for predictiv alidity of the

psychometric battery using th ompletion criteria,

the-overall results regardin: the valid ty'on those postprogram criteria indicates, again, that e WRAI and SE --

in that order -- reprekent the'two most slid scales of the

psychometric battery.

5 tr)
39 .

Predictive Valiaitiis eor Psychoustric liatfory Measures Using

. Thavelloete PollesOrp Criteria* t ,

Partkipeote

40*

)

gatiskiamlak

1. :bb_ Stains 32;e1

Premat .11W

2. ills To Misr

lit Job

Mad

4. lb. am Jobe

Coos

Job Satisfaction

6.. In SchOol ar

Training lima

.7. illshest Pay

8." lavel'ef;..bb

dolga (nut Job)

9. Job IttoV1adei

10. Pasily'Faelizsa

Atom Provo

:0

11. Weakly contgibution

to Pamily Mims

12. !mortal:lake(

ra*PirS gilt a

Trcubla

13. Activity Stew

(10p37,35,39)t0io Work CT Stool;

Plat ties wit or.

School; Pun Tire

link or School

311 44

,

.23

...,,i7

.17

41

2

(46)

(4,13)2

at

=.

\

17

-

.(-.16)2

(-i13)2

.16

-47

.15

-,.17

..

.14 .13 .19 .16 . .20

'119 .12. ,

s .10 .17

. . ..16 .12

,

t

. .11 .13

..

.16 .20 .22 .20 .16

.13 .14 .12

. .

. 4.1

-.10/ -.10 -.12

/

.10 .13

.

.

.12 .10__-

-

.

.

.

* All is .significant at the .05 Milan= lora ar betty

1 Sampleolse range !fie V 120 to 11 790

just short of significance at /05 Uhl; p .06
.

40 --

MEASURES APPLICATION*

The study design,and,sequence of measuresadMinistration for

thii'esipmation research was'developed to meet tLe need for

a comprehensive set of procedures' for all participating youth,

smnploymentPrograms. These procedures Specify time of

administration for each of the following infitruments in the

Standardized Assessment Systet:

r Individual Participant Profile (IPP)

STEP.Reading Test

,Pretest/Posttest:-Battery

a) Vocational Attitude

b) Job Knowledge

c) Job Holding Skills

d) Work Relevant Attitudes

e) Sex Stereotypes of Adult Occupations

f) Self Esteem

g) Job Seeking Skills

Program Completion Survey ,, Control Group Status Survey . Program Follow-Up Survey (3 months and 8 months),

Control Group Follow-Up Survey (3 months and 8 months)

/

.

, One of the more important features of. the study/design and

measures administration is the differentiation/among subgroups

of examinees with respect to the amount of time they participate

in their respective programs. In the case of out-of-school

programs (excepting summer-only programs), two critical time

standards were defined -- 10 program days and 3 calendar months.

Program-participants were then classified in one of three time

segments: JO those who remain in the program for less than

10 program days, (b) those who remain in the program for 10 or

more prograM days, but less than 3 calendar months, and (c)

those who remain in the program for,1 or more calendar months.

All of these groups (and the control group(s) are administered

the IPP (Participant's Characteristics Section), STEP, and protest battery during the-first week of the program. Likewise,

alLere administered the appropriate Follow-Up Survey 3.and 8

months after exit from the program. However. at the time of

exit. from the program, the administration design calls'for

progressively More comprehensive data collection from

participants who remailonger in the program. Thus, for those

who remain for less than 10 program days, only an IPP_(Pro(tam

Status Section) is completed; For those who remain-for at

least 10 days-but less t an 3 months, the posttest battery is

eb

added to the IPP. Tho who remain 3 or more calendar months

ake administered thd P ogram Completion Survey in addition to

t e IPP and posttest battery.. The Control groups, after a time

i terval comparable to the length of the employment program,

,are adMinistered the post-test battery and the Control Group

Status Survey.

55
The entire administration sequence for out-of-school programs

is shown in the following. Honorarium -payments are optional

for various participants and control subgroups, while mandatory

in othe;cases. For participants,,iayment is'optional in the

situation where a.youth leaves the program before being postteited. Fok controls, payment maybe made for data collection

at both the beginning and end of the program, if deemed necessary

by the program dikector.

It should' also be noted that although many programs have a

fixed start and end date common to all participants, there are

certain programs with differing start and/or,end dates for

individual participants -- the so-called "rolling admission!'

and/or-"rolling exit" programs. The above administration

design is also applicable to:these kinds of programs, simply

by defining entry into and.exit fromprograms.o an'individual

basis. The comparable preiest-posttest interval for control

group' data collection is typically set at the estimated mean

,pretest - posttest interval for program participants. In

Situations where two or more comparable control groups might

be availableT-consideration is given to the possibility of more

than one pretest - posttest interval for controlgroup data

collection.'

The-measures administration design for in-school and summeronly programs is completely analogous to that for out -ofschool programs, except that the upper time standard is set

at 60 program hour's rather than at 3 calendar months. Thus,

for in-school and summer-only programs, participants are

categorized into the followirq three time segments: (a) those

who remain in the program less than 10 program days, (b) those

who remain in the program at least 10 program days, but less

than 60 program hours, and (c) those who remain in the°program

60 or more program hours.. The standard in terms of program.

hours (rather than calendar months) was adopted in recognition

of the fact that in- school and summer -only programs are likely

to vary considerably \with respect to the compression (in time)

of their program services. Thus a standard in terms of

program time, rather than 'calendar time, was judged to be

more appropriate'. In ,almost all other respects, the

administration design is analogous to the previously

described design for oUt..of-school programs. The single

exception, noted below the figure, is that YCD Demonstration

Projects require administration of the posttest battery to

program participants who remain in the program for 60 or more

pro/rsi hours, not 10 or more program days as do the other inschc.:i programs.

Th' project and process information for year-;round or inschool only programs is gathered from project operators annually

with an April 15 report date. For summer-only projects, the

report date is August 15. The project and process information

material was implemented in fiscal 1980 so that summaries for

earlier projects had to be. reconstructed where feasible'and

appropriate. Sponsoring organizations for multi-site projects

are responsible for collecting and validating this information.
Those who remain in

/ - program for less than.

/10 program days,

Those who remain in

program for 10or more

prograi days,,but less

than 60 program hours

Those who remain in

program for 60 or

more program hours

Control Group

e. 4 2

Sequence of Instrument Administration

for In-School or, Summer-Only Programs

Three Months3

Wring First At Exit From (or After Exit

Week of Proaram End of) grogram From Proaram

Eight Month!

After Exit

from Pratte.

.Participant n Status - .Program .Program

Characteristics -

..Program

(IPP) Follow-Up Follow-Up

(IPP) Survey Survey

STEP

Pretests

.Participant's .Progrtm Status- .Program .Program

Characteriitics- (IPP) Follow-Up Follow-Up

(IPP) Posttests Survey Survey

STEP

Pretests

.Participant's .Progrfm Status - .Program .Program

Characteristics - (IPP) Follow-Up Follow-Up

(IPP) Posttests Survey Survey

STEP Program Completion

.Participant'! .Posttests .Control .Control

Charamteristics- Control Group Group Group_

(IPP) Status Follow -Up Follow-Up

STEP

2 Survey Survey Survey

Pretests

1 If you can-determine when a participant will leave the program, test in last 2 weeks

prior to leaving. If the youth leaves before testing has taken place, locate youth for

/ testing and, if necessary, pay youth $5.00 to complete testing,(this includes both the

Posttests and the Program Completion Survey, if required).

2Date set in consultation with OYP or contractor staff.

3Pay $5.00 to each youth who completes the 3 month Follow-Up Survey.

4If necessary, you may pay control group youth for participating in testin 'g. $5.00 for

Pretest', and $5.00 for Posttestand Control Group Status Survey.

NOTE: YCD Demonstration Project requires posttesting for any youth who has been in't a

program for 60 or, more program hours.
- 43 -

Sequence of Instrument Administration for

Out-of-School (Except Summmer-Only, Proarams

During. irst

Week of

Program

'At Exit From,

(or end of)

PrograM

Three Monihs3

After Exit

From Program

Eight Months4 , q After EXit

nom Prograi

Those who remain

in program for

lass than 10

program days

Those-whii,remaiv

in program for

10,cr" rte

proi:an lays,

but less than

3 calendar

months

-.Participant's

Characteristics

(IPP)

.STEP

;Pretests

..Program

Status-(IPP)

.Program

Follow-Up

Survey.

.Program

Follow-Up

Survey

.Participant's .Program .Program .Program

Characteristics Status-(IpP) Follow-up --Follow-up

.(IPP) .Posttests Survey Survey

.STEP

.Pretists

Those who remain

in program for

3 or more

calendar months

.Participant's .Program .Program .Program

Characteristics Status-CIpP) Follow-Up Follow -Up

(IPP) .Posttests' Survey Survey

,.STEP .Program

.Pretests Completion

Survey

Control Group .Participant's .Posttests` .Control Group .Control Group

Characteristics .Control Group Follow-Up Follar-Up

(IPP) Status Survey Survey

.STEP

2

Survey

.Pretests

1If you can determine when a participant will leave the program, test in

2 weeks prior to leaving. If the youth leaves before testing has taken

locate youth for testing and, if necessary, pay youth $5.00 to complete

(this includes bothothe Posttests and the Program Completion Survey, if

2Date set in consultation with

3Pay $f.0" to each youth who completes the 3 month Follow-Up Survey.

4Pay $10.)0 to ech youth who completes the P month ow-Up Survey.

last

place,

testing

required).

5f necessary, you may pay control group youth for partiipating in ing.

$5.00 for Pretests,'and $5.00 for Posttests and Control Group

5U

us Su
44.=.

BASIC ANALYSIS PLAN

comprehensive evaluation of the 'youth demOnstrationprOgrams

must (3) have bbth formatiire and a summative,component, and

(7) take advantage' of the longitudinal -'nature of the design.

'The.summative component. ii primarily.concerned with what works

"best" and fdr whom while theformative component etTERgts to

identify 12_2y.n particular' programs work. Traditional statistical

.methods atTgled-td data gathered in randomized experiments

carried out under-laboratory conditions can piovide information

on both what and why. Howeverfas one moves to field experiments,

nontraditional and, innovative Approaches to data analyses

must supplement not replace) the traditional approaches.

In a literal sense there is no "sampling" with respect toenrollees

at a demonstration site since evaluation data are to be

collected on the performance of all enrcl.lees at a particular

site,- The control group -at a palFibular site, however, does

represent a sample from a hypothetical population that is,

hopefully, similar toethe enrollees with respect to important

background and, ability variables. To the extent that the

enrollees (ind the controls) are similar to other populations

of inter3st (e.g., other CETA qualified individuals), we can

make generalizations concerning program effects to other employment

and training contexts, If the controls are found to be quite

different °from the enrollees, any, conclusioni about treatment

effects becomes quite tenuous regardless of the sophistication

of the:statistical adjustment used to correcting for the preexisting differences.

The smallest unit of analySis will be the enrollees within

site. This,will be the common unit of analysis when comparing

participant and control groups within site. The results of

such comparisons make up the summative component of the

evaluation model. When a particular demonstration program has

a number of sites and one or more control groups, the repeated

comparisons of treatment outcomes with control outcomes will

be considered independent replications within a program. In

the latter stages of the analysii, data on sites nested within

programs will be pooled across programs in an\effort to identify

what program attributes (such as processes an0or personnel)

influence; (1) psychometric battery test Score 'gains, (2) scores

on the Program Completion Survey (PCS) scales (3,) positive

program terminations, and (4) three month and eight month post

program labor market performance. Figure 3. In more complex

analysis, we will use both individual characteristics as well

as characteristics of the sites in the same model. The

"charac-teristics of sites will come from a process survey. In a

sense, there will be two units of analysis in such models -- the

individual and the program sites. Results based on the pooled

data which includes both individual and site characteristics

make up the formative component of the evaluation model. This

type of analysis will tell the degree to which particular

programs were effective.
45 -

The input description will include the following basic crosstabs, which will provide a population description by site

within a program. Frequencies and percentaget responding to

each item in the'IPP will be.tabulated within site by the

following cross-classification: (1) Ethnic groups by treatment

and control, (2) sex by treatment and control. (3) age groups

and-(4 totals, i.e.', simply breaking out each IPP item by

treatment and control.. When the Individual Participant

Profile (IPP) response is'on a continuous metric, means and

standard deviations as weil'aS the appropriate statistical

tests will'be presented in.the tabled output.

However, since findings of statistically significant differences

may simply reflect large sample sizes rather than practical

differences, all mean differences will be divided by the pooled

standard deviations yielding an index of group differences in

terms of standard deviation units. Thisapproach allows one

to compare more readily results from a large site with those

froma small site. ,In the social science literature a third

lalf a standard deviation. difference is -considered a

'medium"-effect while a one-fourth to -one -third is considered

a "small" effect (cf. , J. Cohen, 1968).

The above cross-classifications Provide sample descriptions by

site within program as well as information as to what sites

can be pooled within programs without raising questions of

interpretation in subsequent statistical inference. Sites

within programs which have 40 or 16-0-s youths in either the

program or control group will be considered candidates for

pooling in subsequent analysis of- covariance gain score

analysis7

The above analysis provides a description of the population

of participants and controls by site within program. A

description of the population of programs in terms of programmatic

and/or process variables will also be carried out in an effort

to (1) define program dimensions on which there is sufficient

inter-program variability to be useful as explanatory variables

in later structural or path analysis models and (2) provide a

description of the population being served by selected categories

of prograths. That is, resulting tables would present not only

the number of programs according to selected categories but

also the demographic characteristics of the participants being

served by each program category. Classifications of individuals

being served by types of programs would include: ethnic group,

sex, age groups, economic status and education levels. The

individual characteristics will be taken from the IPP.

Categories of programs will be defined by responses of program

personnel to the standardized Project and Process Questionnaire.

Program clasification categories will include:

CO
Is

T

2

TIME PERIODS

Pre-test Scores (SAB)

Items 1-29 of IPP

T

1

THREE MONTH FOLLOW-UP

. Post -test Scores (SAB) . General Social and

Vocational

Items 30-49 of IPP Adjustment

. Work Motivation-

. Social-Community

Adjustment

. Training Program

Adjustment

.Positive Vocational

Orientation

. From PCS and CGSS

. Employers and

Cohnselors Ratings'

Quantifiable Process

Information (SPQ)

. Cost Effectiveness

InfOrmation

. On the Job Success

and Satisfaction

. Job Search

Motivation

. Job Planning

Competency.

. Short Range Job

Success

. Working vs. Not

Working 9

. Salary Level

. Employer Rating

Items and/or

Scales

FIGURE 3

MRIABLES AVAILABLE AT EACH POINT IN TIME

( 0-i

T

4

EIGHT MONTH FOLt011e,

. General Social*

Vocational

Adjustment

. On the Job SucCeaal

and Satisfactihn:_ ,

. Job Search

Motivation

. Job Planning

Competency

. Short Range Job

Success

. Working vs. Not

Working

. Salary Level

. Employer Rating

Items and/or

Scales
- 47

(Example of Cross Classification for IPP Item 1)

. Male Finale

7 \

Participant 25

Minority (502) (50%)

Group

Member Nonparticipant 20 30

(40%) (60t)

S. Participant 20 15

Nonminority (40%) (60%)

Group

Member Nonparticipant 10

(20%) (80%)

6f)

40

Chi-square =
1. Prime Sponsor

2. School or Educational Institution

3. Non-profit Community Based Organization (CB0)

4. profit.Making Training, Organizations

5. Other

B. 'Service Patterns

1. Primarily OJT or Work Experience

2. Primarily Classroom Work Skill Development

3, Primarily Classroon Training iri ZMployability

Developments e.g., GED Attainment, lob Search

Training, Life Skill, etc.

4. Service Mix

5. Other

C. Linkages

1. Significant

2. Moderate

3. Limited

D. Intensity of Program (Total Service Hours)

1. Low

2. Medium

3. High

E. Unit Costs

1. Low

2. Medium

3. High , Estimates of the reliability of the,measurements afforded by

psychometric instruments are necessary for the appropriate

interpretations of the "gain" scores. Reliability as defined

here is an estimate of the relative stability of the individuals'

scores on two forms of the same test. This estimate is usually

obtained by scoring separately two halves of the same test on

a given sample -of individuals, The- -two scores for each

individual based on the separate halves provide two "independent"

measures of his or her performance on the trait or attitude.

The correlation between the two scores yields an index of the

stability of the "halved" scores for this particular sample

of individuals. This correlation between "halved" scores can

be interpreted as an index of stability of the half scores

but not the stability or reliability of scores based on the

full length test. However, if we can assume that the

items that go into-each half are measuring the same trait and

one half-test is not more difficult than the other, then one

can estimate what the reliability of the full length test is

by using the Spearman-Brown Prophecy formula (Cronbach, 1970).

Ordinarily, if the half -scores are obtairod by simply scoring

theodd and even items, the two scores should be sufficiently

equivalent with respect to content and difficulty to meet the

assumption of parallel measures of the same traits, and thus

the application of the Spearman-Brown formula to estimate

the reliability of the full test is justified. This type of

reliability is often referred to as a measure of internal

consistency reliability. Simile- to the correlation coefficient is . .. .
- 49 -

----yerler-between-zero-and-1-.-0,---wheie an observed reliability of

I.OwOuld-indicate thWen.individualel score is completely

.stablettoM one parallel test foim to another. Conversely,

if the obtained reliability is zero, knowledge of an individual's

Score oniforavA of'the-testvould;tell nothing about how he

or ,she would, perform if given forM B.

Theextent of-variation inAn_individuall_s_spores,when

`goingifroe, one test fotm to arother is a measure of the error

variance in the obsetited measurements. ,Thus, the reliability

coeffiCient can be interpreiedes a. correlation coefficient,

squired or said diffetently tend somewhat loosely) the percentage Of the.abietved-V-afietitin-betWeen individual, in the

sample that is true variation, When the reliability is equal

to,one 1.0), then all the observed variation among the

individuals' scores are the iesult of "trues between

indiViduil variation_on-stbetreit and thus there is no error

of measurement piesent in the Observed scores.

KnDU_edge 'of the amount of true variance present in test

scores (1;iii.s, the degree of reliability) is particularly

relevent for interpreting the impact of aniintervention on gain

scores. Gain scores are variations of simple difference

scores (depending on how you compute them) and thus have

components of measurement error from both the pretest and

posttest. Thus, the difference score is less reliable than

either the pre-test"or the post-test scores.

Since the analysis.of covariance 001CCAU0 which will be

described in detail later, uses an adjusted difference score

as a-aePendent variable it would be useful to know if the

acceptance of the null hypothesis of no group differences is

simply due to the unreliability of the measurement rather

than the fact that the program did not work.

Similarly adjusted difference scores can also be used as explanatory

variables in various structural equation models (path models).

Knowledge of the test reliabilities will allow one to adjust

parameters in such models and thus get an unbiased estimate

of test score gain as an influence on later labor market

performance.

In view of the abc1e discussion, internal consistency reliability

estimates will be obtained by site for treatment and control

groups on all psychometric battery instruments pre and post.

The reliabilities will be based on odd-even halves which have

bee. Jjusted with the Spearman-Brown Prophecy formula for full

test length.

*IV
- 50-

The basic analysis of gain scores is concerned with'program

and control group comparisons vith respect to test score gains.

In some programs, there will be no control but there will be

more than one treatment. In these cases, the treatments with

programs will be compared among:themselves, assumihg reasonably

similar inputs and the usual statistical adjustments to correct

for pre-existing differences. In those cases where there is

- one site within a program having no control group, a within

group gain score analysis will be carried out. In addition,

possible cross program comparisons will be made with other

programs having similar inputs.

The, analytical method to be used in th'e basic analysis of gain

scores will include; (a') changes in means adjusted for initial

status using the three different analytic models of; the

analysis of variance of difference scores, the analysis of

covariance,,and standardized gain score analysis, and (b)

correlates of changes in individual rank order from pre to

post -test, (i.e., what 'are the correlates of gain within a

program site?) Of particular interest here are the

correlations of IPP responses and gain within a program. That

is, what are, the experiences and backgrounds of the people who

show the greater' gains?

The analysis of covariance (ANCOVA) is often rsed in quasiexperimental design situations to control for pre-existing

differences between comparison groups. In a non-longitudinal

design appropriate r, iomizatioh procedures guarantees (in

the long run) that the comparison groups will not differ in

any systematic way before'the intervention occurs. However,

in a longitudinal study where the individuals are either not

randomly assigned or are randomly assigned but "dropout" in

a non-random manner then some type of design such as the

ANCOVA must be used to correct for pre-existing differences

between the groups who have both pre and post -test scores.

The following figure shows a schematic of the simplest ANCOVA model which

could be used to estimate program impact on post-test scores

while controlling for group differences on pre-test scores.

CN

,J
Poet-test

Acores

X

i

s.Group

Membership

Treatment

of Control

E2

b

1

Unexplained

Variation in

Post-test,

Score

Post-test

Scores

E3

Figure. S

///

A Simple Analysis of Covariance Model for Estimating Treatment Effectsi

The one -way arrows indicate the causal scheme assumed in the ANCOVA model.

r The double arrows simply indicate a correlation and thus no causal direction.

Poitiest performance (x3) is assumed to be a function of (or due to);

(1)-initial performance AS measured by the prat/ est-(xl) and (2) group membership

(x2). The r12 on the double headed arrow indicates the observed correlation

between treatment group membtmthip and pretest -adores is likely to be nonzero.

If random assignment had taken place, the expectation would be that r12 would

indeed be zero. However, because of either thafnebility to randomly assign,or

because of selective dropping out of participants, the subset of the total

sample that has both pre and posttests will also be likely to have a non-zero

correlation between group membership and pretest scores. For mpla, if the

poorer performers at pretest time were more likely to drop out of the program

then we would observe in the reduced sub-sample having both pretest and

1The above pictorial model can, of course, be definedas a spee.al case of the

general linear regression model x3 b31 X1 + b32 X2 + e. In fact, the actual

computations will be carried out within the pneral linear model framework.

ri
-52-

post-test scores and a positive non-zero correlation between group

memberships and pre-test scores. The interpretation of this

positive correlation ieDthat you are more likely to have a

higher pre-test score if you are in the treatment group. The

ANCOVA procedure attempts to control for such differences and

thus make the two groups "equivalent" at pre-test time.

In the simple ANCOVII. model depicted the,relative

size of the partial regression weights, b32 and. b31 When

standardized) indicates how important treatment group membership and initial status on the pre-test score is in determining

the post-test score. The size of the ,nnstandardized partial

regression weight, b32, in relation to its standard error leads

to a judgment about the presence or absence of a statistically

significant treatment effect for group differences on pre-test

scores. The sign of b32'indicates the direction of the effect.

In 'this particular example, a finding of a 'significant sitive -

Js32 would be interpreted as evidence for the presence of a

treatment related gain. on he post-test. A finding of a negative

032 suggests that the treatment has a debilitating effect on

post-test gain when controlling for initial status._ Besides

providing an estimate of the significance of the treatment effect

the regression weight b32 (assuming the above coding of treatment

group membership, i.e., "1" for treatment group, ".0" for control.

and the proper regression solution) yields an estimate-of the

aVb=age number of points gained by being a member of the

treatment group.

So much for the partial regression weight b32, what about the

interpretation of the other partial regression weight, b31 in

figure 3? The partial regression weight b31 is usually referred

to as the pooled within regression weight. Inspection of the

absolute size of b31 can tell one who is doing the gaining.

A finding of b31 less than 1.0 indicatesthat,individuali with

low pre-test scores gain the most. If b31, equals 1.0 hen

gains are "across the bdard". When b31 is greater than 1.0,

those with the high pre-test scores tend to gain the most.

When'b31 is less than 1.0 the questicin arises, are the low

scoring individuals making the greater gains because the high

scorers are "bumping" against a ceiling effect and/or .il it

the result of the unreliability of the pre-test measurement.

Assuming that the pre-test group means are at least one standard

deviation below a "perfect" score then it may be helpful to

.correct b31 for attenuation due to error of measurement in the

pre-test. If the pretest mean is too close to the highest

possible score (too close being arbitrarily defined "Is less than

one standard deviation away) than the observation of b31 being

less than 1.0 is more likely due to'a ceiling effect rather than

due to errors of measurement in the pre-test scores.

6
rV

. " 53

..-i . . . . . These corrections, for attenuation (Unreliability of the 'pretestrcin:IMake a differenCe:in the interpretation of the

import ande of the treatment iffect,'A somewhat betteeunaer..

Astanding;?1the effect of correcting b31, for unreliability on,

subsecius*-treatraent'vrqupcOmparisons can be seen from

expretsing'the-ANCOMmodel shown-in its more traditional

, equiationaL form,- , t

13.32 .1 a ) -b L

) (1)

. t2 c2 3 tl . 01

. Equation (1) shows that:the estimate of the treatment effect

Lthe patiial regression weight,- b32) can 'be expressed as the

differehce between.the two post-test means minus the partial

regressidn weight,.bil, times the difference between the two

1:0;4i.teitlmeans,' If 'we correct b31.forsunreliability. by'

dividing it by'thi.eatimatid reliability:ofthe pre-test . which .will certainly'be number less than 1.0f-then-loll the

Corrected coefficient will -be larger than b31. Thui 12 the

-pre,teit-and Ipoit-testinean for the treatment group,

are .Larger than. the respective control group pre and post-test

means then the,treatment,effect 622 will usuillk-be-emaller if

b31 is used 'in place of b31, ,If the treatment group', las a

lower ,pre-test but a higher pout-test lmean;,than the control

group then, the estimated treatment effect b32 will be larger if

r, rather thai b31 is used'. 'ObviouslY, erelare many other

mafe complicated patterns of relationships which could ,occurand

about which no simple Kedictions can be made about the direction

of the biaslif the uncorrected pooled regression weight is .used.

Both the corrected and uncorrected ANCOVA model will be used in

the standard analysis., However, it4s necessary to use a more

completely specified ANCOVA model. than the simple model used in. ; the above example. That is, A more', satisfactory estimate of the

treatment effect would be obtained if additional control or

_matching variables are used in addition to the pr4est scores,

Variables which-are likely to be used in the ANCOVA models to

measure pre to post gains for the psychometric battery measures

are (1) pretest scores, (2) ethnic group membership, (at) sex,

(4) economic status, (5) previous employment experience,

(6) reading performance, (7) educational level and (8) age.

AS1 of the above control variables with the exception o+'

(1) and (6) are obtained from the IPP. The following figures show

the type of output format that will be generated by the ANCOVA

modt1 ilong with twu variations. The two variations include

the t...41ysis of varianc. of difference scores ,(ANOVA) and

itande.rdized gain sc;:re analysis. Both of these additional,

methods can be shry4n to be special cases of equation Cl).

That is, they differ in that in genel:al they will estimate a

N different value for bil and thus b32 (the estiMaied treatment

effect) will also differ.

6O
54

PARTICIPANT GROUP

G Posttest

Mean X12

Pretest Posttest AdjustedP Reliability

Mean X1 o'i Mean 212 o'21) for

Measure. Pre Post (Unadjustgd) P

tined usted Covariites

VOC.AT

JOB KNOWL

SELF EST

WRAI

JOB HOLD

a JOBS SEEK

. SEX STER

STEP*

where:

4

X2

P

' R2

P

b - i) b al - 1), xl P zi P

* Only obtain pretest mean, sigma and reliability.

1

69

/
Measure

vOC AT

JOB KNoWL

SELF VEST

WRAI

JOB HOLD

JOB SEEK

SEX STER

STEP*

where:

Reliability

Pre Post

- 55 -

CONTROL GROUP

Pretest /

Mean jj /

(Unadlustid)

/

///

//

t1c

Posttest

Mean Xl2c

Posttest Adjusted

Mean X2c a2 fprUnadjusted Covariates

511C R2C 11,4 (Ile R) alc -

* Cily obtain pretest mean, sigma and reliability.
Measure

Covariance

Adjusted

Effect

alp -

a

a VOC AT

JOB KNOWL

SELF EST

WRAI

JOB HOLD

JOB SEEK

SEX STER

STEP*

Effect Size and Statistical Tests

(ANOVA)

Raw Gain

_Effect

(X2p -14p

) (i4c a

b

where: a = a +_11 + a + a

b xi x2 xi X2

c

r

txi rxit . zi,

4

Z2, Z3, 4.

Fan Spread Model

Standardized Significance

Change Effects ANCOVA ANOVA

rTX2 A2

a

a = aX2

P ax

2 2

i.e., the partial correlation between a "dummy"

coded treatment variable indicating treatment-or

control group membership And pretest score,

holding constant the demographic control variables.

The pretest correlation should be computed and

compared with the posttest partial correlation.

* Only obtain pretest mean, sigma and reliability.

4 7
- 57-

The analysis of variance of difference scores (ANOVA) simply

compares the mean §ifference between pre and post scores for

the treatment group with the paralled quantity for the

controls. Algebraically, this procedure is identical to a

repeated, measures design where the two testings provide the

repeated measures. Furthermore, this procedure is equivalent

to using equation (1) with b31 = 1.0. Since this approach

fixes b31 at 1.0 it implicitly assumes in its adjustment for

pre-existing group differences that the gains are across the

board. If there are "ceiling" effects and/or if the empirically

derived ANCOVA estimate of b31 is significantly less than 1.0

when corrected for attenuation, one would have to question'

the use of the ANOVA prccedure. If there are ceiling effects

one would have to question the use of either ANCOVA or ANOVA.

If the empirically derived ANCOVA estimate of b31 is considerably

greater than 1.0 (i.e. high scoring individuals are gaining

the most) one might also question the rationale for fixing

itat 1.0in view of the empirical results, The only reason

.!-r carrying out the ANOVA computations would be if the sample

_Lae of the treatment and/or control group was sufficiently

small to question the stability of the ANCOVA estimate of the

pooled within group regression. weight b31. In general, the

ANCOVA estimate of b31 is usually close to one so these two

methods often converge on the same results.

The third estimate of treatment effect is also a special case

of equation (1) where b31 is equal to the ratio of the pooled

within group pre and posttest standard deviations. That is,

b31 equals d2/d1 where d2 is the pooled posttest standard

deviation and di is the pooled pretest standard deviation.

This adjustment is based on the assumption that any group

differences found at pretest time are the results of differentiaid

growth rates. In practice the ANCOVA model adjustment index

b31, is usually slightly less than 1.0 and thus implicitly

assumes (and the validity of the model depends upon it) that

observed differences at pretest time are relatively invariant

over time in the absence of a formal treatment intervention.

That is, the group growth rates are equal and stationary over

the time interval of interest. Campbell (1969) suggests that

if two groups start out at time 1 (present time), those with

the higher mean gain at a greater rate than those with the

lower mean in the absence of a treatment effect. Campbell calls

this tee interaction of selectiorand maturation. The different

groups are members of different populations living in different

environments. The different environments interact with differences

in ability and create and maintain different levels of performance

and different rates of growth.

7
-58..

A special case of this differential growth phenomenon is the

so-called ."fan spread" model which postulates that increasing

variability within group (as measured by within group standard

deviations) accompanies increasing mean differences aver time

(Kenny,q975). The finding of increases in the group standard

deviations from pretest to posttest suggest that the so-called

"fanspread effect" may be taking place and thus a reasonable

estimate of b31 may well be the ratio of the pooled postest

standard deviation to the pooled pretest standard deviation.

Then, of course, b31 would be greater than 1.0.

An eclectic approach will be employed to estimete the

treatment effect on test score gaixP and "flag" those outcomes when the above adjustment models lead to different

conclusiorl. Further, investigations of the estimated group

parameters; pre and post-test standard deviations, the size

of b

3 ' pre and post reliabilities, and possible ceiling

effects will hopefully give some clues as to which model

would appear to be more appropriate for each set of youth

program data.

The reader will note that in the preceding figures computational

formulas are not the same as presented in equation (1). The

computational formulas are however, mathematically

equivalent to the simpler expressions already given for the

various ways of adjusting for pre-existing differences.

For ease of comparison across measures and programs the ANCOVA

results are presented as differences between adjusted means

in terms of standard deviation units. The ANOVA (analysis,of

variance of difference scores) will also be presented in terms

of differences between adjusted means in-standard deviation

units. The standardized gain score analysis results are

presented as two partial correlations. The first one, rtxl, is

the partial correlation between the "dummy" coded treatment

variable ("1" for treatment, "0" for control) and the pre-test

score. The second correlation rtr2 is the parallel computation

for the post-test score. If the pre-test partial correlation

is smaller_than the post' -test partial correlation then the

treatment was beneficial. The use of the partial correlation

rather than the simple correlation is necessary in order to

partial out the effects of the various self-selection causes

which at lec partially describe ways in which the treatment

and control groups may dgfer..For purposes of comparability

the same control variables which were'used in the ANCOVA model

will also be used here.

It is anticipated that the use of gain scores (however they

may be adjusted for pre-existing differences) measured in

terms of standard deviation units will make comparisons of

cost effectiveness ratios from one test to another or from

one site to another more interpretable. The common approach

to computing cost effectiveness ratios uses the raw score

points which may not be comparable from program to program because

of differences in heterogeneity of the participants on the

Si)
- 59 gr,

traits being measured. A gain of 5 raw score points might be

a half standard deviation gain for a homogenous input sample

at one program site but only a, third of a standard deviation

at a program site with a heterogenous population.

The above three models (4 models if one includes ANCOVA

corrected for attenuation as a separate model) have been

presented in their simplest form for the purposes of exposition.

The use of additional control variables in the models make

the computations more complex but the above substantive conclusions about the model similarities and differences remain.

An additional point about one-of the critical assumptions of

ANCOVA will be made here. The ANCOVA model as used in this

context assumes thrt the relationsh-',p between the post-test

score and the pre -test as well as any of the control variables

is the same within both treatment and control groups. In

somewhat more accurate terminology, the regression of this

post-teston the,covariates (pre-test and other control variables)

is assumed to be the same within the treatment ai.d control groups.

This assumption must hold or the partial regression weight

associated with the pre-test (b31) or any of the other

convariates will be biased and an incorrect adjustment for any

pre-existing group difference will be made. Naturally, if

an incorrect adjustment is made for pre-existing group differences,

the estimate of the treatment effect will also be in error.

For treatment control group comparisons the equality Of the

regression of the post-test on the pre-test will be tested.

If a particular comparison shows both statistical and practical

dofferences between the regressions then we suggest the- approach

due to Belson (1956). The Belson model uses an estimate of

b31, based on the control group data alone. Then predicted

post-test scores are obtained for the treatment group using

the control group regression parameters. These predicted

,treatment post-test scores are then compared with the actual

treatment group post-test scores. If the actual posti-test,

mean for the treatment group is higher than is predicted using

a prediction equation based on control group data alone, then

it is assumed that the treatment had a positive effect. This

would appear to' be the simplest way out of the dilemma,when

nonparallel regressions are found when carrying out the group

comparisons.

The question of who gains is answered most easily within the

multiple regression model. Using data on the treatment group

only, step-wise multiple regressions will be, carried out where

the pre-test scores are forced to enter first and the remaining

dem -:-aphic control variables (age, sex, ethnic group, economic

stattl, reading scores,' previous employment experience and

educational level) will be "stepped in" according to how well

they' contribute to the prediction of the post-test scores.

This will be carried out for each of the SAS instruments.

Those demographic variables that significantly add to the

prediction of post -test scores after controlling for the pre.,

test score will define a profile of the type of individuals

who "gain" the most from the treatment. The similarities

r-y

44.
b

60

and differences among the types of individuals who show the

greater gains can then-be contrastbd across programs. It

well may be that the same. type of individuals show the greater

gain regardless of the type of intervention.

Analysis using the scale scores and other item information

from the program completion survey (PCS), the`control group

status survey (CGSS), and the three and eight month follow-up

surveys will also be carried out',..tiore specifically,

treatment and controls will be compared with respect

to the following scales work motivation, social-community

adjustment, training program adjustment, and positive vocational

orientation. The analysis will be carried out within the

standard ANCOVA format using the-above list of demographisc

as control variables (covariates). -A parallel ANCOVA analysis

will be carried out for the scales in the three and eight

month follow-up.

An analysis of the relationship of the gain scores on the tests

to the above 'PCS and follow-ip scales should also be carried

out within the treatment group. This can be best accomplished

by regressing each of the PCS and 'follow-up scales., one at a

time, on a pre-test score and then adding the 'post -test

information. The increment in R2 (the squared multiple

correlation) due to the post-test indicates the effect of test

score gain on the particular PCS se.ftle or follow-up scale.

Similar analysis can be done within the treatment grOup by

utilizing as dependent variablesthe scales derived from both

the counselor rating form and the scale from the work supervisors

rating form. In addition to examining the relationship of:gain

in test.scores.to the counselor and supervisor ,ratingsf-the

4eM9graphlc control :variables noted above also Will be

correlated with' .the 'rati,ngs.

The standard analysis will also include the prediction of

"positive vs. negative termination (the dependent variable)

from the prerctst and the demographic control variables. This

type of analysis should help identify the characteristic (at

entry) of participants who are not likely to achieve positive

terminations. In addition, this analysis will furnish evidence

for the predictive validity of the psychometric battery.

The above analysis will generate a wealth of quantitative

information for each program site, yet unless some means of

summarization is carried out, the significance of the numerical

output will be lost to. the reader. It is suggested that

certain "signs of merit" based on summaries of the data be

constructed at the program level. Although these signs of

merit will be constructed from the output numbers obtained

at each program, they need not be identified with the program

name. It would seem reasonable to group'the programs according

to the classifications outlined earlier and contrast the signs

of merit across these program classifications.
61

One sign of merit would be simply the percentage of times the

treatment group at a program site denionstrated a greater

adjusted gain using the ANCOVA adjustmerit than the 'control

group for the seven measure SAS battery. A program site whose

treatment group showed a,greater gain than the bontrolgroup

on all seven measures would be assigned a sign of 7..drit of

100%. A site where the treatment group did better than the

control group on four of the seven measures would be assigned

a sign of merit of 5/% etc. The use of percentages allows for

comparisons across programs having different numbers of sites.

A second more rigorous sign of merit would be the percentage

of times a program site achieves a gain of a quarter of a

standard deviation or more. This more rigorous criteria may

yield an index with too little between program variation to

be useful or it may be so highly correlated with the first

index that it can be dropped. It is further suggested that

boza these signs of merit be computed for; (1) the psychometric

battery (2) the PCS scale profiles, and (3) the scales on the three

and eight month followup. The two signs of merit can be compared

for the program categories defined by, (1) type of sponsoring

organization (2) service patterns 0) delivery environment,-

and (4) program intensity, This summary will provide some

preliminary results in an easily interpreted format.

amesesmonstemsEssiiii
0

-62-

References

Campbell, D.T. Reforms as Experiments.' American Psychologist, 1969,

24 409-429.

Cohen, J. Statistical power analysis for the behavorial sciences,

New York: Academic Press, 1969.

Crites, J.O. The Maturit of Vocational Attittdes 'In Adolescence.

American Personne an Gu ance Assoc at on. Was ngton,

1971.

Crites, J. 0. Attitude Scale: The Career Maturity Inventory.

CTB/McGraw-Hill, Monterey, California, 1978.

Cronbach, L.J. Essentials of Psychological Testing (3rd

edition). New York, Harper and Row, 1970.

Duncan, O.D. Introduction to structural equation models.

New York, Academic Press,,1975.

Educational Testing Service. Program For Assessing Youth

Employment Skills (PAYES'). Cambridge Book Company,

New York, 1979.

Educational Testing Service, Sequential Tests of Educational

Progress.

Freeberg, N.E. Assessment of disadvantaged adolescents: A

'different approach to research and evaluation measures.

Journal of Educational Psychology, 1970, 61, 229-240.

Freeberg, N.E. Criterion measures for youth-work training

programs:

The development of relevant performance dimensions.

Journal of Applied Psychology, 1976, 61, No. 51.537,-545.

Freeberg, N.E. Development of assessment measures for use with

youth-work training program enrollees, Phase II:

Longitudinal validation.

Project Report 74-1. Princeton, N.J.: Educational Testing

Service, 1974.

Freeberg,. N.E., and Reilly, R. Development of guidance measures

for youth-work training program enrollees. Phase I:

Measuresent of program objectives and the development of

criteria. Pro Report 71-13. Princeton, N.J.:,

Educational Testing - Service, 1971.

Freeberg, N.E., and Shimberg: B. Validation of assessment

measures for use with disadvantaged enrollees in Worktraining program. Project Report PR-76-20. Princeton, N.J.:

Educational Testing Service, May,-1976.

77
Freeberg, N.E., anditella, P.E. Technical PAYES

Program, Cambridge Bobk Company, New York, I379.

Garrett, C.S., Ein, P. L. and Trethaine, L. The Development

of-Gender Stereotyping of Adult Occupations in Elementary

School Children.

Child Development, 1977, 48, 507,t.512.

Gribbons, W.D. and Lohnes, P. Emerging Careers. New Yorke

Teacher's College(Press, 1968.

Karp, J.M., and Sigel, I. Psychoeducational appraisal of

disadvantaged children. Review of Educational Researchp

1965, 35,-401-412.

Kenny, D. A. A quasi-experimental approach to assessing

treatment in the non-equivalent control group design.

Phychological Bulletin, 1975, 82, 345-362.

Lennon, R.T. Testing' and the culturally disadvantag2L2102A.

New York: Harcourt, Brace & World, 196T7-

Potthoff, R. Statistical aspects of the problem of biases in

psychological tests. (Mimeo Series No. 479) Chapel

Hill: University of North Carolina, Department of

Statistics, May 1966c

Walther, R. The Measurement of Work-Relevant Attitudes. Final

Report, U. S. Department of Labor, June,
-64.-

Appendix 1

CoMponents of the

Stand4rdized Assessment

System

Some of the instruments in the SAS are

has been grAn4-td fob, their use only in

of the SAS are presented.to provide an

system. At this time, they may not be

approval. Requests should be directed

Programs, U.S. Department of Labor.

7D

copyrighted. Permission

the SAS. The components

understanding of the

utilized without

to the Office of Youth
'Individual Participant Pro file IL 1116 1111111111

4. tarmom:

Q I11 11111 I I

loam -mom smut

1 1 1 1 1 1 I. 1

f

1111111111111111 1111111111

IC NO VIER

111111111111.1111111111.-1-11

cm

SEAM VP

\ in the sections below.- circle -the writer or members represeMN the apcio- plate response(s) to ern Ant an.:.c when bares are provided pit one

,number or charocftr per bast

7. alt: -Mole 2 -Female

MUCATION STAN&

1 -145, enter gradec

2 -ItS, Or enter graft En

3 -GED

4 -KS.

5 -Post

ECONOMIC

1 -0M6/70%

2 -71.46%LLSL

3 -56400%

4 -Above 100E. LIM

Schoc:4 Attendee

.-111111C STATUS: 1 AFDC

2 Put ec Assistance-SD

3 Other dash (Public)

11. ECONOMICALLY 014DVANTANGED:

1 -Yee 2 -No

II RACE/ETHNIC GROL4

1 -White (rot Hispani4

2 -Block (not Hisperic)1

3 -Hispanic

4 -American hion/Alikan Native

5 -Asian/Pa:Mc islander

& 'DATE

- OF MIN:

PARTICIPANTIcHARACTERiSTICS

13. UMITED NOUN SPIAI3NG /MUM

1 -Yes 2 -No

14. FAMILY STATUS: 1 -ForMy Head

2 -Fcrnik Member

3 -Hai -dspendeht-IndMdual 15. MARITAL

123 -Divorced

TOTPCMIMISER OF HOUSEHOLD mEMIEris

4 -Separated

5 -Woowed

l& CHILDREN:

1 -Yee (If yes. how roonit

2 -No

17. VETERAN GROUP: 1 -Veteran

2 -Mohan Era

'3 -Special , 4 -Special Cbabhod

5 -Not a Veteran

111. MIGRANT /SEASONAL FARM r FAMILY MEMEL

1 -Yes 2 -No

19. HANDICAPPED: 1 -Yes 2 -No

30. ENTRY

DATE: MEIN DAV YEAR 31. PROGRAM SERVICE GROUP: -Participant Group 1

2 -Participant Group 2

3 -Par ticloont Group 3

4 -Contra Group I

5 -Contra Group 2

1.4, RECEIVED ACADEMIC CREDIT: 9 1 -Ws 2 -No

RECEIVED HIGH SCHOOL DIPLOMA: 1J 1 -Yes 2 -No

31. RECEIVED GED CERTIFICATE: 1 -Yes 2 -No

35. RECEIVED OCCUPATIONAL CERTIFICATE: 1 -Yes 2 -No OTHER PROGRAMPROGINSS: 36.1111111111i

37. 111J

PROGRAM STATUS

TERMINATION: 39. Termination Dote:

TELIA0

20. OFFENDER:

1 are 2 -Edo

21. LASOF: FORCE STATUS: 1 -In School

2 -OrKlererrployod

3 LUnern4oyed

4 -Other 22. UNEMPLOYMENT INSURANCE CLAIMANT:

1 -Yee 2 -No

23. PREVIOUS CETA PARTICIPANT: 1 Mx 2 -No

24.-STEP MADING SCORE: LAST JO& a. Last worked:

111711.11

IIIlII MONTH YEAR 24. Hours Worked Per Week:

27. Job Me:

26. Hourly Wogs:

29. SulaNdlzed Wage:

1-Yes 2 -No

. 0

I l l .1 1 oo....

MU .NU III

AgNrlt

40. Total Program Joust

41. Total

Program

Waget

DAY YEAR 1111

MEM -MI mums

42. LABOR FORCE STATUS: 1 -Employed

2 -Underemployed

3' -Unemployed

4 -ID school

-5 -Other

6 -Status unknown

CENTS

POSITIVE TERMINATION: 43. Entered Unsubsidtsed Employment: 1 -Yes 2 -No

Entry

Date:' tit MONTH DAY YEAR 45. Job Shit 1E1

46. Mouth. Woge: I I El' I I,

T 47. auni Worked Pm West / . /

OW

46. OTHER POSITIVE TERMINATION: 1 -Entered l'ul.ithe academic or

vocation! school

2 -Entered o non -CETA forded

Manpower Procyon)

3 -Interne transfer

4 -Compete -rolicrn objective

not Ivor.. na uriozodoe0

8a1Ployment

5 -Entered Armed Forces NON-POSITIVE TERMINATION: 49. Reason for temslnaton:

I -Laid off

2 -hearth

3 - Pregnancy

4 -Earth core

5 -Tra op:dation problems

6 -Moved from area

7 -Refused to contrue

8 -ActrnristrotNe rePatat)90

9 -Cannot locate

0 -Otto: non-positive

78711028800P25X297302PrintEl in U SA. o.
Definitions for the 'ndividual PorticipCint Nofile

ANTICIPANT'S PERSONAL DATA: Personal d6ta recorded at

ins time the'youth makes application to the program.

1. En I.:. NO.:Assigned by ETS. First two digits designcite

the youth project. Next two digits designate the project

site or location. Next six digits identify an individual

1 youth.

2. NAME: Legal name.

3. ADDRESS: Location at which the youth resides.

4. TELEPHONE: Telephone number at which the youth can

be reached when not at a program site.

5. SOCIAL SECURITY NO.: Number on youth's "Social Secu-

, MY, account no. card" or other document (W-2 Form).

6. DATE OF BIRTH: Exact month, day, and year of birth.

7. SEX Designation tif male or female.

PARTICIPANT'S CO4R/414,1ERISTICS: Personal Information recorded at the time the youth makes application to the 1program,

8. E D i CATION STATUS: Circle only One category which best

scribes the youth's education status.

1 H.S. Student; enter wad.: Youth is enrolled in a secondary school (including Junior and senior high

school or equivalent) or is between school terms and

intends to return to school, or Is enrolled in &lemontory school: enter highest school grade completed.

2 H.S. DropoLit; enter grade: Youth is not attending

any school 61rnd has not received a high school diploma or GED: enter highest school grade--

completed. \

3 GED: Youth has received a GED certificate but has

not attended any post-secondary vocational, technical, or academic school.

4 H.S. Graduate: Youth has received a high school diploma but has not attended any post-secondary

vocational, technical, or academic school.

5 Posit-HIgh School Attendee: Youth has received a

H.S. diploma or GED certificate and is attending or

has attended a post-secondary vocational, technical, or academic school.

9. ECONOMIC STATUS: Circle only one category which best

describes the youth's family income.

1 0M6/70% LLSIL: Youth is a member of a family which

has an annual family Income which, in relation to

family size and location, does not exceed (a) the

most recently established poverty levels determined

In accordance with criteria established by the Office of Management and Budget or (b) 70% of the

lower living standard Income level: whichever Is

greater.

2 71-85% LLSIL: Youth is a member of a family which

has an annual family income which, in relation to

family size and location, is 71-85% of the lower living

standard Income level.

3 86400% LLSI4Youth Is a member of a family which

has an ann&il family income which, in relation to

family size and location, is 86-100% of the lower living stan Ord income level.

4 Abo 400% LLSIL: Youth is a member of a family

which has an annual family income which. In rata--

ton to family size and location, is above 100% of the

lower living standard Income level.

10. PUMA: ASSISTANCE STATUS: Circle all applicable categories which describe public ossistonce status of the

youth or the youth's family.

1 AFDC: The youth or the youth's family is receiving income CT-money-pents pursuant to a State plan

approved under the Social Security Act, Title IV (Aid

to Families with Dependent Children, or if the applicant is a foster child [family of one] on behalf of

whom governmental payments are made).

2 Public AssisIance-SSI: The youth or the youth's family

is receiving supplemental Incohle or money payments pursuant too State plan approved underthe

Social Security Act, Title XVI (Supplemental Security

for the Aged, Blind, or Disabled), or pursuant to the

Indochina Migration and Assistance Act of 1975,

Public Law 94-23.)

3 Other Cash Public Assistance: The youth or youth's

family is receiving State or local cash assistance

based on need.

11 ECONOMICALLY DISADVANTAGED: / A youth who can be

described by any of the following categories: (a) a

member of a family which receives public assistance;

(b) a member of a family whose income during the

previous six months, on an annualized basis, was such

1

that the family Would have qualified for public assist-

) if it had applied for such assistace and/or it does

not exceed the OMB poverty level of 70% of the lower

living standard income level: (c) a foster child on whose

behalf State or local government payments are made;

(d) a client of a sheltered workshop: (é) a handicappec4

Individual with a family income of 100% or less of the

loWpr living standard income level: Ma person confined

to dg Institution or facility providing, 24-hour support

such Os In prison or hospital: jg) a regularout-patient of

a mental hospital, rehabilitation facility .or similar

institution.

12. RACE /ETHNIC GROUP: Circle only one category it.rhich

best describes the youth's race, ethnic group. (NOTE:

The category which most closelYJef ects the individual's

recognition In his/her community 'mild be used for

youth who are of mixed racial and e lorigIns.) ' 1 White (not Hispanic): A youth having origins in any of

the 'original peoples of Europe. North Africa, or the

Middle East.

2 Black (not Hispanic): A youth having origins in any of

the black racial groups of Africa.

3 Hispanic: A youth of Mexican, Puerto Rican, Cuban,

Central or South American, or other Spanish culture

or origin (including Spain), regardless of race. , A American Indian/Alaskan Native: A youth having

origins in any of the original peoples of North America, and who maintains cultural idehtification in

tribal affiliation /community recognition.

a
a.

5 Aslan/Pacitip islander: A youth having origins in any

Of the original peoples of the For Eait7SiTufffeast

Asia the Indian .subcontinent (I.e.. India, Pakistan,

Bono Sri Lanka, Nepal, Sikkim. and Bhutan).

or .the' -Pacific s. This area Includes, for example.,China. Japan, ea. the Philippine islands,

and Samba. - 13. UMITED. ENGLISH SPEAKING NAM. th's native

language is not English and his pler ability to unitate in English is a job handicap.

14. FAMILY STATUS: Hrst entelthe number of household mem-

. ben. then circle only the one category which best describes the youth's family status.

Number of Household Members: Enter total number of

persons living.in youth's household. - 1 Family Head: Youth is a member of ofamilrof two or

more persons and considered to be the head ofine

family by members of the family.

2 Family Mernber:Youth Is alniember of o family cf two

or more Persons and Is not considered to be the

head of the farnily by members of the family.

.3 Non-Dep6ndent Inclividucii: youth is either (a) living

with' his /her family, 18 or older, receiving less than

50% maintenance from the family, and not the

head of family:or the spouse of the head of the family, or (b)14 or older cnd not living with his or her family, or-(c) a foster child on behalf of whom State or local government payments are made. Such youth

should be considered as families of one for determining Economically Disadvantaged. Underemployed and Lower Liviog-Standard Income Level

status:

15. MARITAL STATUS: Circle oration* category which best describes the youth's marital status.

16. CHILDREN: The youth has responsibility for support ofone

or more dependent children; enter exact number of dependent children.

17. VETERAN GROUP: A veteran is a person who (a) served on

active duty fora period of more than 180 days, and was

discharged, separated, or released therefrom with other than a dishonorable discharge, or (b) was discharged

Or released from active duty fora service connected disability. The term "active duty" (or -active military, naval, or air service") means full-time duty in the Armed

Forces, other than duty for training in the reserves or National Guard. Any period of duty for training in the reserves or National Guard, including author zed travel,

during which an Individual was disabled from a disease

or injury incurred or aggravated in the line of duty, is

considered "active duty". Circle all applicable categories which describe the youth's veteran status.

1 Veteran: AVouth who meets the definition of veteran

contained in the introduction to this item.

2 Vietnam Era: A veteran 34 years of age and under

who served on active duty between August 5, 1964,

A and May 7, 1975,pnd discharged or released within

4 years preceding application to the program.

3 Special: A veteran who served in Indochina' or

Korea, including the waters adjaCent thereto, between August 5, 1964, and May 71, 1975, inclusive,

and who received other than a dishonorable

discharge.

;--SpeClal Disabled: A youth who served inthe Armed

ForCes and who was discharged-Or jeleased therefroM with °Vier thana alsiyoribtable dischargeand

who has ben given a disability rating of 30 riercenturn or more, or a v.tasecilsdharge or releaSe

frOm active d s for disdbility incurred or aggra- vated in the e of duty. , . 5 Nat a an: A youth whb meets none of the

abo categories. - . 14. MIGRANT/SEASONAL FARM FAMILY MEMBER:A

Is a,rnember of atamily which had one or morepe

Who, during the preceding 12 month's, worked atleast

25 days in farm work and worked less than 150'coaseCti-. , Jive days at any one establishm8nt. Intiude botttrnigro-:,-

top), and nonmigratory form rkers, butdo not include , supervisors or nonmigratoty I vIdua)s wtib &eft:1141mi,

students or farmworkers who or not "seasonal"tbs del,* .:

fined in the, preceding sentenb r .".

I e 19. HANDICAPPED:A youth who pas Ohysical or criiiiitdi dis- ability which, at the time of e rolinienh .constitutes'a,

substantial barrier to empioyme t, and the individual is ;

unemployed -as a result of this disability.

20. OFFENDER:A yodit; vino has been subJectio any of

the criminal justice proce-s, for'whom employMent and

training services may be beneficial or who, requires Cs.;

slitance in overcoming artlficial barrieretb employment

resulting fro'm 'a record of arrest or correction. r

' 21. LABOR FORCE STATUS: Circle only one category. which

best, describes the youth's labor force status.

1 In-School: A youth who $::3) is currently enrolled full-

'time in, and attending, a secondary, trade, technical, vocational school or community college or is

scheduled to attend full-time the next regularly

scheduled quarter or semester of any of these

zchools oc(b) hcs not completed high school and is

scheduled tc) attend a program leading to a secon-

,dory school diploma or its equivalent.

2 Underemployed: A youth wfio Is (a) working parttime but is seeking full-time work, or jto) %forking fulltime but receiving wages,'not in excess of the higher

of either the OMB poverty level or 70% of the lower

living standOrd income level,

3 Unemployed: A youth who Is without a Job for at

least 7 conecutive days prior to application. The

youth shall be considered as being without a job if,

during those 7 consecutive days he/she fulfilled any

of the following four conditions: (a) worked no more

than 10 hourt and earned no more than 30.00 and

was seeking and available for work; (b) was a client

of a shAered workshop; or a prisoner eligible for, or

in, a work release program; or institutionalized Ina

hospital or similar institution; (c) was cr,person 18

years of age or older, whose family would be eligible

to receive public assistance if both parents were not

present in the home; or (d) was a veteran who has

not obtained permanent unsubsidizedeTployment since being released from active duty.

4 Other: A youth who meets ,one of the above

categories.

82
22. UNEMPLOYMENT INSURANCE CLAIMANT: Ayouth who has

filed a claim and has been determined monetarily ellgiblefor benefit Payments under one or more State or

Federal unemployment insurance programs, and who

has not exhausted benefit rights or whose benefit year

has not ended.

23.- PREVIOUS CETA PARTICIPANT: A youth who has at any

time received employment and training services and

any related wages or allowances by participating in a

program financed locally -under the Comprehensive

Employment and Training Act (CON.

24.. STEP READING SCORE: The number of correct answers on

the 20-Item STEP reading test.

LAST JOB:

4

28. last Worked: The erect month old yearwhen youth left

last job held.

Hours Worked Per Week: The number of hours usually

worked per week on the youth's last job.

26.

27. Jib Title: Write In the name of the job, which best describes the work performed by the youth.

28. Hourly Wage: The exact hourly wage earned on last job

when last worked (see Q. 25).

29. Subsidized Wage: Last job held by the youth was a imporary position which was part of another CET!

program.

PROGRAM STATUS: information and data on the youth's participation In and termination from the program.

30. ENTRY DATE: The ¶xact month, day and year a youth first

received service' rom the program. _ 31. PROGRAM SERVI ,,E GROUP: Definitions for these cotegoiles will be detOrrnined by ETS. (Do not circle any number In item 31.)

Participant Gjoup

2 Participant Group 2

3 Participant Grot.ip 3 >. 4 Control Group

5 Control Group

32 RECEIVED ACADEMIC CREDIT: The youth is awarded H.S.

1 credit for spe_clal work- or other program- activities

adopted by the program.

33. RECEIVED HIGH SCHOOL DIPLOMA: Youth received H.S.

Diploma in the course of participating in the demonstration project.

34. RECEIVED GED CERTIFICATION: Youth received GED Certification in the course of participating in the demonstration project.

35. RECEIVED OCCUPATIONAL CERTIFICATE: Youth earned

special certification for attaining special; skills meeting standards set by a State or other agency or organ'.

zatior? independent of thelprogram.

OTHER PROGRAM PROGRESS: Definitions for these categories

will be determined by ETS. (Do not make entries in items 36.

37, and

36.

37.

, 38.

TERMINATION

89. Termination Date: Exact month, day and year that a

youth was terminated from participation in the

program.

40. Total Progrum Hours: Total number of hours a youth actually attended program activities.

41. Total Program Wages: Total wages and/or allowances

received while enrolled in the program.

42. LABOR 'FORCE STATUS: Circle only one category which

best describes the youth's labor force status at the time

of his/her termination from.the program.

)1 Employed: A Vouth who is (a) worling part-time and

is not seeking full-time work, or (b) working full-time

and receiving wages in excess of the higher of either

the OMB poverty level or 70%' of the Jower living

standard income level.

2 Underemployed: A youth who is (a) working parttime but :s seeking full-time work, or (b) working fulltime but receiving wages not in excess of the higher

of eitner thaOMB poverty level or 70% of the lower

living standard income levels

3' Unemployed: A v.:XI', who has not been placed

in or obtained an in,4ubc. c,iied TOP.Cind has not

worked for more thar hoJrS per week earning no

more than $30.00 per week and is seekinp and available for work.

4 In School: A youth who (a) is currently enrollectfulttimerin, and attending, a secondary, trade, technical. vocational school or community college, or is

neduled to.attend full-time the- next regularly

sc,-hech sled quarter or semester of sy-,ly of these

schools, or (b) has not completer high school and is

scheduled to atterr.1 a program leading to a secondary school diploma or its equivalent.

. _ 5 Other: A youth who is working, but meets none of the

requirements for the above categories.

6 Status Unknown: A youtt- who has left the program

and cannot be located- (this person should also be

listed under category 9. item 49 under Non-Positive

Termination),

POSITIVE TERMINATION

43. Entered Unsubsidlied Employment: Youth has entered

employment in the private or public sector and receives

wages which are not subsidized under CETA or some

other government program.

44. Entry Date: Exact month, day and year that a youth entered unsubsidized employment.

45. Job Tifle: Write in the name of the job which best desCribes the work performed by the youth.

46. Hourly Wage: The exact hourly wage earned by the

youth on his unsubsidized job.
ti 47. Hants Worked Per Week: The number of hours usually

worked per week on the youth'sUnsubsidized job.

48. OTHER POSITIVE TERMINATION: Circle only one category

vihich beitde4cribes the positive termin4o47 of youth

Who has not entered unsubsidized empl nt.

1 Entered fulilime academic or vocational schobl

2 Entered o non-CETA funded Manpower Prograrn

3 Into* le Trairfor: Youth has entered another CETA

prOgram.

4 Completed program Objective not involving UnsubsIdliedurnployment

5 Entered ;#4nned forces

\

NON POSITIVE TERMINATION

49. Reason for Termination: drCIO only one category which

bed describes the non-podtive termination of youth.

-1 Laid off

2 Health

3 Pregnancy

4 Family care

5 Transportation problems

6 Moved from area

7 Refused to continue

8 AdmInIstiativeteparation

9 Cannot locate

0 Other non-positive
TEST ADMITSTBArR'S pIRECTJONS

[Before you give the STEP Test, you should have the participants fill out

the information in the top left section of the test book.]

0

"Please print your name on the first line. Then fill in the name

of your project or school and the name of the city and state in

whichour project or school is located. If you are in school

now, check whether you are a junior or a senior. Please fill-in

your age and check male or female. Last, write in today's date."

"Read the directions on the front cover of the,test book while I read

them aloud."

Directions

Each cl.estion in this test is followed by four suggested answers.

Read each question and decide which one of the four suggested answers is best.

Then circle the letter next to the answer youhave chosen.

"Are there any questions?"
SAICRIE$

"Read the paragraph and answer sthe two sample questions below by circling the letter next to the

answer you have chosen."

[Allow a few minutes to read the paragraph and complete the two samples.]

The Herring Gull is especially good at seizing food from other birds. It is about,

twenty-four inches long, and it is the gull that you most often see at the beach. It will often chase a bird that is carrying a fish or a stolen egg home to eat. The

Herring Gull keeps attacking the other bird until it drops the egg or the fish. Of

course the egg will break if it hits the ground. But Herring Gulls are so fast and

agile they can sometimes catch an egg in mid-air.

SAMPLE

Which of the following s the best title for How long is a Herring Gull?

this passage?

6, How Herring Gulls Get mood

B) Catching Eggs

C) How Herring Gulls Fly Faster Than

Other Birds

D) Eating Habits of Birds

SAMPLE 2

A) 12 inches

B) 18 inches

24 inches

32 inches

"In SAMPLE 1, the correct answer is A because

how Herring Gulls get food. Therefore, A is

"In SAMPLE 2, the correct answer is C because

the Herring Gull is twentyfour inches long.

"Are there any questions?"

the passage is about

circled."

the passage states that

Therefore, C is circled."

"You will have 15 minutes to answer the questions.

question. Begin."

Turn to the first

[During the administration, move quietly about the room to see that each person

is marking his answers by circling a letter in the test booklet.]

[At the end of 15 minutes, say:]

"STOP! Even if you have not finished you must stop and lay down your

pencil."

Copyright 01979 by Eclu.ational .esting Service. AU rights reserved.

8G
ETS ID No. L

Name

Last First

Project or School

City State

Grade (if in school) Ir.

.1111111. Age Male Female Date

t."""r° .

Step

Directions

Each question in this test is followed by four suggested answers.

Read each question and decide which one of the four suggested-answers is best.

Then circle the letter next to --the answer you have chosen.

SAMPLE

Read the paragraph and answer the two sample questions below by circling the letter next to the

answer you have chosen.

The Herring Gall is especially good at seizing food from other birds. It is about

- twenty-four inches long, and it is the gull that you most often see at the beach.

It will often chase a bird that is carrying a fish_or a stolen egg home to eat. The

Herring Gull keeps attacking the other bird until it drops the egg or the fish. Of

course the egg will break if it hits the ground. But Herring Gulls are so fast and

agile they can sometimes catch an egg in mil-air.

SAMPLE 1 SAMPLE 2

"Which of the following is the best title for

this passage?

How Herring Gulls Get Food

Clt? Catching Eggs

C) How Herring Gulls Fly Faster Than

Other Birds

D) Eating Habits of Birds

How long is a Herring Gull?

A) 12 inches

B) 1g inc.hes

2 inches

I1 32 inches

Copyright 01979-by Educational Testing Service. All rights reserved.
aQuestions 1-5

From loaves lying ready on the table, each

person cut a thick slice, of bread. This he placed in

front of him to serve as a plate, or "trencher."

Everyone reached into the serving dishes with

both hands and carried the food, sauce and all, as

best he could to his trencher.

"Do not dip your fingers into the sauce

deeper than the second joint," Lady Alice told

Robert when he first came to the castle. "That

would not oe good manners."

Dogs prowled about the tables, waiting

hungrily for bits of food and bones to be thrown

to them. They often fought over the scraps.

The floor of the great hall was covered with

rushes reeds cut from the river bank in spring

and spread ow lo lessen the chill of the cold

stone. At first this carpeting was soft and

fragrant. But it soon .became foul with bits or

bone and food.

4

1. What did the people in this passage use

as plates?

A)

B)

C)

D)

Silver

China

Bread

Napkins

2. The floor of the great hall was made of

A) stone

B) wood

C) dirt

D) mud

3. Rushes were put on the floor in order to

A)

B)

C)

D)

cover up the used trenchers

make t1:. carpet look pretty

make the floor feel warmer

keep the dogs away

4. The story takes place in a

A)

B)

C)

D)

restaurant

castle

school

church

5. The story says that the dogs were

A)

B)

C)

D)

big

fast

hungry

ugly

Go on to next page p
(s)

Questions 6.11

Chocolate comes-from a white bean produced

by the cacao tree. The Aztec Indians first made

chocolate by whipping ground, roasted cacao

beans in hot water and then adding vanilk,Ireen

pepper, and spices. Later the Spanish found that

\by adding sugar to the ground cacao beans they

could make a delicious new drink. This new drink

was to become very popular among the kings of

Europe. Chocolate was sometimes. drunk by sick

(10) people as a tonic. Special chocolate houses in

,Spain,, France, Italy, Holland, and England

served this new treat to fashionable customers.

Now chocolate is a flavor we enjoy in candy,

cake, ice cream, and milk shakes.

6. What is the best title for this story?

A) How the Spanish Discovered Chocolate

B) The History of Chocolate

C) Flavors of Candy and Ice Cream

D) How to Grow Cacao Trees

7. What did the Aztec Indians put in chocolate?

A) Milk

B) Sugar

C) Whipped cream

D) Green pepper

8. The kings of Europe drank chocolate with

A)

B)

C)

D)

sugar

spices

green pepper

vanilla

9. Tonic in line 10 means

A)

B)

C)

D)

medicine

soda

treat

candy

10. The first people in tip.; passage to use chocolate

were the

A)

B)

C)

D)

Spanish

French

English

Aztecs

11. Fashionable in line 12 means

A)

B)

C)

D)

sick

intelligent

stylish

young

Questions 12-14

Early people. had little reason for measuring the

__,tarda of land. They did not have permanent homes

And were nomadic. In larlr,years, people began to

stay in one place on land that they owned.

Therefore, they needed some kind of description

or measure of land.

Seeds were the earliest measures of\ this kind

and the amount of land which a given amount of

seed would plant became a unit of measure.

A common unit of land measurement today is

an acre. At first an acre represented the amount,

of land a farmer could plow with his oxen in one

day. However, this was never a very precise way

of measuring the land. An acre still means a

certain amount of land in one country and a

different amount in other countries.

12. The passage is mostly about

A)

B)

C)

D)

how land is measured

how people lived long ago

why seeds were used to measure land

why farmers used oxen to plow

13. Nomadic in line 3 means

A)

B)

C)

D)

farming without modern tools

moving from place to place

living in caves

fighting with other groups of people

14. Why was the use of oxen not a very good way to

measure land?'

.A)

B)

C)

D)

Different people could plow different

amounts of land in a day.

Some ,people planted more seeds than

other people.

People did not know how to farm

efficiently.

People did nct care how much land they

owned.

Go on to next page

SD
Pail 4

Questions 15.20

' As a young society woman Mary Cassatt might

have done the conventional things, but she was

different. She a passion for drawing and

painting; she was determined to be an artist, and a

(5) good one, regardless of whatftunily and friends

thought.

After several years of studying in Italy,

Spain, and Belgium she ended her art pilgrimage

by settling in Paris. There she was soon attracted

(10) to the work of the artist Degas. His work is known

here in America chiefly

a way she became

copies ies of his drawings

of ballet girls.

She was willing.to accept his suggestions and

criticisms as she would not accept anyone else's.

(1 5) More than any other artist of modern times,

Mary Cassatt made herself famous by painting

the single theme of mother and child.

15. The main purpose of this story is to discuss

Mary Cassatt's

A)

B)

C)

D)

career

family

friends

travels

16. Mary Cassatt is most famous for paintings

she did of

A) ballet dancers

B) scenery in Paris

C) scenery in America

U) women era their children

17. The word conventional in line 2 means

A)' humble

B) artistic

C) miraculous

D) usual

797660

Y49P21 pp. 4 atn,

Printed In U.S.A.

18. The word pilgrimage in line 8 means

A)

B)

. C)

D)

journey

friendship

purchase

masterpiece

19. A disciple (line 12) is a person who

A) nurses another back to health

B) follows the teachings of another

C) inspires someone to do great works

D) receives money from someone who has

died

20. It can be concluded that Mary Cassatt believed

that Degas

A)

B)

C)

D)

was a difficult person to understand

painted like artists she knew in America

wanted to go to America

had, good ideas about painting

STOP! If you finish before time is called, you may check your work.
YOUTH PROJECTS

ADMINISTRATOR'S MANUAL

Educational Testing Service

Princeton, New Jersey

91

,t
NOTE TO THE ADMINISTRATOR

The set of measures in this booklet are to be presented to-participants

enrolled in your program and to a control (or similarly matched) groupW

people not enrolled in your program. The booklet includes seven (7) heparate

measures. The measures are to be administered in the order in which they

appear in the bdoklet. The instrument designations and order ,gf presentation

are:

Vocational Attitude Scale (VA) - 30 items

Job Knowledge (JK) ---33 items

Job Holding.Skills (JHS) - 11 items

Work Relevant Attitudes Inventory (WRAI) - 16 items

Job Seeking Skills (JSS) - 17 items

Sex Stereotypes of Adult Occupations (SSAO) - 21 items

Self'Esteem (SE) - 15 items

These measures are to be administered in an informal manner--unhurried

and in a conversational tone. The group of participants should be no larger

than 15 to 20 persons. A short break may be taken at the end of section 4 if

.needed.-111 seven sections are to be administered at one testing session.

The directions for each section and all items are to be read aloud by

the administrator. All answers are to be marked in the test booklets, themselves. Pencils should be used to mark responses in the booklets.

Distribute the booklets and pencils.
ADMINISTRATOR'S DIRECTIONS'

4

ii

Say: We appreciate the fact that you're willing to help.us. You may be

able to picksup some infornetioiCabout yourself by trying out sUfferent

kinds'of questions and'statements you'll see in this booklet. We'd . like you to answer all the questions but you don't have to answer any

question you feel we li.Vve no right to ask. If you think that the

correct answer for a question is not given, leave it blank. If you're'

not sure of an answer, it's OK to guess.

Now, please print your name on the front cover of the booklet. Then

fill in the name of your project or school and the name of the city

and state in which your project or school is located. If you are in

high school now, please check if you are a junior or a senior. If

you are not a junior or a senior, leave it blank. Please fill in your

age and check male or female. Last, write in today's date.

0 ;
iii ; ADMINISTRATORS DIRECTIONS Vocational Attitude Scale (VA)

SAY/Okay, now turn to page 1. Look at the directions in the box at the top

of the page. .Read them silently'while I read theth out loud.

For each statement, if you agree or mostly agree, mark the box

under the letter T. If you\disagree or mostly disagree, mark.

the box under the letter F.

0 Look at number 1. I often daydream about what I want to be, but I really

haven't chosen a line of work, yet.

0

If you agree or mostly agree with that statement, mark the,box under the

letter T. If you'disagree or mostly disagres7with the statement, mark

the box under the letter F.

Look at number 2. Everyone seems to tell me something different; vs a

result I don't know which kind of 'workto choose.

Check T if you agree....F if you disagree.

2 NOTE TO THE ADMINISTRATOR /

Continue t read the rest of the statements in.the same manner. Check to

see that the respondents are following the directions. When the group has

'completed number 30, go on to the next section.
ADILitNaLATOR'S DIRECTIONS] Self Esteem.(SE)

SAY: Turn to page 35. Look at the directions in the box at the top Ofthe page., Read them silently while I read them out loud.

Thepicturesthatfollowshoweverydaykindsofsituations in which a person might be involved.

Mark the space next to the iay that tells trust how you would feel if you were the person in that

scene. ,

SE

,

NOTE1TO THE ADMINISTRATOR ( When there is a ballotin inla piCtre, read the quotations in the balloon.

L

Copyright 0 1976 by Et Ito:salmi, Testing Scrim

SAY: Number 1. If you were the person getting the diploma how would you feel?

The diploma's liot going to 'do anything to heposomeone:like me.

I might get.a-little butter job with the diploma.

I could really have it made with a diploma.

' Mark the answer that ,tells best how you would feel.

,1

Nutt TO THE ADMINISTRATOR

Continue reading the rest 0 the terns in a similar manner.

/, ... *011

-1

IA

x'
-1-

For each statement, if you agree or mostly agree, mark the box

under the letter T. If you disagree or moskly disagree, mark

the.box under theietter.F.

/ VA

T F

1. .1 often daydream about why t really haven't chosen a 1 ne of work yet.

T0 2. veryone seems to tell me something. different; as

iesult I don't know which kind of work to choose.

IF f

. 4T F

.

. 3. Its probably, justias easy to be, successful in one

occupation as it is in another.

----->'

14, T F

-----0 / 4. it doesn't matter which job you chooie as long as ---- it pays well. 1

. T F

1:::1 5. I plan to follow the line of work,;pv parents

suggest.

T F

6. You should, decide for yourself what kind of work

to do.'

T F . 7. There are so- many- things to consider in choosing

an occupation, it is hard to make a decision.

i

1 T F

, 8. The best thing to do is to try out several jobs,

and then-choose ,the one you like best. /

From the Career Maturity

1- t4y Attitude Scale, Counseling Form B-1

devised by.John 0. Crites. odi ied and reproduced 1-. ! permission of the

pul?lisher, CTB/McGraw-Hill, Del onte Research Perk, Monterey, CA 93940.

Cofyright & 1978 by McGraw-Hil ,,Inc. All Rights Reserved. Printed

in the U.S.A.

G A
-2-

T F 0 9. There is no ,point.deciding on a job when the future

is so uncertain.

T F

10. There is only one occupation for each person.

T F 0 '11. Your parents probably know better than anybody else

which occupation you should enter.

T F

12. When it comes to choosing a job, I'll make up my

own mind.

T F

E] 13. I keep changing my occupational choice.

T F

14. As fat as choosing an occupation is concerned,

something will CGMe along sooner or later.

T F

15. I'm not going to worry about choosing an occupation

until I'm out of school.

T F

[7] 16. You get int8 an occupation mostly by chance.

\ T F

\ E] 17. You can't go very far wrong by following your

parents' advice about which job to choose.

T F 0 18. Choosing an occupation is something you hays- to do

on your own.

T F

CD 19. I'd rather work than play.

T F

ED 20. I don't know whether my future occupation will

allow me to be the kind of person I want to be.

97
-3-

T F

0 21. It's who you know, not what you know, that's

important in a job: ,

T F

CD 22. You shouldn't worry abo c oosi a job since you

don't_have anything t say about i anywgN

T F

[:] 23. 'I don't want arents to tell m which occu ation

I should oose. T F

24. If.someone would tell me which occupation enter,

I would feel much better.

T F

25. I don't know whether my occupational plans are

realistic.

T F

26. I have so many interests it's hard to choose any

one occupation.

T F

[:] ED 27. You should choose a job in which you can someday

become famous.

T 'F

28. Entering one job is about the same as entering

another.

T F

29.. Parents usually can choose the most appropriate jobs

for their children.

tT F

[:] 30. 'I feel that I should do what my parents want me to

do.

C PLEASE STOP HERE.

WAIT TO BE TOLD HOW TO DO THE NEXT GROUP OF QUESTIONS.

98
,

C tbrINISTEATOR'S DIRECTIONS1

iv

Job Knowledge (JK)

SAY: Now-please turn to page 4. Look at the\directicns in the box at the

) top of the page. Read them silently while I read them out loud.

//

In these exercises you'll see pictures of differentlObs and.the name of each job. Next to each

picture are questions about the job, like how much that job pays or what kind of work you do gn

that jOb.

Please answer the questions about each job. For each question, mark the space next to the

answer that you think is the right one.

r Copyright 1978 by Educational Testing Service:

SAY: Page 4 shows a picture of a cashier.

Number 1. Where would you work on this job most of the time?

In a factory on an assembly line.

In the kitchen of a restaurant:

In a store.

In your own home.

T Mark the'answer that you think is the right one.

A NOTE TO TdE ADMINISTRATOR

COntinue reading the rest of the questions including the names of the jobs

and the responses.

The tools that are pictuied in several of the items are not to be read by

the administrator.

9 (1
74--

In these exercises you'll see pictures of different jobs and the name of each job. 'Next to each

picture are questions about the job, like how much thatob pays or what kind of work you do on

that job.

Please answer the questions about each job. For each question, mark the space next to the

answer that you think is the right one.

JK

2.

What kind of work do you think you would do on

this job?

0 Design and repair cash registers

0 Check out goods,for customers and collect money

CD Operate a copying machine and keep financial

records

0 Help people borrow money when they need it

Copyright © 1978 by Educational Testing Service.

Where would you work on this job most of the time?

C_D in a factory on the assembly line

O In the kitchen of a restaurant

0 In a store

0 In your own home

3.

When are you supposed to do the work on this job?

0 Mostly at night from midnight to 8 o'clock in

the morning

0 Might be any hoursday or night, weekdays

or weekends

C:D Only weekendsall day Friday, Saturday wad

Sunday

0 Only during the day from Monday to Friday
SCHOOL TEACHER

4.

When are you supposed to do the work on this job?

0 Monday to Friday, from about 8 o'clock in the;

morning to 4 o'clock in the afternoon

0 Mostly at night from about midnight to 8 o'clock:

in the morning or 5 days a week

C) Any kind of hours day óF night from Monday

to Friday

00nly at nights and on weekends

S. 6.

..

. About how much would you get paid when you start How much schooling do you think you'd need to get

on this job full-time? this job?

c- $6.75 to $7.50 an hour 0 Grade school only

0 $1.25 to $1.65 an hour 'Zr' A high school diploma

4C). $2.0016 2.65 an hour 0 A college degree

0 $3.75 to $4.25 an hour 0 No special schooling
0

CARPENTER

8.

8

Where would you work on this job most of the time?

CDT" your own-home

C:3 In i garage or service station

CZ) In a fadorystnan_assemblk-line

CD Outdoors, where new houses are being built

9.

Which one of theser jools would you use in this kind When are you supposed to do the work on this job?

of work? .1 C) Any hours of the day or night

C? Mostly during the day from about 8 o'clock in

the morning to 4 o'clock in the afternoon

CD Mostly at night from about midnight to 8 o'clock

in the morning

0 Only on weekendsFriday, Saturday and Sunday

CD

102
-7

'HOSPITAL ATTENDANT

V

When are you supposed to do the work on this job?

CD Only during the dayfrom about 8 o'clock

in.the_morning to 5 o'dock in the afternoon

0 Only at nightfrom-about midnight to 8 o'clock

morning

CDMight"be any days or almost any hours during

the day or night , . CD Every Friday, Saturday, and Sunday onlyeither

during the day or at night

What kind of work do you think you would do on About how much would you get paid when you

this jab? start on this job full-time?

CD Decide what medicine &sick person should C:-.) $4.25 to $4.75 an hour

take D More than $5.00 an hour

Help a sick person by feeding him

CD $2.90 to $3.50 an hour

CZ) Do blood tests tell what is wrong with a

sickperson CD $1.85 to $2.25 an hour

c Give examinations to tell if someone is sick

0

1 0
-8-

14.

About how much would you get paid when you

_stailpn this job full-time?

CD $6.00 to $7.00 an hour

CD $2.25 to $2.75 an hour

ot=:, Over $8.00 an hour

CD U.60 to $4.75 an hour

13,

What kind of work do you think you would do on

this job?'

CD Give medicihe to sick people

CD Teach people how to read and write

(DTA with people about theiproblems

G-170 Arrest peOple who commit crimes

15.

How much schooling do you think you'd need to

get this job?

No special schooling

A college degree

OA trade school certificate

CDGrade school only
-

POLICE OFFICER

16.

When are you supposed to do the work on

this job?

0 Only-during the day from about 8 o'clock

in the morning to about 5 o'clock in the.

afiernoon

Could be any days or any hours during

the day of night

0 Whatever hours you feel like working

0 Only at nights from about midnight to

8 o'clock in the morning

17. - 18.

How much schooling do you think you'd need to Where would you work on this job most of

get this job? the time?

0 Highschool\diploma 0 In a factory on an assembly line

0 Grade school only O h, your own home

0 Trade school certificate 0 Outdoors on a farm

0 No special schooling O Outdoors in a city

105
MAIL CARRIER

°

20:

What kind of work do you think you would do on

this job?

CD Keep records of all letters received in a

post office

CD SelLstamps in a post office

CD Sort letters and deliver them to people

CD Fix mail-sorting machines when

break 'down

19.

About how much would you get paid when you

start on this job full-time?'

C-.) $3.75 to $4.25 an hour

CD $1.30 to $1.65 an hour

CD $2.20 to $2.50 an hour

$8.50 to $9.25 an hour

21.

When are you supposed to do the work on

this job?

CD Only at nightfrom about midnight to

-8 o'clock in the morning

C:::1 Might be any day or any hours during

the day or.night

CD Mostly during the week from about

8 o'clock in the morning to 5 o'clock in

-the.afternoon--

CD Every Friday, Saturday, and Sunday

only
-11

12.

4

fts

Which one of these tools would you use in thiS

kind-of work? -

CO

23. . 24.

How much schooling do you think you'd need .. About how much would you get paid whet: you

to get this job?

,. start on this job full-time?

c. No special schooling CD $6.00 to $6.75 an hour

CD At !mitt college degree and special

schooling

CD Only a grade school diploma

C.D A trade school certificate and training

on the job

0

CD $2.60 to $3.50 an hour

CD $7.90 to $11.50 anhour

CD $4.00 to $5.25 an hour

to
..,

,

. .

.1 ..# %

11 127

, .

.1 --- ---'' . - / #

TELEPHONE OPERATOR

26.

About how much would you get paid when you

start on this jab full-time?

0 $4:85 *.0 $5.25 an hour

0$1.75 to $2.25 an hour

0.$3.25 to $4.00 an hour

0$5.90 to S6.75 an hour

o

1

.

, Where would 'you work on this job most of

the time? 'i

0 In a factory, on an assembly.line

0 In a shop, like a hardware store

\0.0 In your own home

. , 0 In a building, like an office building

, 27

, What kind of work do you, think you would do

on this job?

- , CD Fix broken telephones

0 Help people make telephone calls

, 0 Send telephone bills to people

. 0 Help design new telephones

. 10.S

v

Mr
29:

How much schooling do you think you'd need

to get this job?

cp A high school diploma only

O A college degree and special training

CD Trade school certificate and special

training

0 Grade school only

13--

.. 28.

Which one of these tools would you use in this

kind of work?

CD

30.

When are you supposed to do the work on

this job?

0 Only from about.8 o'clock in the morning

to.5 o'clock in the afternoon

0 Almost any hours during the iiight Or day

0 Nights only frommidnight to 8 o'clock

in the morning

CD Only on weekendsFriday, Saturday

and Sunday

10,9
. LAUNDRY WORKER

14

31.

When are you supposed to do the work on

this job?

CD Nights only from about midnight to

8 o'clock in the morning

CD Weekends onlyFriday, Saturday and

Sunday

CD Whatever hours you feel like working

CD Mostly during the day from about

8 o'clock in'the morning to 5 o'clock in

the afternoon

32. 33.

About how much would you get paid when How much schooling do you think you'd need

you start on this job full-time? to get this job?

0 More than $5.00 an hour 0 High school diploma

CD $1.75 to $2.25 an hour CD Trade school certificate

CD $1.00 to $1.40 an hour 0 College degree

CD $2.75 to $3.25 an hour CD No special schooling.

PLEASE STOP HERE.

WAIT TO BE TOLD HOW TO DO THE NEXT GROUP OF QUESTIONS.
ADMINISTRATOR'S DIRECTIONS I

MS

V

Job Holding Skills (JHS)

SAY: Turn to page 15. Look at the directions in the box at the top of the

page. 'Read them silently while I read them'out loud.

The questions that follow are to find out how much you know about how people should act on a

joband what Lou might do on a job when certain things happen.

SAY: Number 1. If you feel too s

and let the boss know?

Copyright C 1978 by Educational Testing Service.

ck to go to work one day, would you call

I wouldn't bother it would't make any difference to the boss.

I might call if I happened to think of it.

I would always call otherwise the boss would get sore.

Mark the answer that tells what you would do.

NOTE TO THE ADMINISTRATOR

Whefe there is a balloon in a pictu' read the quotations in the balloon.

Continue reading the rest of the items in the same manner.

1

ra
15

The questions that follow are to find out how much you know about how people should act on a

joband what you might do on a job when certain things happen.

JHS Copyright 0 1978 by Educational Testing Service.

1.

If you feel too sick to go to work one day, would you

call and let the boss know?

0 I wouldn't botherit wouldn't make any

difference to the boss.

CD I might call if I happened to think of it.

0 I would always callotherwise the boss

would get sore.

2.

If you're-short of money before payday each week,

would you borrow what you need from somebody

at work? s 0 I would do that anytime I felt like it.

CD I might do that sometimes.

0 I would never try to do that at work.

0

3.

What would you say to a boss who said this to you?

0 Sorry, I'll try to get here on time from now on.

0 It's tough for me to make it here on time but

I'll try.

CD I do my workwhat's the difference if I'm late.

14.4
a

-16-

4.

How do you feel about this person who's dressed in a

*shippy way at work?-

C.5 It doesn't matter how you dress as long as you

do your job.

CD You should dress sharperthaillie rest so you

stand out and are noticed.

0 You Should dress about the same way asthe

other people and look neat.

S.

How do you feel about placing afew bets each

day at work?

CD I wouldn't do thatmost bosses wouldn't

like it. I. CD There's nothing wrongwith ita boss

wouldn't mind.

might do that sometimesbut I wouldn't

overdo it.

6.

When someone who's the buss gives you a lot of

orders about what to do, how do you feel?

CD It rubs me the wrong way to take orders and I..

usually letthe boss_know it.

CD I don't like taking orders, but do it if the

boss doesn't push it too far.

CD I usually don't mind. The boss's job is to give

orders.

1

-I
7.

How do you feel about someone taking a drink this

Way on the job?

CD It's okay, if you try not to get caught too often.

CD You should never do ita boss wouldn't like it.

C=> It's your own business when you take a drink:

most bosses wouldn't mind. .

8.

What would you think if someone working next to you

said this?

CD That person better lay off or I'll hit 'em right now.

CD I'm going to quit this job if the people here

bug me.

CD I won't look for trouble, I'll just ignore it.

/

9.

If you get a paycheck and you think the amount is

wrong what would you do?.

OLet.it go.it doesn't paY to make a fuss.

0 Go tell the boss he or she's a thief.

0 Tell the person who gives out the pay that you

think there's a mistake.

114
10.:

What would you say if. the boss asked you this?

U Not me, I only work if I'm paid to do it.

C) I would do it if I could get some extra time off.

CD It's too much trouble for me to work after

regulahours.

11.

If you finish the work that's given to you in the

morning and the boss doesn't tell you what to do

the rest of the daywhat would you do?

CD Just knock off and relax. It's best to work only

when you're told what to do.

CZ) Look-for-something else that needs doing on the

job or maybe help somebody else with theimork.

CO Try to look like I'm working at somethings° the

boss won't think I'm goofing off.

r

PLEASE STOP HERE:

WAIT TO BE TOLD HOW TO DO THE NET GROUP DF QUESTIONS.

U5
!ADMINISTRATOR'S DIRECTIONS!

vi

Work Relevant Attitudes Inventory (WRAI)

SAY: Turn to page 19. Look at the directions in the box at the top of the

page. Read thwosilently while I read them out loud.

The statements that follow have to do with wept youeel about

yourself; about-otherpeople and gaffing-along with them. For

each statement mark the boX next to the-way that tells how you

feel.

*A SAY : -=Number 1. You feel that you have little influence over the things

that happen-to you.

Strongly, agree.

Somewhat agree.

Somewhat disagree.

Strongly disagree.

Mark the way that tells how you feel.

NOTE TO THE ADMINISTRATOR

Continue reading the rest of the items in the same,manner.

a
The.statements.that follow have to do with ways you feel about

yoUrOeif, about other people and getting along with them. For

each'Statementmark the box next to-the way that tells how you

feel.

1. You feel that you have little influence over the things

that happen to you.

'strongly- agree

somewhat agree

0 ioniaiiifdiiiiree

0 strongly disagree

2. You don't' get much fun but of life.

strongly agree

0 somewhat agree

somewhat disagree

El strongly disagree

3'. Becoming a success is mainiy a matter of luck; hard work

N doesn'i'helVvery much:

strongly agree

7/that,.agree

0 somewhat- disagree

L:3 ,strongi.disagree

llodgied andreprinied permisdion of 'the author
ti

-20-

4. You feel that you are ab capable and smart as most people.

strongly agree

somewhat agree

somewhat disagree

strongly disagree

orN

The wise person lives for today end .lets tomorrow take

Care

0 _strongly agree

somewhat agree

CM somewhat disagree

strongly disagree

6. It is hard to get ahead without breaking the law now and

then.

0 strongly agree

somewhat agree

O somewhat disagree

. strongly disagree

7. Most people cannot be trusted.

strongly agree

somewhat agree

0 somewhat disagree

strongly disagree

8. MoSt bosses have it in for you and give you.a hard time.

strongly agree

somewhat agree_

0 somewhat disagree

Q strongly disagree 118
rr 4 9. Most work is dull,, and boring.

strongly agree

0 somewhat agree

0 somewhat disagree

- 0 strongly disagree

10. You feel happy.

0 almost always

0 usually

sometimes . 0 almoit never

11. Teachers have had it in for you and have given you a hard

time.

[] almost always

usually

sometimes

0 almost never

0

12. You feel that you are a failure.

almost always

usually

C:3 sometimes

0 almost never

r l:. During your spare time, you haye something to do that you

like doing.

almost always

usually.

sometimes

1:3 almost never
-22-

14. You get even with people who wrong you.as soon as you can.

almost always

usually

sometimes

ilmost never

-15. Would you say that your chances of becoming a respected and

law abiding member of your community are:

excellent

AD reasonably good

not very' good

0 very unlikely

16.. How many enemies do you feel you have:

a great many

some

E:1 a few

almost none

PLEASE STOP HERE.

'WAIT TO BE TOLD HOW TO DO THE NEXT GROUP OF QUESTIONS.
ADMINISTRATOR'S DIRECTIONS

vii

1- Job Seeking Skills (JSS)

SAY: :Turn to page 23. Look at the directions in the box at the top of

/SS

the page. Read them silenefrifiiii-Tread them out loud.

The questions thatfolloW are to find out Inuckyou know ahout_lookintforjobs. For each

qut.atien, mark the space next to the answer that you think is theright one.

Copyright 0 1978 by Educational Testing Smite:

SAY: Number 1. If you go to the State Employment Service to help you

find a job -.what would they charge if they found a job for you?

4

They would charge one week's pay.

They would charge whatever you can afford.

It doesn't cost you anything.

They would charge half a week's pay.

Mark the answer that you think is (fit , right one.

NOTE TO THE ADMINISTRATOR

Continue to,read the rest of the items in the same manner. ,

S

121-
-23-

nithat folio w are to fin Outhow muctru know about looking for jobs. For each

mark the space next to the answer that you think is the right one.

1. ,

so. Copyright 0 1978 by Eduestkcallesting Service.

If you go to the State Employment Service to 1.1001-

you find a job -what would they charge if they

found a job for you?

CD TheyWouldcharge oniweck's

C They would charge whatever you ;Ian afford.

:0 It doesn't cost you anything.

CD They would charge half of a week's pay.°

2.

On.which day are there more "help wanted" ads

for jobs.in the newspaper?

0 On Sunday 0 on Saturday

0 It'sthetme any day

0"on Tuesday

NOTE'TO THE ADMINISTRATOR

In the 'following sets 6f questiong, do not read material in want ads or

any content of application blanks.

12 2
SAY: Beginning with question43 we're going to look At some job "want ads"

._- _ _ ---

like the kind that you see in the newspaper when someone-wants to hire

people for jobs. Read the first want ad on your own and then I'll read

the questions to you that-ask about, what's in the want ad.

-NOTE TO THE ADMINISTRATOR

Allow approximately 30 seconds for reading of thead.

KEYPUNCH!:

NItES

AMORY THROUGH FRIDAY

Minimum !..2 Yrs Exp

GOOD STARTING SALARY

EXCELLENT COMPANY BENEFITS

IMMEDIATE 20% DISCOUNT

GN ALL FASHIONABLE MDSE

For 40110;n11111fIt OR 571000, EXT 477 . . kWorthman Smith

569 Washington St. N.Y.C. . 7ts Ale Local. heuston St sta)-

4.

. What kind of benefits do you get on this job?

O You,can make free telephone calls.

-0 You get your lunch free.

0_

O Yo can buy things cheaper than the

r ,gular price.

0 They pay your airfare to get to work.

t

NOTE TO ADMINISTRATOR

Follow the same procedure for' the want ads for questions 6, 7, and 8, and

for 9, 10, and 11.

0

3.

How much experience would you need for this job?

0 You don't need any experience.

,

0 At least one or two years.

0 Less than a year.

0 At least five years' experience.

5.When would you work on this job?

0 WeekendsFriday, Saturday and Sunday.

0 During the day from 8 o'clock in the morning

to five o'clock in the afternoon.

0 Five nights a week.

0 Sometimes during the day, sometimes at

night.

123
SALES HELP.WANTED

_ SALESPERSON,

CAMERAS

APPLIANCES

RADIOATV

ROBERTSON'S

VALLEY STREAM

Top notch experienced sales

person needed for our newest

store In the-Green Acres Shopping Ginter.

1 PM-10 PM, 6 days or

-6 PM-10 PM

High Salary + PM*

Opportunity.to Advance

Excellent Benefit Program

Apply at

Employment Office

480 Sunda* Hwy, Valley Sam

An Equal Opportunity Employer

7..

What would you do to gef this job?

0 Write a letter to the manager of the place.

0 Call the telephone number given in this ad.

theState_Employment Service and tell

Mein you want to apply.

0 Go to the address given in the ad and ask

about the job.

25--

6.

What would you do on this job?

CD Fix cameras-and radios-.

0 Sell things bra store.

0 Teach people how to drive a car.

0 Sell hoises to people.

1.

When would you.work on this job?

CD In the mornings forlive days a week.

O At night, on the weekends only.

CD Anyhourxotany_days-you-want-tcrpick:

CD Iiithe afternoon or at night for five days

a week. -

124
-26-

10.

L CLERIC

ADV. AGENCY

Handle diitribatiOn of mail

& N.-at

appretiOaii. FM_ giiicts or

-nialayeisietion. Good ad-

-itincemept 'appty for alert

person.

RADIO CITY AREA

Ms.Morse Circle 51000

9.

Where would you work on this job?

, CD In a department store.

CD In an office.

CD Ina factory.

0 In your own home.

11.

What would you do to get this job? What would you do in this job?

CD Call the State Employment Service and ask CD Sell things in a store.

them about the job. CD Write letters and mail them.

CD Write a letter to the place that has the job. CD Sort mail and make deliveries.

O Gb to the place yourself and ask about the job. CD Fix adding machines and typewriters.

O Call the place on the phone.

SAY: Questions 12 through 17 are about parts of an application blank--

like the kind you fill out when you go to get hired for a job. Look

at this part of the application blank and the things it asks you to

fill out.

NOTE TO THE ADMINISTRATOR

Allow approximately 30 to 45 seconds for the group to look over the blank.

Tben,read each question and alternatives.

1.2.5
12.-

-27-

APPLICATION FOR EMPLOYMENT

PERSONAL INFORMATION - Social Security Date Number

I NAME

2 PRESENT ADDRESS

PERMANENT ADDRESS

Last - First Middle

Street City State

4 PHONE NO.

Street City State

CITIZEN YES 0 OF U.S.A. NO 0

5 KIND OF WORK YOU PREFER

CIRCLE THE NUMBER OF- 6 EDUCATION THE HIGHEST GRADE COMPLETED 7 8 9 10 11 12 13 14 7 OFFICE MACHINES YOU OPERATE

IF RELATED TO ANYONE IN OUR EMPLOY,

I STATE NAME AND DEPARTMENT REFERRED BY

If you finished high school and got your diploma,

which number would you circle on line 6?

CD 14

13.

What would you put on line number .2 where it

says "present addreis?"

0 The address of the place where youlive now

0 The address where a friend lives.

0 The address of the place where

you're applying for the job.

0 The address of the place where you worked

last.

14.

If-you-have a.runcle who works for this place and

he told you about coming here to apply for a job,

where would you write your uncle's name?

0 Line 1

c:::) Line .4

'RD Line 7

Line 8

SAY: Now look at the next application blank on page 28 that someone filled

out. Read that over.

NOTE TO. THE ADMINISTRATOR

Allow about One minute for the group to look over the material. Then read

each question and alternatives.

1 96

I
28

FORMER EMPLOYERS (LIST BELOW LAST FOUR EMPLOYERS. STARTING WITH LAST ONE FIRST.) DATE MONTH AND YEAR

- . - -- NAME AND ADDRESS OF EMPLOYER SALARY POSITION REASON FOR LEAVING

May 1977

FROM . . Joy Manufacturing Co.

55 Eaton Place ,, 'Chicago, Illinois

$4.00

Parts

Assembler

4 Still employed

TO Prehent FROM Jan 1977 White's Dept. Store

735 Walton Blvd.

Chicago, Illinois

$2.90 Packer - Wanted better

job

TO . May 1977

July 1976

FROM West Side Housing Project

12 W. South'Street

Moline, Illinois

$2.50

Maintenance

Helper

Wanted more

money

Nov '1976 TO

Mar 1976 FROM

Speed Messenger Service

15 State Street

Chicago, Illinois

$2.20

____. _ Messenger

- Wanted moremoney May 1976 TO

EMPLOYMENT DESIRED

POSITION A

ARE YOU EMPLOYED NOW? Yes

CAN START DATE YOU An time SALARY DESIRED $4.50

IF SO MAY WE INQUIRE OF YOUR PRESENT EMPLOYER Yes

EVER APPLIED TO THIS COMPANY BEFORE? No WHERE WHEN

15.

.What job did this person hold right before the job

he or she holds now?

0 Packer

0-Maintenance helper

0 Messenger .;

C Parts Assembler

16.

In the blank where the A is, which of these things

would make sense if you wrote it?

0 247 Jay Street

0 $4.40 an hour

0 Sales person

0 No night work

17.

What should be put in the place marked B?

0 Don't put anything.

0 Other places the person applied for jobs.

0 Where the person worked last.

0 Where the person lives.

PLEASE STOP HERE.

WAIT TO BE TOLD HOW TO DO THE NEXT GROUP OF QUESTIONS.

1??
ADMINISTRATOR.S7-DIRECTIONS

viii "

eseSterkitY0i4 Of-Adult Oddipationi MAO

NOW tliril.to page 29.

'Look at 'the directions in the box at the top of the page. Read them

silently.Whili I reacrthem out loud.'

_- The ,statements that foIlowgive.the names of jobs that people do

and tell elittle ai?out.the-jOb. After each job it asks who you

think SHOULD-do that SO. Answer the question by putting a mark

in the ,box that tellshawyoU feel' about who should do-the job.

SSAO

SAY: Number 1. Sewing coaching operators are people who sew, clothing on

-machi4es'to sell to other people.

Who do you think should,be_sewing machine operatori?

Only women.

More women than men.

About the same number of men and women.

More. men than. .women.

Only men.

O

Mirk the, answer that tells how you feel. about who should do the job.

NOTE TO THE ADMINISTRATOR

ee'---

Continue reading the rest of the items in a similar manner.

_
/' AlThe statementsat follow give the names of jobs that5'people do

and. tell -a little about the job. Aftv.- each job it asks whe'you

thinkSHOULD do that job. Answer the qUeation by putting a mark

in,the box that tells haw you' eel about who should do the job.

ss10

, 1. Sewing machine operators are people who sew clothing on machines

to sell to other people. -":

. Who do You think SHOULD be sewing machine operators?

'ID Only women

El' More women than men

About the same number of men and women

C.3 More men, than women

Only men

\\2t lire fighters work at putting out fires.

S.

Who do you think SHOULD be fire fighters?

Only women

More women than men

About the same number of men and women

CI More men than women

Only 'men

3. Airplane pilots are people whose job is flying airplanes.

Who do you think SHOULD be airplane pilots?

Only women

El More women than men

E:1 About the same number of men and women

Q More men than women.

ED Only men

12)

Modified and reprinted by permission of the author
:Grade ichooiteacheriv.iiM,peOle-whe,teachkindergartei or one

of=the.;fixst!'six4fidett. ' ' WhO'dO-You think SHOULD,be.-srade school teachers?

C:1 00IY women

'MOre,womenthan men

El About the Beim number of men and women

El- -More men than women

0 Only men.

5. Nurses help take care of people when they're hurt or sick.

Who dd you think SHOULD be nurses?

ID Only women

El More women than men

0 About -the same number of men and women

El More men than women

6. Store salespeople. work -in shops or stores selling things to

their customers.

Who do you think SHOULD be store salespeople?

Only won en a More women than men

ID, About the same number of men and women

,E] More men than -women

0 Only-men

7. Train engineers are people who are paid to drive trains.

Who do you think SHOULD be train engineers?

a 0 Only women

0 More women -than men

El About the.eame number of-men and women

0 More men than. Women

Only men
-31-

8.* Elevator operators are people who are paid to. run elevator:,

taking-people up and down in tall buildings.

Who do you think, SHOULD be elevator operators?

Only women

More women than men

- About the same' number of men and women

El More men than women,

Only men

. 9. Mail carriers bring mail to homes and businesies.

Who do you think SHOULD be mail carriers?

Only women

More women than men

About the same number of men and women

More men than women

O Only men

10. Ballet dancers are people who work performing for others by

dancing gracefully to music.

Who do you think SHOULD be ballet dancers?

Only women

More women than men

About the same number of men and women

More men than women

El Only men

11. Writers are peOple whose job is using written .words to tell

others about their thoughts and feelings. They write books,

stories, plays, and poems.

Who do you think SHOULD be writers?

'Only, women

. More women than men .

About the same number of men and women

[] More men than women

[:] 'Only men
12. Carpenters are people who -make things out of wood

Who do you think SHOULD be carpenters?.

0

0

OnlY-woMen

More women than men

About the same number of

Moie men than' women

Only men.

;13. Librarians are people who work

and out.

WhO do you theinkSHO6LD be

0 Only women

Q. More women than men

r. men and women

to sell.

in libraries checking books in

librarians?

Abolit the Same number of men

0 More men than women 0 Only men

14. House cleaners are people who ar

houses clean.

Who do you think SHOULD be

o

D

0

Only women

.Morewomenthan men

About. ,the same numbef,of men and women

and women

paid.to keep,Other peoples

ouse cleaners?

More men

Only men

than women

1\

15. Secretaries are ,people,Who work in offices answering telephones,

-typing letters and papers, and greeting visitors to their offices.

Who do you think SHOULD be secretariesi

0 Only women

0

0

.(23

More women than men

About the same-number of men and women

More men thin women

Onlimen

Q
-33-

16. Plumbers are people who fix pipes in houses and other buildings.

Who &you think SHOULD'be plumbers?

only

El .More women than Men

El 'About the same number of men and women

-Mbre.men than .woken

Only men .

17.. 'Football coaches are people who coach football teams.

Who do you think-SHOULD-be football` coaches?

El' Only women

More_women.than-menE] About the same number of men and women

ED More men than women

0 Only men

. 18. Restaurant cooks are people who fix meals for the redtaurant's-

. customers.

Who do you think SHOULD be restaurant cooks?

0 Only women

0 More women than men

About the same number of men and women

More men than women

0 Only men194- Bus drivers are people who drive bused, collect -tickets`-nk money,

and give.ditections and information to the passengers.

Who do yOu think SHOULD be bus drivers?

Only women

More women than men

About the same number of men and women.

More ed-ihanromen

Only men

0
z

°

4

. / 20. Ship captains are people who are in charge of

C., Lt

-34=

Who do you think SHOULD be ship captains?

0 Only women

0 -More-women than. men

0 About the same number of men and women

large boats.

0 More men than women

0 Only med.

21. Airplane attendants work on airplanes serving food and drinks.

They make sure the passengers are safe and comfortable.

Who do you think SHOULD-be-airplane attendants?

0 Only women

0 More women than men".--

0 About the same number of men and women

0 More men than woien

El Only men

PLEASE STOP HERE.

WAIT TO BE TOLD HOW TO DO THE NEXT GROUP OF QUESTIONS.
The pficturei thin follow show everyday kincis,of -situations in which I person might be involved,

"Mark the space next to theWiibiitt tells beg ho*-you-would-feel-if you -were the -personin that

:scene.

Copyrigit 0 litbyEdocatioiserfestias.

I

- 1.

If you were the person getting the diploma how

would you feel?

CZ:Me diploma's not going to do anything.to

help someone like me.

*0 I might get a little better job with the

diploma.

0 I could really have if made with a diploma.

2.

What would you think if someone at the job said

this to you?

0 I would probably be the last person they

would give a.promotion to.

0 I would be one of the first people fo get , promoted on a job.

C.) I might stand a chance for getting a promotion.
-36-

3.

4s How do you think you'd make out if you were in

court?

CDThe judge would probably go easy with me--

I'm worth giving a chince

C:) I would probably get the worst possible .

punishment there is.

C=0 I would probablyget a small fine' -:but

nothing too bad.

r

4.

If you were the person the counselor was trying to

help, how would you feel?

CD There's probably nothing much the counselor

could do for me.

CD I know I could 'bake good on a job if the

counselor helped me get the right training.

01 might stand some chance on,Oob if I could

get some kind dtraining.

.5.

How:would you feel if the teacher said this to you?

.CDIt wouldn't do any goodI wouldn't learn

anything anyway.

CDIt's always wcillryingI can learn with

some help.

CD I'll tryt but it might be a waste of time.
:,," "^^" ; , .

SOYOU THINK ti

WAWA JOS HERE?

7,1

:

)

6.

What would you think if one of yourparents told

you this?

CZ) I could really make good if my family was

willing to make plans with meand back me up.

0 Planning out my ftiture with my family

wouldn't do any goodI wouldn't make

it anyway.

CD Maybe I could make it if my family backed me

upbut I have my doubts. f

7.

How do you think you'd make out if a job interviewer said this to you?

0 They would probably tell me I'm someone

they can't use.

0 I think they would offer me a pretty goodjob.

CD They might offer me some kind of a jobbut

nothing too good.

CONNMONNEmmar

8.

Hows.would you feel if/iou.saw this sign?

0 I might try for a jib in that store, but they

probably wouldn't want me.

CD They would turn me down cold and wouldn't .

think I'm worth giving a chance.

0 They wouldithink I'm worth hiring for a job

in that store.

13 7

I

1
r

=38

9.

'Whit usually happens ifyou are waiting to be

taken care of in a store?

CD I usually wilt until the sales person gets

around to paying attention to me.

CD I would walk away if the sales person is busy

they usually treat me like nothing.

CD I usually demand good service and expect to

be waited on right away.

10.

What would you do if someone came to your door

and said this?

CD I would register and vote--it's the only way

I can make things better.

CD I'd register and votebut it's not much use

for improving things.

0 No sense in even botheringmy vote doesn't

change what happens to me.

11.

How do you think you would make out if you were

one of the people the cop was talking to?

CD

1 If !didn't do it, the cop wouldn't blame me.,

(

:-; CD The cop might blame mebut I don't think I ;

would get in trouble if it wasn't my fault.

-CD I'm usually the one who would get blamed and

arrested even if I didn't do anything.

1 3f).
-39-

How would you feel if you were the perSon talking

to the group?

0 I'm not sure people would pay too much

attention to what I have to say.

CD People wouldn't listen to someone like me.

CD I can get across what I have to say and people

usually listen.

111=10111

13.

Suppose you were the person being cheered by

the crowd. How would you feel?

CD I'm the kind of person who can be a leader and

whO people look up tolike in this picture.

0 I could never be like that person in the picture

with people cheering me.

CD I might be good at some things that people

would look up to me for. \

14.

How do you feel when a sales person is showing

you something?

CD I'm always treated right and get good service

from a sales person in a store.

0 People who sell things in a store often treat

me like I'm nothing and try to get rid of me.

CD Sometimes sales people don't pay too much

attention to me but it depends on the store.
z-40-

15.

How do you think you would do if your supervisor

at work said this to you?

C") I might stand a chance for getting more pay

on a job.

CD The odds are they wouldn't think I'm worth

more pay.

CD I riuld usually be the first one to get more

spay on a job.

1 40
CONTROL GROUP STATUS' SURVEY

Interviewer Instructions

Print the name of the project'at the top of page 1,of the survey..

Please read the directions on the cover sheet of the Status Survey to the'

respondent. Then read aloud each of the questions in the survey. You

may show-the,questiens to .thereepondent:at>you read them. Fill in the

4. responsies.as the respondent answers each question. This instrument should

be filled in by the interviewer

_ . Try to read and explain the material in.a relatively informal manner. A The approach is one of working with the respondent to obtain information

. that is as accurate as possible.

Item ETS ID Number. Enter this number in the

Items 7-20: 't

boxes provided. The ID numbers for the

pretested .control group members are listed

on the rosters sent to you by ETS.

Self-explanatory. Fill in as-indicated..

(Notice the "skip" instruction for Item 15.
It 21-11: Self- explanatory. Fill in as indicated.

For /tie 32, Make a note.regarding respondents

_JOho pre'not looking for a job because they

Items 34-42:

41:

Item 44:

ta are in sChooli

Probe for a job choice when the respondent

seems hesitant or uncertain about whet_job-----

me or she would look for or likeliest.

Obtain a specific occupation-when probing

-fora job choice. For exempla, "salesperson"

or "stock clerk" rather than "work in a store";

"secretary," "receptionist," or "bookkeeper"'

rather than "work-in an office."

Self-explanatory. Fill in as indicated. For

Items 37, 38 and 39, if asked, fsmily may be defined,

as "anyone related to you.by blood'or marriage".

, -(Notice the "skip" instruction for Item 37.)

NoticthSt"trouble with Police" is defined as

"arrested, charged or booked." As noted in the

survey, status offenses would include parking

tickets,or runaway.

Self-explanatory.

1 2
,

114-ofthe.informstien-obtained in this its

.4hould.be.d4plieeted and hastily:your fili

. since 'it could be helpful in future follow-up

siffoits..

Y. Rearifie part, about payment for follow-up

only= if yoU are. sure that youeprogram is

'doing 'so.

Ask the respondent if there ere-any comments

fk

that he, or .she Would like to make. Such comments

might include feelings about the survey questions;

the interviewer, etc. Please record these cimMents,,

as close to verbatimas possible..

143s
a. Program Completion Survey

We're trying to find,out how people vittO have been in your training

program for awhile have been doing since they left the programlike'

how things haVe been working:Out for you iriganeral; what ye: think

you've gOtten out of beN in the,Progrcirn: what you are doing now and

what things you would like to do from now on: 'how you feel about Jobs;

and so on.,Ilkwould like to know your feelings about these things. V

plan to we what we find, out to make programs Ilke this better for trainees.

:

The answers you give to any questions will all be confidential and private.

Anything we find'out frorn these Interviews will be reported for a whole

group at a time (like a feil hundred) so that no one can know your answers.

However, all the information,requested is Important and your response to

each question will be appreciated.

Let me go over the questions with you. It should take no more than about

. 25 minutes. Remember, if you don't understand a question, please stop

me to make sure It's clear.

1 11111110, ASSESSMENT OF YOUTH ALTERNATIVES PROJECT

144
Counselor's Rating Form PROJECT

CITY

Counselor's Name

Nome of Enrotlee

Being Rated.

Date

11L,U6L1,f°11-

On this sheet are 11 statements about enrolleebehalvlor that counselor consider important in

detemining how well an enrollee is coming along. We would appreciate your evaluationof

how these apply to this enrollee.

Please read each statement carefully. Then circle one of the numbers 1 to 5 that best indicates how the staternent applies to the enrollee:-

1 = This 2 = This is 3 = Sometimes 4 = This is 5 = The enrollee

describes true this is not Is not

the entollee most of true of the usually ,like this

perfectly. the time. enrollee. so. at all.

CIRCLE ONE NUMIIR.

1. Pays attention to good grooming and dresses appropriately 1 2 3 4 5

2 is not very open about discussing personal and Job problems 1 2 3

3. Shows a lot of resentment and hostility 1 2 3 4 5

4. Is cooperative and willing to listen to advice 1 2 3 4 5

5. Makes realistic plans about future lobs 1 2 3 4 5

6. Shows little poise or self-assurance 1 2 3 4 5

7. Is coherent in expressing himself (herself) 1 2 3 4 5

8. is motivated to want to work and expend effort

9. Does not show good day-to-day planning so that he (she) can it

1 2 3 4 5

handle the job (let's borne life interfere, for example) 1 2 3 4 5

0 10: Indicates a willingness to enroll in school

or some sort of training on apart-time basis 1 2 3 4 5

Shows qualities that indicate he (she)

will do well atter.leaving the program 1 2 3 4 5

L1
,

-ASSESSMENT OF YOUTH ALTERNATIVES PROJECT

Program CoMpletlOry Survey

I. ETS I.D. NO

3. NAME OF RESPON DENT

4. HOME ADDRESS

C'

PROJECT

-CRY

2. DATE OF

INTERVIEW

1111=1=1 11111111121111

MONTH DAY YEAR

LAST NAME FiRST NAME MI. MAAR AND STREET

CRY AND STATE 5. TELEPHONE NUMBER WHERE RESPONDENT CAN BE REACHED

RESPONDENT'S SOCIAL SECURITY NUMBER

7. SEX 0 Male 0- Female

AREA CODE ri

DP CODE I

TB.EPHONE NUMBER

8.. DATE OF BIRTH

=ompar

LUUlI MONTH

.1.111111,=, DAY 9. Since you'Ye been in the pv)gram have you been

einrking on a general eguNaiency nigh schcor diploma (GED)? Yet 0 No

9.a. If yes. have you gotten the GED yet? Yes o No 0

9b. Did you get (or will you be getting) your high 'school diploma in June?

0 Yes o No: Will get it later (end of 0 No: Leaving school

summer or next school year). without it.

10. What types of things did you 10t.3 part in during the program? (Check all those which are applicable.)

1 Personal counseling 5 Cossroom training: GED

2 0 Job counseling 6 0 Working on a Job

3D ,Classroom training: vocational skills 7 0 Referral to a firli-rtme permanent Job

4 0 Classroom training: remedial education 8 CI Other (specify)._

'11. Of the things you took part in during the program, which two do yot think wore most.heipfui to you?

(Check only two.)

1 0 Personal counseling 5 CI Classroom training: GED

2 0 Job counseling . 6 0 Working on jot

3 0 Classroom training; vocational skills 7 0 Referral to a full-time permanent Job

4 0 Classroom training: remedial education 8 CI Other (specify):

12. How doyou feel about your experiences in'the program?

0 Very satisfied 0 Satisfied 0 yery dissatisfied . 13. Are you still enrolled In the program?

0 Yes (Skip to question 22.) 0 No (Go on to question 14.)

14. How many weeks has it been since you left the program?

1 d 6

_WEEKS

a , ;
45. ,,Are you presently Working? ,

y e i Od On to question 16.) NO (Bklio'to question'20.)

16. ,Are,you presently working 36 Or morlitiOUrfper week?

[];Yes (GO onto question 17.) 0 No (Skip to qusspoil 21.)

--47. Where do you work? NAME OF COMPAW

NUMBER MO STREET OR ROUTE CV AND SINE

TELEPHONE NUMBER 18. What is your lob title?

49. How helpful was the program in preparing you for this job?

. 13 it was nQ help at oll. 0 It helped,a little, now and then.

. .a

It was very helpful ink lot of ways.

\

, . tkic:to question 22.)

ave.you walked since leaving the program? Yes

. o 2 . Whaire you doing nova (Cheti..k aN those which are applicable.)

.1 0 Looking for a Job 3 0.1h another CEA training program

2 0 in s\

chool 4 Nothing special (probe: How do you spend yoUr day? Briefly describe.)

:

_

0 A

I would like to know What you think about certain things relating to Jobs.

, 22. 'Do you think you'reable to get the kind of job you want?

0 I would have a \ I might have I won't have

very good chance. \ some chance. much chance.

23. Do you think you're able to do the work on the job well enough io satisfy any boss?

El I won't be able,

to, on most Jobs.

\ I might be able I definitely will be

to, on most jobs. able to, on any job.

24. Do you think you're able to get by oh a jobWithout a lotof help from the boss, or the people you.work with?

0 Yes, I'll get by on bI may need a little I'm going to need a lot

my own without help help sometimes. of help from other people.

, \

or ;

, . 25. Do you think, if you have to learn somethlrig new on a Job, you could learn enough to do the Job right?

:Yes, I could, anytime Maybe I could learn,

for any job. - for sortie jobs.

I would have a tough time If I

have to learn something new.

26. Do you think you've got, what it takes to get proinoted and move up to better jobs?'

0 It wouldbe rough for l 0 Maybe I could, 0 I could get promoted

4 me to get promotions. on some jobs. for sure, on any job.

27. Do you think being In the program has made you feel more like you're going to be able to make It in a job

later on?

it was no help to me at all. 0 it was some help to me. It helped me a lot.
S, !'

People look for different things in their choice of Jobs

28. How important b it that you get to dO inteiestIngwork?

. -That's real irriportant a maybe Important;

to me.. .I'm riot too sure.

f 0

-29. How Important is I that yOu baryget paid what you're worth?

n That's real im ant - It mcy be Important; I don't really

;. . . .

to me. I'm not too sure. care about that. - ,.

.:

. .,30. How Important It It that you wouldn't have to worry about being fired?

I don't really

care about that. - -

a

.: .- 0; 31. Hoy Important Is It that you feel like you're doing work that's worth doing?

That's real Important. It may be Important; - - I don't really

'to me.. I'm not too sure. care about-that. '4,

0, That's real Important

fro me.

It may be Important; I don't really

I'm not too sure. care about that.

32. How Important is It that you can det raises and make more money fast?

jj- Thcii's real Important It may be Important;

-* ' to me. I'm not too sure.

.

33. How Important lilt that you have a chance for steady work?

'0 That's real Important It may be Important; \L.

tome. I'm not too sure.

I don't really

care about that.

_ I don't really

care about that.

cd)...1-`

34. WY' at kind of full -time Job would (did) you look for right after leaving the program?

Write In kindof Job:

35. What kind of full-trrne Job would you Ilke best right now?

Write In Job choice.

36. What Is the lowest hourly pay you would

take for the Job you would Oko but right now?

37. How much do you know about what It takes to do the Job you would like but right now?

I know a lot about

that kind of Job.

0 !know a few things I don't really

about what the Job takes. know much about It.

PER HR.

38. How willing would you be to go Into a tilning program for the Job you would like !mist tight now?

(Check only one.) 1 I wouldn't bother. 3 1.would do It for no pay.

2 I would do It only If rwere paid. 4 I am already trained fdr that Job.

39. Since being In the program, has there been a change In the way you get along with your family?

I have no family. (Skip to question 42\.)

I get along better with . I get along worse with

them than I used to.

get along with them

them than I used to. the same way I always did.

40: How does yhur family feel about now you're doing (did) In the program?

They think I'm doing great. They thinkTm getting by okay. They think I'm doing poorly.

1 or)

I )
M. Out of whatyou earn a week, about how much do you give to yourfamily? S

I don:t.eam any money.

42., Since being in the program have

there been any changes in the

way you use your money?

(Check one box in each row.) .

1) Savings

2) Home and family (rent, food, clothing)

3) Leliure (recreation)

4) Educption

"43. How often as you save money from your pay? (Check only One.)

I don't earn any money.

.0 I save something Sometimes I save.

wore

. El

PER WK.

LESS MAE

I never save.

any of my pay.

44. Which of these people or agencies have been giving you a hard time lately?.(Check as many as apply.),

1 Supervisor at work

2 la People you work with

3 Social worker (Welfare)

k Employment agencies

5 Program people

(like the counselor)

6 The police orjhe Courts

7 Neighbors

8 LaWyers

-9 Credit colleCtion outfits

10 Storekeepers

14 Otheri (specify):

11 Somebody in your family

12 'A hospital, or people who

work in a hospital (like the doctors; the clerks)

13 . Friends

..)1 X 45. Since you've been in this program have you gotten in any trouble with the police: been arrested, charged

or booked? (Do not Include status offenses, such as pdrking tickets or runaway.)

No

Just once. A couple of time More than g couple of times

46. How Important is It to you to keep out of trouble with the police and the law? t

Real important: I go out of I usually try to steer clear unless Not too Important: If I get In

my way to avoid trouble. I'm pushed real hard. trouble I don't care too much.

fej

47. We appreciate the help that you've given us. We would like to keep up with how you're doing. We will want

to talk with you again In the next few months. At that time we'll pay you $5.00 for your help. Please give, us

the names and addresses of two people who will know where to reach you during the next few months.

Name Name

Address Address

Phone Phone

Relationship Relationship

48. Additional Additional Comments by Youth: ,

ATTACH ADDITIONAL SHEET. IF MORE SPACE NEEDED. .

Additional InformaSon [To be obtained by Intnriower from the program records]

1) 'Number of different work sites to which the 2) Number of. days

/ enrollee has been assigned since joining the program absent from the program

14_
. .

77 2- ,

iiperv4ors: thitiiig Form PROJECT

CITY Work Supervisor's Name

1...117

Date

Name of Enrollee Being Rated 1 1 1 1 1 1 1 How many months has the enrollee worked for you'

I I ETS1.D. NUMBER

Below are 10 statements about things that work supervisors consider Important when It comes

to how the enrollee is doing. We would. appreciate your telling us how each one applies to

this enrollee. The informai;on Is strictly private and will not have any effect on the enrollee

In any way.

Please read ea ch statement carefully. Then put a check in one of the five boxes to show how

that statement fits the enrolleethat you're rating.

1

- KNOWS HOW TO FOLLOW INSTRUCTIONS PROPERLY.

This describes This is true . Sometimes this This is not

just hoW the most of the Is true of the usually so.

enrollee is. time. . enrollee.

, I -

..

. 0

1.

00

The enrollee

ispotilke

this at

2. RESENTS TAKING ORDERS FROM THOSE WHO SUPERVISE HIMIIER. , .--

, .. , This describes, This Is true . Sometimes this This Is not _ z The enrolee

just' how the , most Of .the Is true of the usually so. Is notlikrso,

enrollee Is. .- time. enrollee. thit at all. _ 0 r- .: 0 ., R Li 0 - : .- e .1. r, ,

3. KNOWS HOW TO DRESS RIGHT FOR THE JOB. V

This describes This is true Sometimes this This is not The enrollee

just how the

I

most of the Is true of the sually so. -.. Is riot iike . , e.

enroileeis. % time. .enrollee. ...9 i

,, this of all.

P 0

4. TAKES SOME PRIDE IN THE WORK AND DOESN'T JUST RUSH THROUGH TO GET IT FINISHED.

, . This describes This is true Sometimes this This is not The enrollee

just how the most of the Is true of the usually so. Is not like

enrollee is. time, enrollee. this at all.

O"

15C

PLEASE CONTINUE ON REVERSE...

J
1 5. HAS TO BE TOLD WHAT TO DO EVERY MINUTE OR CAN'T KEEPBUSY. \

This describes ThIs Is true Sometimes this This Is n4 The enrollee

Just how the. most of the Is true of the Is not like

enrollee Is. time. enrollee.

usually

this at all. '

- 6. GETS ALONG WITH OTHERS ON inE JOB.

This describes= This Is true Sometimes this This -is not The enrollee

Just hoW the most of the Is true of the usually so. Is not like

enrollee Is. time. enrollee. this at all.

. '

7. 'CAN'T GET TO WORK ON TIME.

This describes ThIs Is true Sometimes this This Is not The enrollee

Just how the most of the Is true of the usually so. Is not like

`,.' enrollee Is. time. enrollee. this at all.

t 0

8. SHOWS SOME INITIATIVE IN TAKING ON A PIECE OF WORK.

This describes This Is true Sometimes this This Is not The enrollee

Just hOw the most of the Is true of the usually so. Is not like

enrollee is. time. enrollee. this at all.

-0 . //

9. DOESN'T MAKE TROUBLE ON THE JOB.

ThIs describes This Is true Sometimes this This ls not The enrollee

Just how the most of the is true of the usually so. Is not like

_enrollee is. - time. enrollee. this at all.

10. ASKS QUESTIONS IF PROBLEMS COME UPDOESN'T JUST GO AHEAD AND DO THE JOB WRONG.

This describes ThIs Is true Sometimes this This Is not . The enrollee

just how the most of the Is true of the usually so. Is not like

enrollee Is. time. enrollee this at all.

0 , 0

1/

5609901 RR69P30V 284141 Printed In U.S.A.

151
CONTROL GROUP STATUS SURVEY

This instrument is to be filled in by the interviewer,

NOT the respondent.

We're trying to .find-out how people have been .ac.) 1g since

our. last contact--like how things have been working out

for you in general; 'what you are doing now and what things

you would* like to do from now on; how you feel about jobs;

and so on. We would like to know your feelings about these

'things:

The answers yob give to any questionswill all be confidential and private.

Anythir1g we find out froM these interviews would be reported for a whole

group at a time (like a few hundred) so that no one will know your answers.

However, °lithe inforMation requested is important and your response to

each question will be appreciated.

Let me go over the questions with you. It should take no more than about -

25 minutes. Remember, if you don's understand a question, please stop

me to make sure it's clear.

ASSESSMENT OF YOUTH ALTERivATIVES PROJECT

a

IL'Nt.r /...#r.)

, .

. 1
H11111111

ASSESSMENTOFYOUTHALTERNAIV/ES PROJECT i

CONTROL GROUP STATUSSURVEY_- PROJECT

cav

1. ETS I.D. NO. 11 [it I 11111

3. NAMEOF RESPONDENT

2. DATE OF

INTERVIEW H 11 f 1 1 MONTH DAY YEAR

'Al NAME FIRST NAME

, 4. HOME ADDRESS moaramostim

CItY AND SIAM

MI

5. TELEPHONE NUMBER WHERE RESPONDENT CAN BE REACHED

6. RESPONDENT'S SOCIAL SECURITY NUMBER

7. SEX Male Female

AREA CODE 1REPHONE HUME*

8. DATE OF BIRTH I .1 1 1 -I 1 MUMH DAY YEAR

9. Have you beenworking on a general equivalency

high school diploma (GED) over the last 6 months? Yes 0 No 0

9o. If yes, have you gotten the GED yet? Yes No

9b. Did you get (or will you be getting) your high school diploma in June?

Yes No: Will get it later (end of 13 No: Leaving school

summer or next school year). without it.

10. Over the last 6 months, tave you been enrolled in any government

sponsored employment and training program? (If yes, on this list,

enter the number of weeks that you were in each program.)

1 CETA Job or Training r-r-1 weeks

2 Summer Youth Program c7r-1 weeks

3 job Corps weeks

4 Young Aault Conservation Corps r-r-/ weeks

5 Neighborhood Youth Corps )T1 weeks

6 Other-Government Sponsored Programs rri weeks

(Name of Program)

11. What types of things have you taken part in over the last 6 months?

(Check ALL those which are applicable.)

1 0 Personal counseling 5 Classroom training: GED

2 0 Job counseling 6 Working on a job

3 Classroom training: vocationol skills 7 Referral to a full-time permanent job

4 Classroom training: romedial education 8 None
12. Of the things you took part in over the last 6 months, which TWO

of you think were most helpful tc you" _(Check only TWO.)

Personal counseling 5 Classroom training: GED

2 Job counseling. 6 Working on a job

3 Classroom training: vocational skills 7 Referral to a full-time permanent job

4 Classroom training: remedial education 8 None

13. How do you feel about what you have been doing over the last 6 month's?

Very satisfied Satisfied Very dissatisfied

14. Have you worked at any time during the last month?

15. Are you presently working?

Yes (Go on to question 16.) No (Skip to question 20)

16. Are you preset-fly working 30 or more hours per week?

Yes N0

17. Where do you work?.

18. What is your job title'

NAME OF COMPANY- NUMISER AND STREET OR ROUTE CtD ANO STATE TELEPHONE NUMBER f

0 Yes 0 NO

19. Which one of the following sources' was most helpful in getting you your

job? (Check only one.)

1 School Counselor

2 Friends or people in neighborhood

3 Newspaper

4 Government (public) employment agency

5 Non-government (private) employment agency

6 Family

7 Help wanted sign

8 Church or community leaders (e.g., teachers, ministers, and so on)

9 0_ Other (specify):

() None of the above: I got it on my own.

20. What are you doing now? (Check all those which are applicoblo.)

1 Working and not

looking for a job

2 Looking for a job

3 0 In school

4 In a CETA training program

5 Nothing special (Probe: How do you spend your day? Briefly describe)
I would like to know what ydu think about certain things relating to jobs.

23.. Do you think you're able to get thekind of job you wont?

I wouldhove a I might hove I won't have

vary good chance. some chance. much chance.

22. Do you think you're able to do the work on the job well enough to satisfy any boss?

I won't be able I might be able I definitely will be

to.con most jobs. to, on most jobs. able to, on any job.

23. Do you think you're able to get by on a job withoutd lot of help from the boss, orthe people you work with?

Yes. I'll get by on

my own without help

I may need a little I'm going to need a lot

help sometimes. of help from otherpeople.

24. Do ycu think. if you hove to learn something new on a Job, you could learn enough to do the job right?

Yes, I could, anytime Maybe I could learn,

for any job. for some jobs.

25. J you think you've got what it takes to get promoted and move up to better jobs?

it would be rough for Maybe I could. I could get promoted

me to get promotions. on some jobs. for sure, on any job.

I would hove a tough time if I

hove to learn something new.

26. How important is it that you get to do Interesting work?

That's real important r;_i It may be Important;

to me. I'm not too sure.

27. How important is it thot you con get paid what you're worth?

That's real important It may be important;

to me. I'm not too sure.

I don't realty

core about that.

I don't really

core about thot,

28. How important is it that you wouldn't hove to worryabout being fired?

That's real important It may be important; I don't really

to me. I'm not too sure. core about that.

29. How important is it that you feel like you're doing work that's worth doing?

fiats real importont

to me.

It may be Important: I don't really

I'm not too sure. core about that.

30. How important is It that you con get raises and make more money fast?

mot's real importont

to me.

It may be important; I don't really

I'm not too sure. core about that

31. How important is it that you hove a chance for steady work?

fiat's real important It may be important;

to me. I'm not too sure.

!don't realty

core about that.

)
32. Whqt kind of full-time job would (did) yo-u look for right now?

Write in kind of job.

33. What kind of full-time job would you like best right now?

Write In job choice.

34. What is the lowest hourly pay you would

take for the job you would like best right now? S Pis Hs

4...

,

35. How much do you know obout what it takes to da the jab you would like but right now?

I know o lot about

that kind of job.

I know a few things

about what the job takes.

I don't realty

know much about it.

36. How willing would you be to go into a training program for the, jab you would like best right now?

(Check only one.)

1 I wouldn't pother.

2.0 I would do it only it I were paid.

37. Over the last 6 months, has there. been a change

I hove no fomily. (Skip to question 40

3 I would da it for no pay.

4 I am already trained for that Job.

in the way you get along with your

family?

I get otong better with I get along worse with I get along with them

them thon't used to. them than I used to. the some way I always did.

38. Haw does your family feel about how you've been doing over the last 6 .months?

.. They think I'm doing great. They think I'm getting by okay. They think I'm doing poorly.

39. Out of what you eorn a week, about how much do you give to yourfamily? S Prawn.

I don't earn any money. MORE LESS SAME 40. Over the last 6 monthslhave A) Savings

there been any changes in the 2) Home and family (rent, food, clothing)

way you use your money? 3) Leisure (recreation)

(Check one box in eoch row.) 4) Education

41. Haw often do you save money from your pay? (Check only one.)

I don't earn any money.

I sove something Sometimes I save. I never save. . any of my poy.

42 .Which of these people or ogencies have been giving you a hard time lately?.(Check as many as apply.)

1 Supervisor of work 6 The police or the courts 11 Somebody in your family

2 0 People you work with 7 Neighbors 12 A hospital. or people who

work in a hospital 3 0 Social (Welfare) 8 0 Lawyers (like the doctors: the clerks)

4 Employment agencies 9 Credit collection outfits 13 Friends

5 School Counselor 10 Storekeepers

1 4 0 Others (specify)

1 a
43. Over the last 6 months, have you gotten into any troubjet-with the. police;

been arrested, charged or booked? (Do not include status offenses, such

as parking tickets or runaway.)

0 No

0 Just once. 0 A couple of times .0 More than a couple of times

How important is it to you to keep out of trouble with -the police and the law?

0 ,Real important; I go out of

. my way to avoid trouble.

0 I usually try to steer clear unless 0 Not too important: If ! get In

I'm pushed real hard. trouble I don't care too much.

45 4re appreciate the help that you've given us. We would like to keep up with, how you're doing. We will want

to talk with you again in the next few months. At that time well pay you $5.04 for your help. Please give us

the names and addresses of two people who will know where to reach you during the next few months.

Name Name

Address Address

Phone 'Phone

Relationship Relationship

46 Additional Comments by Youth;

ATTACH ADDITIONAL SHEET. IF MORE SPACE NEEDED.
PARTICIFANT'FOLLOO-UP,SURVEY

Interviewer Instructions

,

The Follow-Up Survey is to be administered to every individual who has

participated in the program and every individual who nerved as part of the

contra group for the program. Different forms of the Survey have oeen

developedlor.participantsend controls so please be sure that you are

administering the correct one.

Follow-Up Survey

Please read the introduction on the cover sheet of the rollow-Up Survey to

the respondent. Then read aloud each of the questions in the survey. You may

show the questions to the respondent as you read them. Fill in the responses

as the individuAl answers each question. This instrument is to be filled in

by the interviewer only.

Try to read and explain the material in a relatively informallianner.

The approach is one of working with the respondent to obtain information that

is as accurate as possible.

In the box in the upper right corner of the first page of the survey,

indicate how many phone calls, home visits or other contacts were needed to

complete the questionnaire.
_ Item -1: ETS,ID number., Enter this number in the boxes provided.

The ID numbers for.the pretested respondents are listed

on' the rosters,sent'to you 14 ETS.

items 2-10: Self-explanatory. Fill in as indicated. Notice the

"skip" instruction "for Item 10.

Item lls

Them 12:

Item 13:

Self-explanatory. Notice, however, that for this item

' and a number of'those that follow, the interviewer is

required to read the correct tense in relation to the job

experience items. That is, "do" or "did," II is II or "was,"

"present" or "last" are required when asking job-related

questions Of those who either are working full-time new or

have worked full-time since leaving the program or since

program personnel's last contact with the respondent.

Obtain a brief and simple description from the respondent

of what constitutes the main business of the company named

in Item 11. (For example, "a factory where auto parts are

made"; "a store where clothes are sold.") Disregard box at

eha-of response line.

Be as specific as possible in the job title. If the respondent

does not know the job title, obtain a specific description of

the job tasks or duties. Disregard box at end of response line.

15D
-3-

Items 14-23: Self-explanatory. Fill in as indicated. Notice the

"skip" instruction for item 19.

Item 24: After reading the question, Asmmi.all ten-options: Check

only one box as directed.

/tems'25-36: Self-explanatory. Fill,in as indicated.

Item 37: Notice the "skip" instruction.

For Item 37a, indicate the specific type of work

perforted on the job and obtaih a job title if possible.

Items 38-39: Self-explanatory. Fill in as indicated. Notice the

"skip" instruction for Item 38.

Item 40: After reading the question, read the 8 options and check

all boxes that apply.

Items 41-43: As indicated in the "Interviewer note," Items 41-43 are only

for respondents presently employed. Note that the "skip"

instruction in Item 42 should have appeared in Item 43.

Item 44: Item 44 is intended only for respondents who are presently

unemployed.

1
Item 45:

-4--

Self-explanat6ry: Fill in.as indicated. Probe for a

job choice when the respondent seems hesitant or uncertain

about what job he or she would look for. °Obtain a specific

,occupation when probing for a job choice. For example,

staesperson te or "stock clerk" rather than "work in a

store"; "secretary,". "receptionist," or "bookkeeper" rather

than'"work in an office." Disregard box at end-of responseline.

Items 46 -54: Self-explanatory. Fill in as indicated. Notice the "skip"

instruction for Item 49.

0. Item 55: Self-explanatory. Check as many as apply.

Item 56: Notice k, "trouble with the police" is defined. as

"arrested, charged or booked."_ As noted in the survey,

status offenses would include parking tickets or runaway

and should not be included.

Item 57: Self-explanatory.

Item 58: Copy this information and keep it in your file as a

reference for a possible 8 month follow-up. Read the

part about payment for follow-up only if you are sure

.that your program is doing so.
0

Item 59:

I

Item 60:.

-5-

. Intended only for iesptdents who answered' Item 11.

I

I t e V

. __

-., 'After reading the item, show the Employer's Rating Form

to therespondent and go over .the questions if necessary.

Stress confidentiality ofthi infprOationo be obtained.

If 'the respondent answers 41 to Item 59, check the appro1 priate box and Malin the.name of the job supervisor as

indicated:. If the respondent answers no to Item 59, check

.the the appropriate box and do not attempt to contact the

employer.

Soli .t any additional comments that the respondent may

care to make about future job plans, feelings about the

training program or school, or the survey just,completed.

Employer's Rating Form

Fill in the ETS ID nuMbet of the `respondent being rated and the date of the

Follow-Up Survey at the top of the form. Print-the name of the individual on

the appropriate line in the introduction. The completed form shoulebe

hm. returned to you. Please remember to enclose-a stamped, return- addressed

envelope.

S. 1 6 2,
=

Program Follow-up Survey

=MP

AMP o

p We're trying to find out how people who have been in your training

program for a while have been doing since they left the prograri;Ilke

how things have been working outsfor you in paneral; what you think

you've gatten out of being in the program; what you are doliig now and

what things you would like to do fromnow on; how you feel about jobs:

and so on. We would like to know your feelings about these things. We

plan to use what we find out to make programs like'this better for trainees.

The answers you give to any questions will all be confidential and private.

Anything we find out from these interviews would be reported for a whole

group at a time (like a few hundred) so that no one will know your answers.

However, all the informatan requested is important and your response to

each question will be*appreciated.

Let me go over the questions with you. It should take no more than about

. 25 minutes. Remember, if you don't understand a question, please stop

me to make sure It's clear.

ASSESSMENT OF YOUTH ALTERNATIVES PROJECT

16
4

ASSESSMENT OF YOUTH ALTERNAINES PROJECT

P.

Follow-up Su.;Vey,

MN.

49 NO. OF PHONi NO. OF HOME WITS NO. CONTACTS

, 1. ETS J,D. NO. I. .

3. NAME OF RESPONDENT

2. DATE OF

INTERVIEW_

4. HOME ADDRESS

NUMBER AND STREET

CRY AND STATE

, 5. TELEPHONE NUMBER WHERE RESPONDENT CAN BE REACHED ....

DP CODE

AREA CODE TELEPHONE NUMBER 6. RESPONDENT'S SOCIAL SECURITY NUMBER _L LNI I NI 11

'7. SEX Male Female

%,

8. DATE OF BIRTH 11111:11111

MONTH DAY YEAR

9. How many weeks has it been since you left the program (number of weeks)? WEEKS 10. Are you presently working fu(I-time (30 or more hours per week)?

I la yds. (Go on to question 11.)

No, out I have wormed full-time since leaving the program.

- (Go on to question 11, but ask about most recent job.)

No, I have never worked full-time since leaving the program.

(Skip to queition 36.)

11. Where do4did) you

(most receetly) work? NAME OF COMPANY' NUFASER AND STREET OR ROUTE a. e

12. What kind of business is (was)' that?

CITY AND STATF TDEPHONE NUMMI

13. What is (was) your job title?

14. How helpful was the progrom in preparing you for this job? ,

was no help at all. It helped a little, now and then. If was fiery helpful in a lot of ways.

15. How many hours a week do (did) you usually work? I 1 HOURS 16. How many weeks have you been (were you) on that job? a WEEKS

a,
17. How much per hour do (did) you earn at that job? 1 DOIt

(Was) this a temporary CETA Job?

CENTS

PER HR ,0 Yes No

19. Is (Was) ih your first job since you left the program? 0 Yes No

(If yes, skip to question 23.)

20. How many jobs (full -time and part-time) have you had since leaving the program? low

.1; Why did you leave your last job?

I quit. I was laid off. I was fired.

21a. What was the main reasdnfo that? (Probe )

`S.

/ I

22. What was your starting hourly salary

on your first full-time job after leaving the program? 1111511111 F ER HR. 23. How long were you out of the program/before you got your first full -time job? WEEKS 24. Which one of the following sources was most helpful in getting you your first full-time job?

(Check only ono.) 1 `Program staff

2 Friends or people in neighborhood

3 Newspaper

4 Government (public) employment agency

5 Non-government (private) employment agency

6 Family

7 Help wanted sign..

8 Church or community leaders (e.g.. teacherszt ministers, and so on)

9 Other (specify)

0 None of the above: I got it on my own.

25. In how many places did you actuqiljr

fill out an application before you go; your first full-time job? PLACES 26. In how many places did you actually get to have an interview before you got your first job') PLACES 27. Did the program staff refer you to_any of those places

where you either filled out an application or were interviewed? Yes No

28. Since leaving the program how well have your jobs gone?

Much better thar I

thought I would do

Worse than I About what I

expected to do expected I would do

29. Have you gotten a raise on your present (or last) full-time job Yes No

No

29a. If yes: Number of raises

;

30. DId ybu get a promotion on that full-time job? Yes No

1 6 5

Q
I would like to find out how you feel about your most recent full -time job,

(Refer to job indicated in'answer to question 11).

31.What are your feelings about the kind of work you do (did)?

It's great; I like it a lot. It's okay. Don't like the work at all.

1 32. How's your pay for the kind of work you do (did)?

Good pay for the kind Just about what it A lot less than the

of Job I have.(had) - should be Or the job job is (was) worth

/

A Il

33. If you nave your way, would you want to wc.,k for ttlis place five years from now?

Definitely not. I'm not ure. Yes, I'm sure I would.

34. If you knew then what you know now about this obwould you have taken the job?

1-1 / Nonot if I knew I'm not too sure. Yes, I would take

what I know now. the job there again.

35. When you finish a day's work, db (did) you feel like you did something worthwhile?

I almost never feel Sometimes I feel

(felt) that way. /(felt) that way.

36'. Since you left the program, have you

received any unemployment compensation payments?---

36a. If yes, how much per week /

have you received'? S

I almostalways feel

(felt) that way.

Yes No

PER WK. 37. Since you left the program have you done any part-time work fc,r less than 30 houis

per week? 0 Yes No

If no, go on toguestion 38.

37a. If yes, what kind of work aid you do on your last job?

(Describe type of job.)

37b How much per hour did you echn at that job? ....' S PER HR.
Here are some question

with.

about educational or training activities you might presently be involved

38. Are you now going to school or involved in any other training activity? Yes No

(If no, skip to question 41.)

39. Is your education or training program full- or part-time?

Full-time Part-time

40. Which describes your education or training program? (Check all those that apply.)

1 Other CM training program

2 Regular high school

3 Business/vocational, technical/trade high school

4 High school equivalency program (GED)

5 College

6 0 Business/vocational technical postsecondary (not high school)

7 Union apprenticeship program

8 Other (specify)

/

I would like to ask you a few questions about your plans for the future.

[interviewer note: For those respondents who are employed, continue with question 41.

For those respondents who are unemployed, skip to quc:tion 44.)

FOR THOSE PRESENTLY EMPLOYED:

41. What's the highest pay you expect to get on thejob you have ry. HR 42. What plans do you have for the next six months? (Check all those appropriate.)

1 Keep working at job/no particular plans 4 Go to school full-time

2 o Look for a better job 5 Join the armed forces

3 Go into another if-dining program 6 Other (specify)

(Skip tc question 46.)

43. What kind of a full-time job would you look for if you were to leave this one?

Write in job choice

PLEASE CONTINUE ON NEXT PAGE...

6
FOR THOSE PRESENTLY UNEMPLOYED:

44. What plans do you have for the'next six months? (Check all those appropriate.)

1, IS -Look-for-et job

2 Go into another training program

3 Go to school full-time

4 Join the armed forces

5 other

6 Nothing special (Probe: How do you spend your day?)

45. What kind of a full-time job would you look for if you were now.looking for a job?

Write in Job choice.

46. What is the least hourly pay you would take for that job? $

47. How much do you know about what it takes to do that job?

I know a lot about I know a few things about

that kind of job. what the job takes. . I don't really know

much about it.

48. What do you think is the highest hourly pay you could earn for the job? $

PER HR

rid I I PER HR 49. Since being in the program, has there been a change in the way you get along with your family?

I have no family. (Skip to question 52.)

I get along better with

them than I used to.

I get along worse with I get along with them

them than I used to. the same way I always did.

50. How does your family feei aboui how you did in the program?

They think I'm doing great. They think I'm getting by okay. They think I'm doing poorly.

51. Out of what you earn a week, about how much do you give to your family? $

52. How often do you save money from your pay? (Check only one.)

I don't earn any money.

I save something

each week.

53. Do you buy things on credit?

No, I don't use credit.

Sometimes I save, I never save

sometimes ! don't. any of my pay.

I buy some things I buy most everything

on credit. on credit.

PER WK

54. Since you left the program have you gone to a public Employment

Service Office (e.g., State or government employment service?) Yes No

54a. If yes, how many times (number of Visits)?

C'S
55. Which of these.people Or agencies have been giving you a hard time lately?

(Check as many as apply.)

1 Supervisor at work

2 People you work with

3 Social worker (Welfare)

4 Employment agencies

5 Program people (like the counselor)

6 The police or the courts

7 Neighbors

8 Lawyers

9 Credit collection outfits

10 Storekeepers

11 Somebody in your family

12 A hospital, or people who work In a

hospital (like the doctors; the clerks)

13 Friends

14 Others (specify):

56. Since you've been in this program have you gotten in at., trouble with the police: been arrested,

charged or booked? (Do not include status offenses, such as parking tickets or runaway.)

No

Just once A couple of times More than'a couple of times

57. How important is it to you to keep out of trouble with thepolice and the law?

Real important; I go out of I usually try to steer clear unless Not too important; if I get in

my way to avoid trouble. I'm pushed real hard. trouble I don't care too much.

58. We appreciate the help that you've given us. !% would like to keep up with how you're doing. We will

want to talk with you again in the next few months. At that time we'll pay you $10.00 for your help. Please

give us the names and addresses of two people who will know where to reach you during the next

few months.

Name

Address

Phone

Relationship

Name

Address

Phone

Relationship

s160
InteMewer Note: Ask this question only of respondents who answered question 11.

59. Wit hope to make future programs better by talking to employers who have hired youth from the program

you were In. We want to ask your present (or former) employer how you are (were) doing on your full-time

job. Can we ask these questions? (Show questions for Employers Rating Form and STRESS CONFIDENTIALITY).

Yes No

If yes, obtain job supervisor's name.

60. Additional Comments by Yoett

5609902 - L.3P30 - 200253

1 LI
. ETS I.D. NO.

Employer's Rating Form

o

-.\-....-

DATE

We're doing a study of young people formerly enrolled in a government sponsored Youth

Program. We would appreclateour giving a brief assessment of one of your employees (or

former employees) who was enrolled in one such program.

The individual we are interested in is

We have his (her) permission to ask you for this information.

. Your responses are completely confidential.

C

ASSESSMENT OF YOUTH. ALTERNATIVE PROJECT

1. Do you feel that this employoe puts in a decent day'swork?

Yes, definitely Generally; not always Definitely not

2. Does this employee get along with the other workers onthe Job?

Yes, definitely Generally; not always Definitely not

a Would you promote this employee?

Yes, definitely Maybe; not certain Definitely not

4. Would you hire this employee again?

0 Yes, definitely Maybe; with a few reservations Definitely not

5. Compared to other young people who work here, howwould you rank this employee? .t Clearly better than most About the same as most Clearly worse than most

I.

5609901 Y59928 200251

171

,
10/29/79

CONTROL GROUP. FOLLOW-UP SURVEY

co. This instrument is to be filled in by the interviewer,

NOT the respondent.

We're trying to find out how people have been doing since

our last 6ntact--like how things have been working out

for you in general; what you are doing now and what things

you would like to do from now on;, how you feel about jobs;

and so on. We would like te-know your feelings about these

things.

The answers you give to any questions will all be confidential and private.

Anything we find out from these interviews would be reported for a whole

group at a time (like a few hundred) so that no one will know your answers.

However,all the information requested is important and your response to

each 4iiestion will be appreciated.

Let me go over the questions with you. It should take no more than about

25 minutes. Remember, if you don't understand a question, please stop

me to make sure it's clear.

ASSESSMENT OF YOUTH ALTERNATIVES PROJECT

172
a

ASSESSMENT OF YOUTH ALTERNATIVES PROJECT

Follow -up Survey

NO OF PHONE ICA1.0 NO. OF HOME VISITS NO OF OTHER

CONTACTS ATTEMPtED

1. ETS I D. NO. I

2. DATE OF n [

1 1 1 1

INTERVIEW

MONTI I DAY YEAR

3. NAME OF RESPONDENT

4. HOME ADDRESS

hi,?..c1FR AND 5121:0

C.r`i AND STATE 5. TELEPHONE NUMBER WHERE RESPONDENT CAN BE REACHED 11

CODE

I 1

6. RESPONDENT'S SOCIAL SECURITY NUMBER

7. SEX . 0 Mole Female

ZIP CODE

I 1 TELEPHONE NUMBER _JI

8. DATE OF BIRTH !MIMI MONTH DAY AAR

9. Are yOu presently working full-time (30 or more hours per week)?

Yes. (Go on to question 10.)

No, but I have worked full-time

within the last three months.

(Go on to question 10, but ask

about most recent job.)

No, and I have not orked full-time.

within the last three months.

(Skip to question 33)

10. Whore do (did) you

(most recently) work? NAM' or COMPANY

NUMBER AND STREET OR ROUTE CITY AND STATE

1E10110Ni NuMIR R ( 1 11. What kind of business is (was) that?

12. What is (was) your job title?

13. How many hours a week do (did) you usually work?

14. I low :licitly woks hove you boor, (wore you) on riaT101).' ......

173

I 1

111,,ks

N.
15. How much per hour do (did) you earn at that job? O a .$ III I I COUARS

_ 16. is (Was) this a temporary CE1A job?

CENTS

PER HR.

..- yes No

17. Is (Was) that the only job you've had in. the last

three months?.... Yes No

(if yes skip to question 21)

18. How many jobs (full-time and part-time) have you had in

the last three months? 4 . JOBS

19. Why did you leave your last job?

0 1 quit. I was laid off. I was fired.

(

19 a. What wos the main reason for that? (probe) '

20. What was your starting hourly salary

on your first full-time job in the litst three months? $ 1 raj 11 PER HR. 21. When did you get that full-time job?

EADNIH DAY YEAR 22. Which one of the following sources was most helpful in getting you that full-time job?

(Check only one.) ,

1 School' counselor

2 Friends or people in neighbbrhood

3 Newspaper

4 Government (public) employment agency

5 Non-government (private) employmtnt agency

6 Family.

7 Help wanted sign

8 El Church or community leaders (e.g.. teachers. ministers. and so on)

9 Other (specify):

0 None of the above: I got it on my own. . 23.. In how many places did you aqtually 0 ,

fill out an application before you got that full-time job? 1 RACES24. In how many places did you actually get to hove on interview before you got that full-time job?

25. Have you gotten a raise on your present or last) full-time job 9 Yes No

. 25a. If yes: Number of raises

- 26. Did you get a promotion on that full-time job? ; Yes No

37. How well have you done on the job during the last three months?

Much better than I

thought I would do

Worse than I About what I

expected to do "7s. expected I would do

%

0

H(
gk

I would like to find out how you feel about your most recent full-time job.

28. Whof ore your feelings'obout the kind of work you do (did)?

It's gloat: I like it a lot. It's okay.

29. How.% your poy for the kind of work mld° (did)?

Good poy for the kind 0 Just about whot it

of job I havo (hod) \ should be foi the job

Don't like the work of all.

A lot less thon the

job is (wos).worth

30. If you hove your woy. would you wont to work for this ploce five years from now?

Definitely not. I'm hot-sure. Yes, I'm sure I would.

31. If you knew then whot you know now obout this jobwould you hove token the job?

O Nonot if I knew

whot I know now.

I'm not too sure. 0 Yes, I would toke

the job there ogoin.

32, When you finish o cloy's work, do (did) you feel like you did something worthwhile?

I olmost never feel

(felt) that way.

Sometimes I feel I olmost olwoys feel

(felt) that woy. (felt) that woy.

33. Have you received any unemployment compensation

payments during the last three months') Yes No

33a. If yes. how mvsh per week

hove you received/ S PER VA( 34. Have you done any part-time work for less than 30 hours

per week during the last three months?

If no. go onto question

34a . If yes. whot kind of work did you do on your lost job?

(Describe type of job )

34 b How much per hour did you eorn of thot job?

35]

El Yes No

Ii
I

Here are some

with.

ivit/O-c-, now

36 .IS your e

37 Which de

questions about educational or training actNities you might presently be involved

going to school or involved in any ether training activity? Yes No

(If no. skip to question 38.)

ucation or training program full- or part-tiMe?

Full-time 111 Part-time

ribes your education or training program? (Check all those that apply.)

1 A CETA training program , 2 Regular high schobi

3 Business /vocational, technical/trade high school

4 High school equivalency program (GED)

5 College

6 BusinessANcational technical postsecondary (not high school)

7 Union apprenticeihip program

8 Other (specify)

I would like to ask your a few questions about your plans for the future.

'17 (Interviewer note: For ttlose respondents who are employed, continue with question 38.

For thliose respondents who are unemployed, skip to 'question 41,1

FOR THOSE PRESENTLY EMPLOYED:

38..What's the highest pay you expect to get on the job you have now? S Li I PER HR 39 8 What plans do you htnie for the next six months? (Check all those'appropriate.)

1 Keep working at job/no particular plansi7 4 Li Go to school full-time

2 a Look for a better job 5 Join the armed forces

3 Go into a training program 6 Other (specify)

40. Who) kind of a full-time job would you look for if you were to leave this one?

WOO in job choice.

(Skip to question 43.)

ff

PLEASE CONTINUE ciN NEXTPAGE...
FOR THOSE PRESENTLY UNEMPLOYED:

41. What plans do you have for the next six months? (Cmeck all those appropriate.)

Look for a job

2 Go into another training program /

3 Go to school full-time

4 Join the armed forces

5 Other

6 NOthing special (Probe: How do vr,u spend your cloy))

42. What kind of a full-time job would you look for if you were now looking for o job?

Write in Job choice.

43. What is the least hourly pay you would take for that job?

44. How much'ao you know about what it takes to do that job?

I know.° lot obout

that kind Dt job.

I know o few things about I don't really know

what the job takes. much about it

SIel-45. What do you think is the highest hourly pay you could earn for the job? Th PER HR 4 6 Within the last three months, has there .b en a change in the way _you get along

with lyour family?

O.I hove no family. (Skip to question 49. )

t get along better with I get along wors with I get along with them

them than : u3ed to. them than I use la. the some way I always did.

t

47. Howdoes your family feel about how you ve been doing over the last three months?

Theykthink I'm doing great. They think I'm g tting by okay. They think I m doing poorly.

,---

46. Out of what you earn a week, about haw much do you give to your family? S ITT] KR wx

49. How often do you save money from your pay? (Check only one.)

I don't earn any money.

1 saveisamething

each week.

5 0. Da you.buy things on credit',

Sometimes I save, I never save

sometimes I don't. any of my pay.

No, I don t use credit. I buy some things 0 I buy most everything

on credit. on credit.

51, During the last three months, have you gone to a public Employment

Service Office (e.g., State or government employment service?) yes No

51a. If yes, haw mony times (number of visits)'

H
;0 52. Whictyf these people pr agencies hove been giving you a hard time lately?

(Check as many as tipply.)

1 0 Supe rvisor at work

2 0 'People you 'WOK with

3 0 Social worker (Welfare) , 4.0 Employment poOncisis

5 0 .School' Counseloi..

6 0 The police or the court;

7 0 Neighbor

8 0 Lawyers

9 0 Ciadit collection outfits

10 0 Storekeepers

11 0 Somebody in your fomily

12 0 A hospital. or people who work in

hospital (like the docrots: the clerks)

13 0 Friends

14. 0 Others (specify):

53. Within the last three months, hive you gotten in any trouble with the police;

been arrested, charged or booked? (to not include" status offenses, such as

parking tickets or runaway.)

O No

O Just once '0 A couple of times

. . 0 More thon o couple of times

4 54 . How important is it to you to keep out of trouble with 'the police and the low?

O Real important: I go out of 0 I usually try to steer clear unless c.1 Not too important: if I get in

my way to ovoid trouble. I'm pushed real hard. trouble I don't core too-much.

55. V oppreciote the help thot you've given us. Werwould like to keep up with how you're doing. 1Ne will

wont to talk with you (Nom in the next few months. At that time we'll pay you $10.00 for yotir help. Please

give us the names and oddresses of two people who will know where to reach you during the next

few months.

Nome

Address

Phone

Relationship

Nome

Address

Phone

Relationship

1 "I C. 4 ki

41.
Interviewer Note: Ask this question only of respondents who answered question 10.

56. We want to. ask your present (or former) employer how you are (were) doing

on your full-time job. Can we ask these questions? (Show questions for

Employer's ,Rating Form and STRESS CONFIDENTIALITY).

0 `;'es 0 No

It yes. obtainsiob superviso,'s norne

57. Additional Cor-lfrtfelitrby Youth:

.

17)

,.
Employer's Rating\ Form

ETS I.D. NO.

DATE

I 11

We're doing a study of-young people formerly enrolled In a governMent sponsored Youth

Program. We would appieclate your giving a brief assessment of one of your employees (or

former employees) who was enrolled in one such program.

The Individual we are Interested In is

We have his (her), permission to ask you for this Information.

Your responses are completely confidential.

ASSESSMENT OF YOUTH ALTERNATIVE PROJECT

1. Do you feel that-this employee puts in a decent day's work?

Yes, definitely eN Generdiv, not always

2. Does this employee get along with the other workers on the job?

yes,Flefinitely

3. Would you promote this employee?

- YeCdefinitely

Generally; not always

Maybe; not certain

4, Would you hire this employee again?

II 4es, definitely 0 Maybe; with a few reservations

5. iComparea to other young people who woi k here, how would you rank this employee?

Definitely not

Definitely not

Definitely not--

Definitely not

Cleaily better than most About the same as most Clearly worse than most

51509903Y54182010251
PROJECT AND PROCESS

INFORMATION

QUESTIONNAIRE

1. Demonstration Project Descriptors

Project Title:

Name and Type of

Project Operator

Name:

Project Location:

Initial Funding

Date:

Termination Date

of Present

Funding:

0

0

CETA-Prime Sponsor Staft

Other Local Government Agency or

Unit

Secondary Public School

Post-Secondary School or Other

Educational Institution

Private "For-Profit" Training

Organization

Private Nonprofit Community-Based

Organization Offering Predominantly

Employment and Training Services

Private Nonprofit.Communitv-BaSed

Organization Offering Wide Range of

Services (e.g., health, housing,

child care, senior citizens activities,

etc.)

State agency

Federal agency

Other

Termination Date

of Project:

Type of Project

In-School

In-School & Summer

Summer

Out -of- School

Mixed in School

Out of School

(.

181

D--e Questionnaire

Completed

Title person

.Completing

Questionnaire
Number of Planned Participants at Peak Point

Hunter of Planned Participants Currently

Planned Enrollment

Period for apical

Completer: No. Weeks

-Site

ChitFacteristics

Project Cycles

No. Hours Per Week

The geographical areat(1 mile radius)

surrounding the project location or

target area may best be described as:

Central City Poverty Area

-Central City Nonpoverty Area

Suburban Poverty Area

Suburban Nonpoverty Area

Rural Poverty Area

Rural Nonpoverty Area

The economy of the surrounding labor

market (approximately 1 hour commute

by automobile or public transportations

may best be classified according to its

aggregate unemployment rate as follows:

Excessive Unemployment 10 % or above

High Unemployment 7 % - 10%

Moderate Unemployment 5 % - 7%

Low Unemployment Below 5%

List dates cohorts of your

project begin and ended

6 Group Number of Youth Start Date Ead Date

1

2

3

4

5

6

7

8

9

.
2. Project Services Profile

Program Activities/

Services_

-

(1)

Check mark the

activities/services provided

directly by

your project

(2) Check mark

the activiti 'ser- vi pro- viand to

participants in

your project through

arrangements

with other

agencies

(3) Whit percentage of total

nonclerical

staff time is

spent in providing each

check marked

activity /service in

column 1

(percentages

must add to

100%)

(4) What percentase of participants receive services

or participates in activities

check marked

in either

column 1 or

2

(5) What percentage

of time does a

typical program

completer spend

in each checked

marked activity/

service in

column 1 (Percentage must add

to 100%)

1. Outreach/Recruitgent of Youth

. 2. Eligibility Certification

3. Testing/assessment and employability plan

development

4. Counseling for

career or personal problems

S. Other preemployment services including world-ofwork or basic

skills training,

job application .

and search assistance

Q ---

6. Vocational exploration in classrooe

or in worksite

visits, job rotation in training

capacity in multiple worksites

2

-

_.."

7. Classroom training in vocational

skills

8. Remedial education, GED, ESL

. - 9. On-the-job training

All. Work experience

1. Support services

(transportatior,

medical, child

care, etc.)

.

2. Placement and job

development

L3. Other In-Program

activities

. L4. Post-program

assistance (e.g.,

coaching, counseling and followup

.

1
Which of the following do you think are most critical in achieving

a successful outcome with youth in your program? (Rank them in

order from most important to least important, 1-7.)

Persdnar'and charaeter'development (motivation, self-image,

self-concept, Interpersonal skills, appropriate social

behavior, etc.).

information about jobs and employers (world-of-work orientation, occupational and labor mirEgi information, learning

to search for a job/complete employment application and

interview, etc.).

Work habits v_-,d attitules (acceptance of supervision/

authority, interpersonal relationships in work settings,

promptness in reportihg to and performing lob, etc.).

Training for specific job skills (OJT, classroom training,

job internships, etc.).

Basic educational skills (remedial education, GED, cognitive

skills, atc.).

Brokering placement (connecting youth with placement oppor=

Ealty - IdenaTillg opportunities whether at employers,

schools or military, arranging interviews for these opportunities, getting youth to the placement setting, etc.).

Fo1lowupl support 7nrtices after leaving orogram (job coaching/

counseling - continued contact with youtR and/or employer,

schgol personnel, etc.).

EMPHASIS

Little High;

1 2 3 4
3. Linkages,

Please desbribe as best possible the importance to the various dimensions

of the project of linkages and coordinative arrangements involved in

this project. This should include informal working relationships

formal written agreements as well as contractual relationship,

Design and

Planning

Administration

Recruiting/

Outreach

S Service

Delivery"

Placement

and

Followm_

0

.1.1 34 I

H

I-I2

tv

211 0..00 =

0

0V

kk

sa

ZH

-1Z

C

ASV

k

44am

H

g

r4 ei 0 4.1 44 ilk

ga

= 21-1

S

0Z

cleV

. 14a

E

H

H0

ty

440

H

0

apV

S

0Z

V

c

(0

.1.: i4a

EH

0

c H is 0 jj 41.1 yd.,

0 r4E

...41! S

0Z

cOH1.1 0 ij

hilt!

t! ,$414

N.t.- . E 0 ....ii

HI =MEN

0Z

CETA Prime,Sponuor I

\ CETA Private Industry Council

.

, Other Local Government Agency

or Unit

Other Federal Government Agency

or Unit

Local,PUblic School

.

. Post-Secondary

School

.

. Private-for-Profit Training

Institution

Local Nonprofit

Community-Based

Organization

Offering Employment and Training

Services -

1

Local Private-Nona

Profit Community

Based Organization

Offering Wide

Range of Services

(e.g., health,

housing, child care

WEHS iq e

, 1 _MRIII

National Community

Based Organizations

Local Organized

LaLnr

Private Employers

state'

or Organizations .

I

I

I

185
ot Please describe as best possible the types of linkage and

coordinative- arrangements which have at least minor importance to the

project. Exclude linkages judged no importance in tie previous table.

t

.

.

Design and

Planning. Administration

Recruiting/ Service

Outreach Delivery

placement

and

Followup

o

1J 4,0041 QQA041

a

o gk.Q kaglOk

$,T>,841t lAiwowc

se 40,.4 000

c-oceww

e>seams. ckC0..414COOM N0)0.014404

o-Au

1 0wo We.

...1C0004,1A4

c

0

.1Z.4

41),Z PtIvi cOA

I

944

.40S

0

5

'4: CO

001 C

.4mit

0X

0.°>4341t

;. c 4046.416V.r1 ww000>qco-lew-.4000co..4sw-w000_co.IEw

mme?eams.

040...INCOOM00.44PINC0000...10

V.u030

ouov,we

410.004.1

01-0-.4wo 0 0k41

$411.00$4

sAlvowc

16 a

t06401.44.1N4UMNW0.0.1441644U=Ww,Om4.1044

ov -.4 0

$4 es' ...1Ct7004J.40...1 a g

c

.0

.4 wzowc

.4

4

.40c

4

0

'w

4140.40

00.04>easms. $40...4=kC.1

t

tri C

.4mIs0

0411. >41.cspwc

w

o v 400 OW

e

A o

N wa.cow

aloof

ov0

AJ Cl04., w

w Yrt

m

g

0avt104J wzowc 4-....14

.4U0

4

c

'4

Q40.40

omos>ew.magem lap...14'44_1'Hk

w

at 0 vaiWg

C 0004-$.4 gg404, 0 Is 40kMA

0 aka) m wa.cow

ova. >A1A1v0vg 5 .4

i if t4

o 0g

Awo

8't

Co d E

"COOM

4

uoi.i..1

V

.tr11.4Z

,.4 4

.4UC

0

c

:44 Clc-" 040.44144

M

CAm

Is 0m

0)^1.01

I

0

'Qv W00 mWA

0-r-e4 414 Oka,

w two ? >,8

40

000

lo>40 aka 40A

(i)

o

4

.-4 _0 4

c

014

,

C

0

4V

oA

4

Ist

0.0C 04ZCVC V4-4 CO-4ew7w

xad

IC00.00 1-1 V

.

\..4 4 C

4

UC Is W 0 CM

'1 (0;4 >4 04J A tie

Q40

00,10 kOr4

014 4 g c4

CETA Prime Sponsor

CETA Private Industry Council

- ,

: --N, -

.

.

.

. Other Local Government Agency

or Unit ,

-r

.

- -

Other Federal Gov-

'ernment Agency:::---

or Unit

.

. . . Local Public School- r . Post-SecondaryScho.J1

..

.

, 4

7,

. Private-for-Profit Training

Institution

LocalNonprofit

Community-Based

Organization

Offering Employment and Training

Services

Local \Nnnprofit

Community-Based . Organiiation

offering\ide Range

of lervices\(e.g.,

health, houiing,

child care, Senior citizen activites

and the lihe)

.

.

c

.

National Community

Based Organizations

Local Organized

. Labor .

, Private Employers .. - . State Agencies

or Organizations "

1)(,()
'Total Wait Prnjibrr,

'Characteristics of Paid

Staff.

SEX

Male

Famiie

AGE

:::7)2545 ^35

36 , .

r 45

RACE

White

Non-White

4. Staff-Profile

Full-Time Professional

Staff (Nonclerical

Staff Who Work 30 or

MorCHours Per Week)

Part-Time Professional

Staff (Nonclerical Staff

Who Work Less Than 30

Hours Per Week)

No. Funded Positions No. Funded Position:

Filled' Filledi

No7, Positions W. PoSitions

Vacant Vacant

(Percentages Are Related to Staff in

Positions Filled)

Hispanic 1

EDUCATION

B.A. or B.S. Degree or more

M.A., M.S. or Ph.D..

Degree

EXPEEIENCE

Attended Similar Program

as an Enrollee

Worked as Staff in

Another Program Which

Served Youth. Similar

to Thole in This

Program

Experience in Youth

Programs

Less than 1 year

1-2 Years S 3-4 Years

S or More Years

'187

(Percentages Are Rel'ted to Staff in

Positions Filled)

S

4sr

S
Employed by the Program

Operator for

Less Than 1 Year

1-2 Years

3-4 Years

5 or More Years

Salary

s

Less Than $7,500 % % $7,500-$10,000

$10,000-$12,500, % % $12,500-$15,000 I % $15,000-$17,500 % % $170500-$20,000 % * $20,000-$25,000 % % $25,000 Plus % % Activities

Spend Less Than 25%

of Their Current Work

Week Directly Providing Services to Youth

- Spend between 25 to 50%

- Spend between 50 to 75%

- Spend more than 75%

Volunteer Assistance

Volunteers assisting

staff or participants

Houks Per Week

Averaged by

Volunteers

1 Qr.'

. .

3
S. Project Start-up and Stability

Start -up Time

Number of weeks between signing of contract

for the local project and first enrollment of

participants

Number of weeks between signing of contract and

filling at least 75 percent of budgeted program

slots for youth

Number cif weeks of operation since reaching

point where at least 75 percent of slots were

filled

Organization Experience

Experience of

organization

operating this

project with

employment'find

training programs

exclusively for

ecofiomically

'disadvantaged

youth 14-21

years old

Less than 1 year

1-2 years

3-4 years

5 or more years

Experience of

organization

operating project with employment and training

programs

4,

P

Experience as

recipient of

government (State,

local or Federal)

funds

.4. Project Background

This project represents C) Continuation of similar project

funded under other sources

0 Significant modification of services under previous youth project

0 Change in target group of

previous .youth -projectC) New activity and youth target group

for program operator

(:) Modification of program previously

serving adults; increased emphasis

on youth

Modification

During the first year of operation, to what extent was the project

modified either in terms of emphalis or types of services delivered?

Was not modified

Was slightly modified

Was significantly modified

To what extent are changes still being made in program design

and operation?

No change being made

slight changes being made

Moderate changes being made

Significant changes being made

180
The pFogFam as currently operating is

project already completed 6 unlikely to undergo any kodification in

likely to undergo slight modiftcations

constantly undergoing moderate changes

constantly undergoing major changes

What were reasons for modifications

No modifications

Funding change

Staff change

Performance problems

External Factors

Other

What kind of modifications occurred

No modifications

Ppogrampatic

Administrative

Other

Continuity

the immediate

How many iroject directors have there -13-ett since the

*enrollment

ti

1

2

34 or more

Number

professional staff

(nonclerical work

30 or more hours

per week)

Number part-time

pro'fes5ional staff

(nonclerical staff

who work less than

30 hours)

.0

At

On-Board Peak

NoW Staffing

1J0

Cumulative

Employed

Since

Project

Inception

\ future,

first

'Number - Individuals who

Have Worked With

Project Since

First Enrollment

/
nrollment Patterns

Monthly Average

Number Partici

pants On

"

i

!law

During

Terminations

During Month

Enrollees

Month

1978

January

February

March

April

May

June

,,July

August

Sept.

October

November

December ' 1979

I

.. January

tfebruary

March

April

. . May

. . Jutle . July

August

September , October

November

December

1980

I January

February . March I s April

May

June

July

August (-,

, September

>otober` . /

November---..:

. - /

(4 1 9 1

I December

t
0

6. Cost Analysis, o Total Expenditures to Date

CETA Youth project budget

CETA Title II D'or VI Public Service Employment

workers involved in project

Other funds used in...project

Estimated Value of In-Rind/Contributions to Date

From Any Source

i ' o nditures DUrir. Last Calen. ar Year--

CETA Youth project budget

Other CETA Title XX, D or VT. Pilblic Service Employment

Other funds used in project

Esti! ed Value of In-Kind Contributions During Last

Yo o Expellitures Per Month At Fall Operating Level

CET,: Youth project budget

Other CETA Title II D or VI Public Service Employment

Other funds used in project

Estimated Value of In-Kind Contribution Per Month at

run Operating Level

o Estimated Cost Breakdowns During Month of Full Operations

Administrative and clerical staff required

for operation of program

Administrative and clerical-staff required

to handle research and special reports

which are part of demonstration

Direct services delivery staff

Materials, supplies and equipment

Income support for participants

if Other 1 o Estimated Cost Breakdowns Since Start of 2roject

Administrative and clerical staff required

forioperation of program

Administrative-and-clerical staff required

to handle research ,and special reports

whiich are part oyemonstration

Direst services delivery staff

Materials, supplies and equipment

Income support for participants

Other

I
- 65 -

Appendii 2

Youth Demonstration Projects Utilizing

the SAS through

March-19.80___

More detailed information on the projects listed in this

appAndix is available in 71I4'heKnoleclTWTILamLIliktate

Development Activities, Fiscal 8-1

11 2,
Appendix 2

DEMONSTRATION:PROJECTS UTILIZING

STANDARDIZED ASSESSMENT SYSTEM

THROUGH MARCH 1980

ESTIMATED-

'SAS- YOUTH SAS

tODFe. PROJECT SERVED RESEARCH

NUMBER SITES NAME OF PROJECT* BY PROJECT AGENT

In-School

I-1 31 _Youth Career Development for_

School-to-Work Transition

Demonstration Project (YCD)

1-2 2

1-3 3

1-4 5

1

1-6 1

1-7 1

1-8 1

1-9 8

I-10 55

3400 Educational

Testing

Service (ETS)

Health:Opportunities/Positive 120 ETS

Entry Demonstration Project'(HOPS)

Youth Career Development for School- 300 ETS

to-Work Transition Demonstration

Project (Puerto'Rican Forum)

PUSH/EXCEL School-to-Work 250 ETS

Transition Demonstration Project

New Youth Initiatives in Apprentice- 150 ETS

ship Demonstration Project (BATRockford)

New Youth Initiatives in Apprenticeship Demonstration Project (BATDes Moines)

New Youth Initiatives in Apprenticeship Demonstration Project (BAT-RI)

Junior Achievement Adaptation

Demonstration Project

Jobs for Delaware Graduates

Demonstration Project (Delaware)

Exemplary In-School Youth Program

Demonstration Project (XEMP)

150

150

60

1497'

9200

ETS

ETS

ETS

ETS-JDG

Youthwork
,SAS #

CODE; PROJECT

L- NUMBER .''SITES

xt-off-School

OT 3

0-2' 1

0-3 5

-0-4 1

0-5 1

0-6

0-7

0-8

0-9

0-10

0-11

NAME OF PROJECT*

YOUTH

SERVED

BY PROJECT

Service Mix Alternatives

Demonstration Project (SHAD)

Corporate Career Demonstration

Project

Public vs. Private Sector Jobs

Demonstration-Project

GIANT STEP Youth DevelopmentDemonstration Project

Job Factory Job Search Assistance

Demonstration Project (Brandeis Univ.)

1 The Workshop Job Search Assistance

Demonstration Project (Brandeis Univ.)

13 Rural Youth and Housing Partnership

Demonstration Project

5_ Career Advancement Voucher

Development Project

1 Low-Head Dam Youth Employment

Demonstration Project (DAM)

21 Private Sector Initiatives Demonstration Project (PSI)

1 Alternative Program Strategies for

High Risk Youth Demonstration

Project(VERA)

0-12 2.

.0-13 5

Green Thumb Youth Demonstration Project

Youth Enterprises Demonstration

Project (CYE)

324

113

1520

600

750

600

500

60

SAS

RESEARCH

AGENT

ETS

ETS

St. Louis

Univ.

RTS

Brandeis

Univ.

Brandeis

Univ.

ETS

ESTIMATED

ENROLLMENT

Clark, Phipps,

& Harris

Michigan

State Univ.

1000 CPPV

1350 VERA

300 ETS

350 ETS

1 9 '7
'SAS

NUMBER

. PROJECT,

SITES NAME OF PROJECT*

'0-14 8

7

4

9

8

1

8

Ventures, in Community Improvement

Demonstration Project (VICI)

Summer-Career Exploration-Demonstration Project (Separate Projects at

OIC, RTP, and HRDI)

Summer Career_gxPlOration_Denonsamal.=_

tion Project (RTP)

Summer Career Exploratioi. Demonstration Project (SER)

Summer Science Student Demonstration

Project (Federal. Energy)

Theatrical Alcohol Awareness Summer

Demonstration Project

Impact Evaluation of Summer Youth

Employment Program (ALNA)

S-7 20 Upward Bound/CETA Summer Demonstration Project (Marquette)

5

7

Vocational Education/CETA Summer

Demonstration Project

Summer Careei Exploration Demonstration Project (HRDI)

YOUTH

SERVED

BY PROJECT

1235

1872-

0 SASRESEARCH ESTIMATED

AGENT ENROLLMENT

CPPV

OIC

1070

90

1000

797

400

300

ETS

Oak Ridge

Univ.

Multnoma-

.Wash. - CETA

A.L. Nellum

Assoc.

Marquette

Univ.

PA Dept. of Ed.

Ruttenberg,

Friedman

Killigan

Grotchess & Assoc.

Greenleigh Assoc.
Page 4

.SAS

CODE

NUMBER

a PROJECT

SITES NAME OF PROJECT*

YOUTH

SERVED

BY PROJECT

SAS

RESEARCH ESTIMATED

AGENT ENROLLMENT

S-10 3" Summer Career Exploration Demonstration Project (National Urban League)

226 ETS

-11 3 Summer Camp Vocational Exploration 2134 ETS

Demonstration Project (NFL-PA)

American-Camping-A-ssociation 33 -5 Hawkins &

Mixed Assoc.

M-1 16 , Vocational.Exploration Demonstration

Project (St. Louis NAB VEP)

3200 St. Louis

Univ.

M-2 7 Syracuse Assessment Demonstration 354 City of

Project Syracuse

M-3 1 Assembly Youth Employment Demonstration Project (Nat. Assoc. For Southern

32 ETS

Poor)

M-4 5 Mixed Income Demonstration Project 700 Mark Battle

Assoc.

M-5 7 Volunteer Assistance Demonstration 165 13S

Project (IBS)

*Descriptive information on these projects are available in The Knowlednowlede

Development Activities Fiscal 1978-1979 published by the Office of Youth Programs.

g

2 (1 1