Technical Report 594

Evaluation of the ASVAB 8/9/10

Clerical Composite for Predicting

Training School Performance

Mary M. Weltin and Beverly A. Popelka

(4J Selection and Classification Technical Area

.(_V) Manpower and Personnel Research Laboratory

E .LE,--F, 0

JUL 2 0 1984

U. S. Army

9 Research Institute for the Behavioral and Social Sciences

i October 1983

Approved for oublic release. distribution unlimited.

84 07 19 018
2- :

U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

p" "

L. NEALE COSBY

EDGAR M. JOHNSON Colonel, IN

Technical Director Cuinuander

Reviewers:

Lawrence M. Hanser

Richard Johnson

Karen Mitchell

Hilda Wing

Newell K. Eaton

Io

NOTICES

DISTRIBUTION: Primary distribution of this report has been made by ARI.

Please address correspondence concerning distribution of reports to: U.S.

Army Research Institute for the Behavioral and Social Sciences, ATTN:

PERI-POT, 5001 Eisenhower Avenue, Alexandria, Virginia 22333.

FINAL UISPOSITION: This report may be destroyed when it is no longer

needed. Please do not return it to the U.S. Army Research Institute for

the Behavioral and Social Sciences.

NOTE: The findings In this report are not to be construed as an official

Department of the Army position, unless so designated by other authorized

documents.

.1%'A-d*.'-.%Y %- ---

% J"
_--. UNCLASSIFIED

-. SECURITY CLASSIFICATION Of THIS PAGE (When Veto Ent*rd)

~ READ INSTRUCTIONS REPORT DOCUMENTATION PAGE BEFORE COMPLETING FORM

1. REPORT NUMUER 2. GOVT ACCESSION NO. I- RFCIPIENT'S CATALOG NUMMER

Tlechnical Report 594 :

4. TITLE (and SwbtItiO) S. TYPE OF REPORT A PERIOO COVEREO

.4q -EVALUATION OF ThE ASVAB 8/9/10 CLERICAL --

COMPOSITE FOR PREDICTING TRAINING SCHOOL _-

PERFORMANCE 6. PERFORMING ORG. REPORT NUMBER

7. AUTHOR(s) S1, CONTRACT OR GRANT NUMBER('&)

4..' Mary M. Weltin and Beverly A. Popelka

S. PERFORMING ORGANI ZATION NAME AND ADDRESS I0. PROGRAM ELEMENT. PROJECT. TASK

A REA A ORK. UNIT NUMBERS

U.S. Army Research institute for the Behavioral

and Social Sciences 63731A 310256

5001 Eisenhower Avenue, Alexandria, VA 22333 2Q263731A792

II. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT GATE

Deputy Chief of Staff for Personnel October 1983 13. NUMBER OFPAGES Washington, DC 20310 2828"'

14. MONITORING AGENCy" NAME & AOORISS(It differnt froe Controlling Oipfce) IS. SECURITY CLASS. (of this report

)

UNCLASSIFIED

IS.. OECL ASSI FI CATION/DOWN GRADOING 0

SCIHEOULE

"* 16. OISTRIBUTION STATEMENT (of Adt Revort)

Approved for public release; distribution unlimited.

m, %

"I1. OISTRISIUTION STATEMENT (of the •lbstract eiteredin Block 20. It d/flfoitt from Report) .

1S. SUPPLEMENTARY NOTES

IS. KEY WORDS (Contlnau, on •everse slde it :o aid Iddm'tltp w block atsm'br)

Selection and classification testing

Armed Services Vocational Aptitude Battery (ASVAB)

"2&. A£ISTACT r Me ,oe OW N ueumee"W "d NOmN1' *y block number) "The composite of Armed Services Vocational Aptitude Battery (ASVAB) subtests used to select applicants for entry-level training in Army clerical

schools was evaluated by correlating composite scores with training performance scores. The clerical composite (CL) had high validity/4• =.68)ffor

thi.j criterion but an alternate composite of Arithmetic Reasoning, Paragraph

Comprehension, and Mathematics Knowledge scorespr uced from multiple regression analyses had even higher validity (r = .74). Differential prediction for classification purposes is discussed.

SA 72 3 UNCLASSIFIED

i SECJnMIT'y €.ASSCATION OF THIS PAGE (Sb.. Data Entered)

'N Z.

%
Technical Report 594

Evaluation of the ASVAB 8/9/10

Clerical Composite for Predicting Training School Performance

Mary M. Weltin and Beverly A. Popelka

Submitted by

Newell K. Eaton, Chief

Selection and Classification Technical Area

Approved as technically adequate

and submitted for publication by

Joyce L. Shields, Director

Manpower and Personnel Research

Laboratory

U.S. ARMY RESEARCH INSTITUTE FOR THE BEHAVIORAL AND SOCIAL SCIENCES

5001 Eisenhower Avenue, Alexandria, Virginia 22333

Office, Deputy Chief of Staff for Penonnel

Department of the Army

October 1983

Army Project Number Manpower and Personnel

--206731AT92

App~raed for public relese; distributio. unlimited.

-- '- •'"iii
%*1

ARI Research Reports and Technical Reports are intended for sponsors of

R&D tasks and for other research and military agencies. Any findings ready

for implementation at the time of publication are presented in the last part

of the Brief. Upon completion of a major phase of the task, formal recommendations for official action normally are conveyed to appropriate military

agencies by briefing or Disposition Form.

i.J"

IN.

Ie

iv

.• "• • .• . . . .,'• , .•, , .. '. , ... , '. '. ...*, * ,.*., . -.* , -. \ . *, -... . • . ,, . ...

ts a. %
. FOREWORD

i'C

The Army Research Institute has ongoing responsibility for monitoring

the validity and utility of the Armed Services Vocational Aptitude Battery

(ASVAB) Aptitude Area Composites used to select and place applicants in particular training Military Occupational Specialties (MOSs). With the diminishing available human resources and increasing technological aptitude requirements of Army duties, review of enlistment standards has become essential

to manning a prepared defense force.

The present Technical Report is provided in response to a request f com

the Deputy Chief of Staff for Personnel (DCSPER) to evaluate the aptitude

area composite (clerical [CL]) used to place Army applicants into clerical

MOS training. The results suggest that the addition of a mathematics component to the composite could improve prediction of training performance.

The immediate purpose of this report is to document advice provided to the

2 DCSPER regarding potential improvements in the use of classification tests

in order to increase training performance and to reduce attrition in Army

schools.

SL

N EDGAR M. JOSOtN

Technical Director

1~1

StSO

,*00

T4!

v

.' %z .,•............... ' -S
SEVALUATION OF THE ASVAB 8/9/10 CLERICAL COMPOSITE FOR PREDICTING

TRAINING SCHOOL PERFORMANCE

EXECUTIVE SUMMARY

Requirement:

To evaluate the composite of ASVAB subtests currently used to place Army

applicants into clerical training schools.

Procedure:

Correlation and multiple regression analyses were used to produce combinations of subtests most predictive of higher quality training performance.

Comparisons were made between the multiple R for this optimal set of predictors and that for the composite of subtests currently used for clerical

'classification.

Findings:

In 9 of the 12 MOS samples studied, a mathematics suhtest (either Arithmetic Reasoning or Mathematics Knowledge) had the highest single-predictor

validity correlation with course grade criteria. When the MOS subsamples

were combined, a composite of Arithmetic Reasoning, Paragraph Comprehension,

and Mathematics Knowledge test scores had the highest multiple R (.74) with

the criterion, significantly higher than the correlation obtained for the

composite of subtests currently used (.68).

Utilization of Findings:

This report will be used by the Deputy Chief of Staff for Personnel to

consider modifications of the clerical aptitude area composite (CL) that

would raise the expected performance of soldiers in clerical occupations.

vii

V %, % r. % .'. . % %- r • . % -... -. .-.4 .'* * ..

A-. 0_ %4' .= 
EVALUATION OF THE ASVAB 8/9/10 CLERICAL COMPOSITE FOR PREDICTING

TRAINING SCHOOL PERFORMANCE

CONTENTS

Page

OVERVIEW ............................. ................................ 1

BACKGROUND . . . . . . ....... ................. 1

METHOD ............................... ................................ 5

Subjects . . . . . . . . . . . . . . . . . .. . . . . . . . . . . 5

Measures . . . . . . ..... . .............. ... 5

Procedures ................. .................................. 5

Analyses ........................................... 7

RESULTS . . . . . ........................... 8

DISCUSSION................. . . . ......... . . . . . . . . . . 12

REFERENCES . . . . . . ........................... 15

APPENDI XES

Appendix A. Sample ASVAB Questions ........... ................ .. A-i

Appendix B. Description of Job Duties for Sample Clerical MOS . . . B-1

Appendix C. Intercorrelations Between ASVAB Subtests for Profile

Study Sauile .. ........................................ C-1

LIST OF TABLES

Table 1. Description of the subtests in ASVAB 8/9/10 ...... ......... 2

2. Sunary of military validation research on clerical MOS . 3

3. Sapple clerical OS6 predictor and criterion scores .... ...... 6

4. Correlation and regression analyses .......... . 9

5. Failure rate by OS . . . ............. . . . . . 12

LIST OF FIGURES

Figure 1. Regression lines for AR + PC + 1.K .......... . . . . . 11

2. Fzequency percent distribution of AR + PC + XK .. ....... .. 13

ix

•'•:•.".•'5" % " • '• • •" •" "% •.• "-.% "• ". " • • -~ *.*%•'.% . ". '. "•• "•- " "
NIS

EVALUATION OF TIHE ASVAB 8/9/10 CLERICAL COMPOSITE FOR PREDICTING

TRATNING SCHOOL PERFORMANCE

OVERVIEW

This report documents test validation research undertaken by the U.S.

Army Research Institute (ARI) at the direction of the Army Deputy Chief of

Staff for Personnel. The research was designed to evaluate the effectiveness

of the clerical aptitude area composite (CL) currently used to classify soldiers entering clerical Military Occupational Specialties (MOSs). The clerical aptitude area is a composite of subtests appearing on the Armed Services

Vocational Aptitude Battery (ASVAB) forms 8/9/10.

The ASVAB is a battery of 10 subtests (see Table 1) that are combined

in different ways for use as selection and classification composites. One

set of four subtests is known as the Armed Forces Qualification Test (AFQT)

and is used to screen applicants for eligibility for military service.

After meeting the AFQT criterion for selection, an Army applicant must

achieve a passing score on an aptitude area composite for placement into

a particular uraining specialty.

For clerical training, the Army uses the clerical composite (CL), which

consists of the four subtests of Word Knowledge (WK), Paragraph Comprehension

(PC), Numerical Operations (NO), and Coding Speed (CS). Sample items for each

of the 10 ASVAB subtests are given in Appendix A.

Recent emphasis in Army training has been on criterion-referenced testing,

which has produced dichotomous pass/fail scores (Maier, 1981). Such scores

do not display adequate variance among individuals to reflect differences in

levels of performance. The training criteria used in this report were part

of a special data collection undertaken to obtain continuous criterion scores

suitable for validation.

The purposes of this research were (1) to validate the operational clerical composite of ASVAB 8/9/10 subtests against school performance in Army

entry-level enlisted clerical MOSs, and (2) to identify and evaluate possible

alternative composites that can predict clerical training school performance.

BACKGROUND

The current CL composite has evolved from a series of research pu)jects

on enlisted classification conducted since World War II. Table 2 presents a

summary of relevant validation research on the clerical MOS. The earliest

composite used to classify Army clerical positions consisted of a reading

vocabulary test, an ari-thmetic reasoning test, and a coding speed test. This

reflected job analysis (Trump: tarion, & Karcher, 1957), which found that

perceptual speed, number facility, memory, and perceptual patterning were the

important components in clerical jobs.

At the beginning of World War II, a general ability construct was considered most predictive of performance in training (PRS 808, 1947). Later,

1-.

i• • 'A' • • ,' • *• :• . - " " *~-" -'.<' -"-,• • .. -'.: •"-.:-." "-. --. ".-... .,

•% • -• " " - " v -? : • ÷• •-•'--•" . -'- "--'" .' " --.. '- .-'-b '-" "-•- "-'.- '-.
Table 1

Description of the Subtests in ASVAB 8/'9/10

Number Test

of time

Subtest name Content items (mill.)

,5

Word Knowledge (WK)a Understanding the meaning 35 11

of words, i.e., vocabulary

Arithmetic Reasoning (AR) a Word problems emphasizing 30 3b

mathematical reasoning

rather than mathematical

knowledge

a Paragraph Comprehension (PC) Understandinq the meaning 15 13

of paragraphs

Numerical Operations (NO) A speeded test of four 50 3

arithmetic operations

addition, subtraction,

multiplication, and

division. '".

General Science (GS) Knowledge and biological of the sciences physical 25 11 !

Electronics Information (EI) Knowledge of electronics 25 24

and radio principles

Mathematics Knowledge (MK) Knowledge of algebra, 25 24

geometry, and fractions

Auto-Shop Information (AS) Knowledge of auto mechan- 25 11

ics, shop practices, and

tool functions

Coding Speed (CS) A speeded test of matching 84 7

words and numbers

5%I Mechanical Comprehension (MC) Understanding mechanical 25 19

principles such as gears,

levers, and pulleys

aTests in Armed Forces Qualification Test (AFQT).

"2
'I Table 2

Summary of Military Vldto RearhnClerical MOS

_es Validb tio Researc on .

riein(dots usd m 4 1tu C,'t IpeItknl I... t¶1- )e1 ar'.) Iv Iieni,r

Training c-wowes ACLI A, -T .i'ý .)k (oiN i)44

(1940-1449) KV.AAR-NCS .45 ia.?. W)

knoeL tourse grades Acm-a, %qV#AP'ACS .71 Clerk W-.'4 6 94wu.Jw j'JSS)

(1949-1956) (11/4)

Fnael course grades AcII-St, PV.A"kcS 'SH Stenoi.)i)WI t')

.1t, Pers. M'Int. 11,11.1

.72 Admhan. (HI))

t .. 70 Akcctg. 03)H)

a" *.72 Postal (56H4)

Final course grades LCD-SI ACS5 lrxi l.% (14bfl)

Averawj writtull VE.k:.' At1 j No1 c-t' (1-4, I)j~I

'1⁄2test score Maximum validity .95 (1,043;

Perforoance ratinga VI .76 livlmm U90b)

6prectiLcus 1A* aspects AN .64

VEC4ACS .92

Ti.'ing test Ago Cz; .45 isnerml Q;lezk Franktsldt (147d)

(Isbl-1972) ACS (total) .35 7GAIO

Nustet reverie) .22 140)1

Coding .37

Final course crades Acid)3 CL . AR+WK.AD.CA .68 Clerical Mater 6- luch. (191.1I

(l1173). Maxanum validity .76 (25.0001

Final course grades ASVADI I Adsunastraitivo .BC. Cieri-si Vito..., Mullini &

(1966) Index 'Air Frce~) droll (1973)

(1.776)

Time-to-train LAySV 6/7 NECAD - .27 clerical t141) Swtanson (1979)

4 %6-lAý o) NK+AL"NQ - .16 ((lavy)

4o, nal couavat grades ASVAB 0/9/10 "K .55 Clerical Maisr & Gtatton

*4,(lISO-prseanti AN .53 1 V4) (1981)

No .51

'CM.: (K .50

I V3+CS*No .55

Final course grades ASVAD 6/9/10 CL K* lCSnbo .63 73C (188) Weqnei . Cirs~eyt c

.44 '55 (162) Means (1982)

Tnf to cowlete -. 46 %5 (162)

A=C - firay General Classification Test AR Arathsetic Reasoning VE - Verbal

A CS - Army ý:les.ificaition battery ACS - Arwy Coding Speed No0 - N'.oeria's OvErations

AQS - Aauy gualificetinn flattery AD - Attenktion-tn-Detail MX - Natheilvtics Knowledyc

RV - Reading ant Vocabulary CA - Ikiterest Inventory (Attentiveness) CS - codiny Speed

3
as the available labor force pool decreased, test developers attempted to

design composites that would tap aptitudes (such as perceptual speed) that

would be relevant to specific occupational skills.

"4? When the aptitude area system was reconstituted in 1958, each composite

contained only two tests, one measuring general ability and one measuring a

specialized aptitude. For clerical positions, the composite was Verbal

(VE) + Army Coding Speed (ACS) (Maier & Fuchs, ]969). When the composites

were again reevaluated in 1969, additional tests were added to improve the

validity of the composites, again making their content more general (Maier &

Fuchs, 1972). The testing philosophy of this time reflected a desire to

distribute general mental ability equitably across all the MOS groups and

to use the classification composites as a secondary screen for applicants

of marginal ability.

Although the composition of the subtests and their relative weighting

A% in the clerical composite varied over the years, the same aptitude dimensions

(verbal, math, clerical speed) were included until about 1973. At that time,

a new classification system was inaugurated. For the new battery, the Army

Classification Battery-1973 (ACB-73), the aptitude area composites were constructed to have maximum absolute validity for prodictiag training performance. Each composite had at least one test of general ability; at least

. two of the subtests in each composite required the ability to read. However,

thi cost of building literacy into each composite was that the composites

were highly intercorrelated.

The same aptitude composites developed for ACB-73 were also used for

ASVAB 6/7, introduced in 1976. For the current forms (ASVAB 8/9/10), Maier

and Grafton (1981) built new aptitude area composites by using parallel predictor subtests from ASVAB 6/7 and validating against measures of job proficiency and training succeis.

Research on clerical jobs in the civilian sector has found results simi- lar to those from research in the military. In the Federal Civil Service,

for example, clerical workers have been selected using a test that included

vocabulary, paragraph comp.rehensiun, alphabetizing, simple mathematics, and

typing scales (U.S. Civil Service Commission, 1973). In an early review,

Bennett and Cruikshank (1949) concluded that the best predictor for clerical

occupations was a test of general mental ability in combination with a test

of perceptual speed. Validation research in the clerical area has 9enerally

found validity coefficients between such test scores and training/job success

to be quite high. Ghiselli (1966) reported that test validities for proficiency and training criteria in clerical occupations were on the order of

.75 across test types. Pearlman (1979) collected nearly 3,400 validity coefficients from 700 studies involving a total sample size of over 470,000.

For training performance criteria, he found higher corrected mean validities

for measures of verbal ability (r = .65) , quantitative ability (r = .71),

reasoning ability (r - .4U), memory (r - .59), and perceptual speed (r a .40)

than for such measures as motor ability (r = .35) and spatial ability

(r - .38). This may be due to the relatively academic orientation of most

clerical training programs.

The impetus for the current validation effort stems from recent emphasis on optimizing the use of the Army's personnel resources. Because of the

4

% %% J .¶ • • t I ," .. %. 4 . * ,' ".- .2. . .. ,,-.
large number of clerical soldiers the Army trains each year (18,000 in Fiscai Year 1981 [FY81]), even modest increments in validity can have a dramatic

payoff in monetary savings attributable to higher performance and lower

"training attrition costs.

METHOD

Subjects

., .Subjects were 3,978 new trainees entering the Army for clerical training S in FY 81. Only subjects having ASVAB 8/9/10 scores on record were used because previous forms of the ASVAB were not being validated in this research.

Twelve of the 20 MOSs that use CL for classification were represented in this

sample; the remaining 8 specialties had to be excluded because of either

small sample size or inadequate variance in the criterion measures. Table 3

describes the 12 MOS subsamples and shows sample sizes as well as predictor

and criterion means and standard deviations (SDs).

Measures

Predictors. Predictor measures were the 10 ASVAE subtest scores, which

were transformed to standard scores using a standard Army conversion table.

Table 1 provides a description of these subtests.

Criteria. End-of-course grades were used to measure training performance. These scores were standardized within each MOS to permit comparison

across MOSs. Disposition categories, such as graduates, recycles (who retake

the same course), transfers (who attempt training in a different course), and

failures, were used to set replacement scores as defined below. The graduate/

nongraduate dichotomy was used as a secondary performance criterion for

validation.

Procedures

Data were edited to remove certain cases: scores for students who did

not complete training for medical or disciplinary reasons, scores that were

out of range, and repeated measures on students who recycled through training.

A score replacement technique, following a rationale developed by Maier

(1968), was used to reduce the error variance in criterion scores. Since

the score of record for a student who failed to complete the course may have

represented only a partial score achieved up to that point in training,

scores for recycles, transfers, and reliefs were replaced. A recycle is a

student who attempts training in the same course a second time; a transfer

is a student who is moved to a different course of training; and a relief is

a student who is discharged without further attempts at training.

For recycles, a score that was one-half standard deviation below the

mean of the graduates was substituted for the score of record. For reliefs

and transfers, a score that was one standard deviation below the mean was

substituted. About 5% of the total sample was involved in a substitution

of this type.

5

% ~%\.\:* : 55 ..

- r * *~.,*j**% 5 t -. j .
o0n w m 0 0 %.0 -4 N OD '0 '0 WLA mH .H - 4 -4 r

14

00

0) 0 ) V

W 4~J 4J 4 U)

0 ~~0 0)44ý 0 034

-Hq 0. 04 A4 P44 a4 4 04 0 $4 0 -A-4 0$ 04

$4 U C u C C 00 4- W 00U

0) 1 U 4-) 1t

4) 0 0 a) 0 4-4 U) CU)V 1 ti4.4

(a m tn U) U) to to U) o ) 0 0)4-JO0 0 to $4 4 $4 $4 4 $4 $4 $ 1- 14-) 0 r 4J I41 $4

o :1 =$ : Z3 'aOto $4 -Ht U rOw( Z

o 0 0 0 0 0 0 0 0 w a00 cO 0

o LC) U U C) 0 4Ju rz4- 4

LA Ln 'U enn O m O LA) Ln LA 3%

can 0 0; s a: a' 0ý C; 0; N

U'N r- N 00 C0 ' 0)' %nD .

0 -0 0 0 0 0 0 C (n C a% a%

U) 02S 0 UlA LA LA LA LA LA) LA Ln LA 0

4.1- n (Y) O - 0) LA (3) C% '0 LA 0 C

0

00

$P4 M 0 O m -H q 0 r ) -

O4 L $4I CN q CN ( ~

ý4 0

'0 to4to t 0) 041"4 - 0 V C 'iH 4si >

01 4H 40 ) 4J 4.1

U)' 'U U) U 1 n V Cr. t 02 $ 0 $4w 0 tor4.) 0) . ,4 6 . 4. 4. 4.1 04 Z. 004 04.

0 to U) U40 04 torz z1 

4J -4d -u-f -H 0) -I 0.s-vi 0-- C ' 0)s E-4 0 s- r-i 0 CO -I U) - (AC Cr4 r-4 (A 4J4-i)t o 0 - o.0-H1 ri 0 C4 ACA) CO QO CO 5.41 HC-14 U - -I rH4I-I C) O-.4 0r-4-I 0-H 0-H 04W 0-H $41 (L)-

$4 0040 U WU- Q d 0 to Uo M toM)Uto ,-H M .,8U 00 $4 0

0 '' 0 0 4) $0 $4m $ 0 04V0) 4I-iU 4J 0) s-) 0 C 04 $404 -H 0)04040404)04004 01V 0)04 000)04 o ow u &4 a.t Y) Pto oiw P4 tn 134o m~ w w So (

E-4 f4

OS Ur 0 0 ~ U ' 04% -4Z e A LA LA U) t to % o 
Hý-ing standardized and edited the scores, we then computed covariance

matrices and corrected the subtest-course grade correlatiuns for restriction

in range of the predictor scores using a method described by Lord and Novick

(1968). For an unrestricted population reference, the correlation matrix

derived from the 1980 Profile of American Youth (Wagner, 1982) was used (see

Appendix C). No other corrections were made.

A final procedural decision was how best to combine the information

from the 12 individual MOS regressions. Two methods of weighting the subsamples were compared: one weighted the 12 MOSs equally; the other weighted

by the MOS sample size, which reflected MOS accession size.

Analyses

Descriptive statistics (means, standard deviations) for each MOS were

calculated for the predictor (CL) and criterion (course grade). Frequency

distributions were examined for normality, skew, and kurtosis. Multiple

correlation and regression analyses were performed for each MOS and for the

total combined sample to predict course grades from ASVAB subtest scores. The corrected covariance matrices were used to derive regression equations

using stepwise procedures (Hull & Nie, 1981). The multiple correlation between the current composite (WK + PC + CS + NO) and criterion scores was

compared with that for an alternate unit-weighted, revised composite sug- gested by the regression analyses. The stability of the regressions was tested by dividing the two largest MOSs in half randomly and developing a

prediction equation on one-half of the sample and cross-validating on the

other half.

Differential prediction by sex was examined by comparing the slopes,

intercepts, and standard errors of estimate of regression lines developed

separately for males and females. These lines were computed from a unitweighted composite of predictors--Arithmetic Reasoning (AR), Paragraph Comprehension (PC), and Mathematics Knowledge (MK)--regressed against course

grade. Subgroup analyses by race were not performed because of ambiguity

in the coding of racial categories. SFor'the pass/fail criterion, simple chi-square analyses were performed

to evaluate the predictors' ability to separate students who successfully complete training from those who do not.

Utility analyses were modeled after Brogden (1946) and Cronbach and

Gleser (1965) following an equation developed by Hunter and Schmidt (1982)

for computing the potential utility of a selector by means of a continuous

criterion. Here, the value of an increase in performance (AU) due to use

of a valid test is a function of the number selected (N), the validity coefficient (R )y, the average standard score on the test for those selected

(aZ), and thir-standard deviation of criterion performance in dollars (SD

(Hunter & Schmidt, 1982). It is assumed that the relationship between the

test scores and training performance is linear.

The total productivity gain is

U =N R SD Z.

7

.~~.'.\A 0 -. %s.. s-f:1
RESULTS

Inspection of the frequency distributions of standardized course grades

revealed that, as might be expected in criterion-referenced testing, the

grade distributions were negatively skewed. This skew may be a result of

the schools' use of criteria that maximized the proportion of students who

successfully completed training. Because no attempt was made to normalize

the distributions, the obtained validities may underestimate the true

relationships.

Table 4 presents the results of the correlation, regression analyses,

and composite comparisons. All coefficients reported have been corrected

for restriction in range, Part A of Table 4 lists the correlation between

each of the individual subtests and the course grade criterion for the designated MOS. The highest single predictor validities were observed for the AR

and MK subtests. For the total sample, AR alone predicted training school

course grades as efficiently (r = .69) as the four subtests of the current

composite (r = .68). Lowest subtest validities were found for Auto-Shop

Information (AS) and Electronics Information (EI).

Part B of Table 4 displays the results of the multiple regressions, using the Statistical Package for Social Sciences (SPSS) forward regression

method. For example, for MOS 71D, WK correlated .49 with course grade; next

MK entered the regression, increasing the multiple correlation to .52, followed by AS, and so forth. The multicollinearity of ASVAB subtests makes

interpretation of the regressions problematical. However, a common result

across 9 of the 12 individual MOS regressions was that a mathematics (either

AR or MK) subtest consistently accounted for the most variance in course

grades. Simply adding AR to the current composite raised its validity from

.68 to .73.

In an effort to locate the best composite for all 12 MOSs, we combined

the results of the individual MOS validations. Since the relative importance

of the individual MOS was not determined a priori, two weighting methods were

compared for pooling data across MOSs: weighting MOSs by sample size and

weighting the MOSs equally. Results of both of the pooled-sample regressions

(SPSS stepwise method) were quite similar for either method of weighting MOS

subsamples. A composite of three subtests--AR, PC, and MK--predicted approximately 50% of the variance in course grades, with multiple R = .74.

This multiple correlation depends on equalizing the means of the 12 MOSs

and using beta-weighted predictors in the regression equation. By comparison, the subtests of the current composite (WK, PC, CS, NO) correlated significantly (P < .01) lower (.68) with course grades.

Using the utility formula, AU = NR SDy 1, the savings attributable

to using an improved CL composite can be estiAIatid by substituting

R = .68 (for existing CL composite)

R = .74 (for improved composite)

SD = $1,000; = 1 1; N = 18,000

where N is the number of clerical accessions in FY 81. The value of per- formance S_• was not determined empirically but was inferred to be $1,000, based on estimates of training performance in similar clerical positions

8

.. - %
6n M 0 r oL a 4N en-.-f -. 

30 %DAnLm alV n ýo0N U CO- 4

cn

\QC'4~~ fU uLA A

:7 Vz M4~ A Ln r 0 0 Dr-O c0 

w c

CN0 W qqe O0 f)10Ný TC

Lna %D% L D l. 10le L - -0

Lre, CJMCN

0IU w

m~LIA~If r-M0r D-4m00m ULA U) 1-n0% 

F, In

WC

U) to N~~- H W .-4M 4.) r0 -1 0. 00W D% 4ý : 0 NN - rn N-4 4.1 -T -T- nf)N11L nI n n U n

U)~~ 0 H .

U)t) 0 I

N - 4J _) u

C) U2)

-1 0 %0 1-0 %0ý D% % q L-0

19z -10 U) I 0rAAA 

0 U) C) u0

IA ) 0 0-

en C) r-0 wr 0 1 ()N D HI N L00r 

00

4)0) 0 4.)

9:E -1 4J C:

0 04 LI

-1 -4 *-4

M, 14 1-i 4-1 (

EU 0U .44 M- +I 4

1-4 m) 4)

E4 8 ZI 1EUn

9
(Hunter & Schmidt, 1982). Assuming selection is made at .3 SD above the

mean (Z = .3, • = .381, p = .382, = 1 1) and projecting these figures on

future clerical accessions, the potential savings to the Army for increasing validity from the .68 of the current composite to .74 for the new composite is more than $1 million each year the improved composite is used.

Operationally, unit-weighted composites have generally been preferred

to the less stable beta-weighted composite scores. To evaluate the prediction using unit-weighted subtests, the correlation between obtained course

grades and those predicted from a unit-weighted composite of AR, PC, and MK

were compared to similar correlations using the CL subtests (WK, PC, CS, and

NO). These correlations are listed in the lower section of Table 4 to show

the composites' relative predictive efficiency across the 12 MOS samples.

MOS 75B, for example, had the lowest multiple correlation coefficient; 76J

had the highest. The revised composite improved prediction in 7 of the 12

MOSs but did not significantly alter the rank order of the multiple correlation coefficients of the 12 specialties. Cross-validation in two largesample MOSs obtained corrected correlations of .52 for 76C and .68 for 76P.

The check for differential prediction by sex revealed that, at lower

composite scores, the regression line of a unit-weighted composite of AR +

PC + MK against course grade for females lay above the regression line for

males (see Figure 1), suggesting possible underprediction for women scoring

below the composite mean. The following male and female subgroup statistics

were significantly different (p < .01): validities (rjrm = .396, rf = .316),

predictor SDs (am = 10.1, Of = 9.08) and criterion SDs-(Gm = 19.9-S, Of

19.68). The difTerence in Etandard errors of estimate of-the two groups tof = 18.68; em = 18.29) surpassed the chance level (F = 1.043, df = 3,145,835,

p 7 .01), perhcIps because of the large sample size. However, the two regression lines did not differ significantly in slopes or intercepts. Stepwise

regression for females (n = 836) produced the predictors MK, AR, and NO for

a corrected multiple R of .40, while the regression for males (n = 3,147)

produced AR, AS, and MK for a multiple R of .42. Two of the subtests, AR

and MK, are common to the regressions for both sexes. Differences in the

third predictor as well as the absence of PC may be attributable to subtest

multicollinearity.

The second performance criterion, the dichotomous pass/fail separation,

was analyzed using chi-square tests. Schools were found to vary greatly in

their attrition rates (see Table 5). Some reported virtually no attrition

(76W, 73C) while others (75D, 75E) had more than 20% attrition. Schools

also differed in disposition methods, for example, whether a student having

been unsuccessful in completing course requirements recycled through the

same course or attempted other training.

To evaluate whether composite scores could distinguish between students

who complete training successfully and those who fail, composite score distributions for graduates and nongraduates were compared. Here, graduates

include both regular and accelerated graduates, and nongraduates include

Pany students who recycled, transferred, or were relieved from duty. Using

the current CL composite subtests, significant differences between graduates I and nongraduates were found for two MOSs: 71D (X2 = 17.4, k < .003) and 75E

(X2 = 12.7, p < .03). When an alternative composite of AR, PC, and MK was

used, significant differences were found for 5 of the 12 MOSs: 71D

I Si10

I
0

* -4

.44

IDot - I

eepoiD ovnoo pzipjpu*4

M .5w. ,0

a Y, , 4
(X2 22.8, • < .0001), 71M (X2 - 2 6 .

6k p < .0001), 75D (X2 = 12.5, p < .01),

* 75E (X2 n 28.2, p < .0001), and 76C (x 45.9, p < .0001). Despite these

"significant chi-squares, the overlap of composite distributions for total

sample graduates and nongraduates is high (see Figure 2).

Table 5

Failure Rate by MOS

MOS N % Attritiona Recycles Reliefs Transfers

71D 103 17.1 3.0 16.1 71M 98 4.1 4.1

71N 131 11.5 3.4 7.b

73C 214 0.5 0.5

75B 525 3.8 1.0 2.8

75C 101 --

75D 238 21.8 5.5 15.6 0.8 75E 296 25.9 0.3 0.3 25.3

76C 1,215 6.3 6.3

76J 99 7.0 2.0 5.0

76P 618 0.7 0.7

76W 340 --

Total 3,978 6.9

a Because of inability to comprehend course material.

DISCUSSION

This research has shown the importance of mathematics aptitude in clerical training school performance. The results are consistent with previous

validation research on clerical MOSs. In constructing the ASVAB 8/9/10 composites, Maier and Grafton (1981) found that AR was amonc the four subtests

most predictive of training and job performance in Army clerical occupations

However, at that time it was considered important for all the military services to use the same predictors whenever feasible, and the other services

preferred to avoid a heavy quantitative loading for the clerical areas. It

was decided that the Army would use the same subtests that the Air Force and

Marine Corps used to classify clerical personnel (i.e., WK, PC# NO, and CS).

The reported loss in validity related to the Skill Qualification Test (SQT)

and training performance criteria was 3 points, from .58 to .55 exciuding AR

(Maier & Grafton, 1981). More recent ASVAB validation research has also

found AR to be important in predicting clerical performance (Maier, 1982).

The speeded tests, CS and NO, were not found to be as predictive of

clerical performance as would be expected from a conceptual analysis of the

job tasks. This may result from attenuation due to unreliability of these

subtests in operational use. Compared to power tests, the speeded tests

12
.4.4

4-4

3 .Lto

0 u

13.:

V a-
are subject to additional error variance because of timing and practice cffects that tend to :educe their test-retest reliability. McCormick and his

associates (1982) found that, when applicants were allowed to retest repeatedly, scores of speeded tests showed the greatest improvement. An ongoing

research project at the Army Research Institute will investigate methods that

would permit accurate and reliable measurement of job-relevant perceptual skills.

Although the utility analyses revealed that substantial potential savings might be possible from modifying the CL composite, the design of all

Army aptitude area composites must be considered within the total context

of all the positions to be filled. While a composite of AR, PC, and MK is

suggested as an efficient predictor of training performance, possible opportunity costs of loss in differential prediction must also be considered.

While the Arithmetic Reasoning test can predict training success generally, it may be less suitable for classification purposes than a specific

predictor such as clerical speed or psychomotor skill, since the AR test

would probably predict success equally well in many different areas. ASVAB

AR items appear to tap a general problem-solving aptitude in which the arithmetic operations are not explicit but are left to the subject to choose.

These skills may be more general than the arithmetic operations called for

in the mathematics knowledge subtest. However, adding AR would increase the

intercorrelations among the existing classification composites since it is

already part of four of the composites in addition to the selector composite.

While overall systems optimization cannot be effected in any single research

project, ongoing ARI research is designed to address the best structure of

all ASVAB Aptitude Area Composites using a large sample of Ariy MOSs and a

variety of performance criteria. In the interim, it appears the composite

identified by this research (AR + PC + MK) is the most valid ASVAB composite

for clerical soldiers.

14

2..
REFERENCES

Bennett, G. K., & Cruikshank, R. M. (1949). A summary of clerical tests.

New York: The Psychological Corporation.

Brogden, H. E. (1949). When testing pays off. Personnel Psychology, 2,

171-183.

Brogden, H. E. (1959). Efficiency of classification as a function of number

of jobs, percent rejected, and the validity and intercorrelation of job

performance estimates. Educational and Psychological Measurement, 19,

181-190.

Cronbach, L. J., & Gleser, G. C. (1965). Psychological tests and personnel

decisions (2nd ed.). Urbana, IL: University of Illinois Press.

Department of the Army (October 1949). The Adjutant General's Office, Personnel Research Section. Development of aptitude areas for classification of enlisted personnel in the Army (PRS Report No. 808). Washington, DC.

Department of Defense (March 1982). Profile of American youth: 1980 nationwide administration of the Armed Services Vocational Aptitude BatterX.

Washington, DC: Office of the Assistant Secretary of Defense (Manpower,

Reserve Affairs, and Logistics).

Frankfeldt, E. (December 1970). Comparison of ACB and ASVAB clerical tests

for use in the Army Qualification Battery (Research Memorandum No.

70-6). Arlington, VA: U.S. Army Behavior and Systems Research

Laboratory. (AD A079 349)

Ghiselli, E. E. (1966). The validity of occupational aptitude tests. New

York: John Wiley and Sons.

Helms, W. H. (April 1960). Differential validity of the ACB for courses in

seven job areas (PRB Technical Report No. 1118). Washington, DC:

Personnel Research Branch.

Helme, W. H., Denton, B., & Anderson, A. A. (January 1963). Evaluation of

tests to predict success in Army clerk course (Technical Research Note

No. 130). Washington, DC: U.S. Army Personnel Research Office.

Hull, C. H., & Nie, N. H. (1981). SPSS Update 7-9. New York: McGraw-Hill.

Hunter, J. E., & Schmidt, F. I.. (1982). Fitting people to jobs: The impact

of personnel selection on national productivity. In M. D. Dunnette &

E. A. Fleishman (eds.), Human Performance and productivity, Vol. 1.

New Jersey: Erlbaum.

Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test

scores. Reading, MA: Addison-Wesley.

15

I -•|- : * 4 . . . . • . ..- .4 ". .. . .
APPENDIX A

SAMPLE ASVAB QUESTIONS

(Adapted from DoD 1304.12Z)

. GS General Science

Water is an example of' a a) solid b) gas c) liquid d)crystal.

AR Arithmetic Reasoning

A person buys a sandwich for 50c, soda for 25c , and pie for 4(k

What is the total cost?

a) $1.00 b) $1.05 c) $1.15 d) $1.25.

WK Word Knowledge

Small most nearly means

0-)sturdy b) round a) cheap d) little.

PC Paragraph Comprehension

The duty of the lighthouse keeper is to keep the light burning

no matter what happens, so that ships will be warned of the

presence of dangerous rocks. If a shipwreck should occur near

the lighthouse even though he would like to aid in the resuce

of its crew and passengers, the lighthouse keeper must

a) stay at his light b) rush to their aid c)turn out the light d) quickly sound the siren.

NO Numerical Operations

a) 1 b) 4 a) 5 d) 6

CS Coding Speed

Key

green . . . . 2715 man . . . . 3451 salt . . . . 4586

hat . . . . . 1413 room. . . . 2864 tree . . . . 5972

Answers

room 1413 2715 2864 3451 4586

green 2715 2864 3451 4586 5972

tree 2715 2864 5972 3451 4586

A-1

~"~J S"*>~ .•'. ".,*. •',' S~~ .. x.r..\:..K ~............\
AS Auto and Shop Information

The most commonly used fueZ for running automobile engines is

a) kerosene b) benzene c) crude oil d) gasoline.

-\, 1MK Math Knowledge

If a + 6 = 7 then a is equal to

a) 0 b) I c) -1 d) 7/16.

MC Mechanical Comprehension

:p.•

Which of the other gears is moving in the same direction as

gear 2?

a) Gear 1 b) Gear 3 o)Neither of the other gears

d) Both of the other gears.

EI Electronics Information

What does the abbreviation a.c. stand for?

a) additional charge b) alternating coil

a) alternating current d) ampere current

A-2 A-
APPENDIX B

DESCRIPTION OF JOB DUTIES FOR SAMPLE CLERICAL MOS

(Adapted from AR 611-201)

71D LEGAL CLERK

Prepares legal correspondence, records, and related papers, such as

courts-martial, courts of inquiry, and investigations using knowledge

of Uniform Code of Military Justice, Manual for Courts-Martial, Manual of the Judge Advocate General.

71M CHAPEL ACTIVITIES SPECIALIST

Performs chapel and religious support functions, such as religious

services, counseling, and education as well as general administration

and typing duties.

71N TRAFFIC MANAGEMENT COORDINATOR

Assists in receiving, storing, loading, ztiipping, and unloading supplies,

equipment, household goods, and perzonal effects. Prepares forms

and maintains records covering inbound and outbound shipments, recording

quantity and condition of property, claims for adjustments for property

lost or damaged in shipment.

73C FINANCE SPECIALIST

Performs duties pertaining to pay, leave, travel and maintenance of

personnel finance records of military personnel and other finance

functions, such as quality edit, disbursing aid travel, and general

administration.

75B PERSONNEL ADMINISTRATION SPECIALIST

Performs personnel and administrative functions such as personnel

actions, personnel accountability (SIDPERS), typing, and general

administration.

75C PERSONNEL MANAGEMENT SPECIALIST

Participates in occupational classification and management of manpower

resources, such as assignment, replacement, promotion, reduction,

classification, evaluation, testing, as well as typing and general

administration.

75D PERSONNEL RECORDS SPECIALIST

Maintains officer and enlisted personnel records in records section

of personnel activity, to include in/out processing, personnel

records maintenance, and preparation of SIDPERS input and control data.

B-1

)u~ k ttj. 'LA~AiF.p~ t~ta4 i q t~h¶ JpAa%
75E PERSONNEL ACTIONS SPECIALIST

Processes personnel actions concerning service members and their

dependents, counseling and referring individuals to appropriate support

facilities, preparing documentation for reenlistment, discharge

certificates, casualty reports.

76C EQUIPMENT RECORDS & PARTS SPECIALIST

Performs duties involving supply of repair parts and maintenance of

equipment records. Receives, stores, and issues repair parts. Initiates

and keeps records on equipment use and operation.

76J MEDICAL SUPPLY SPECIALIST

Performs requisitioning, receipt, inventory management, storage,

preservation, issue, salvage, stock control and accounting of medical

supplies and equipment.

76P MATERIAL CONTROL AND ACCOUNTING SPECIALIST

Performs management or stock record functions pertaining to receipt

distribution, and issue of Class II, IV, VI, VII and IX material.

Performs accounting, editing, document control, record keeping,

sales, and direct exchange of such material.

76W PETROLEUM SUPPLY SPECIALIST

Operates and maintains storage and transfer equipment for petroleum

products. Distributes petroleum by connecting tanks, operating pump

engines, and opening valves to transfer petroleun. Reads meters and

gauges and verifies amount and type of petroleum in storage.

B-2

[ • • • ;1, .••,.,,• ;2.,", -:..-. V.1".-' .•-N. -" ' '- . .-.. v . . .. ,-._.; -..-. -.-.-. '. .-
APPENDIX C

Interaorrebtions Between ASVAB Subte-t for

Profile Study Sample

(N-9173J

AIVAB Subtut

AR WK PC NO aS CS AS MK MC El

AR -

WK .71 -

PC .67 .10 -

No .63 .60 .60 -

e$ .72 .60 .60 .52 -

CS .61 .65 .56 .70 .45 -

AS .5A .52 .42 .29 .64 .22 -

MK .A3 .67 .64 .62 .89 .52 .41 -

MC .69 .A1 .12 A. .70 33 .74 .m0 -

El .66 .3 .37 .41 .76 .34 .76 .68 .74

AR - Arithwic Ramming CS a Coding Spee

WK * Word Knowledp AS a Auto and Shop Information

PC a Prarogiph Compfaiftwon MK a Mathematics Knowledge

NO - Nummical Opeamlons MC a Mechanial Comprehension

aS - BaMn Siemlm El a Eikcankas Information

Source: Profile of American Youth, 1980 Nationwide Administration of the

Armed Services Vocational Aptitude Battery, Office of the Assistant

Secretary of Defense (MRA&L), March 1982.

c-i 061984

,.~~~~~~~ - .- %'• . I*.m, N•.. . -W