Interim Composites (beginning January 2002)

In early 2001 responsible Army agencies determined that the changes in databases and
procedures to effect a change to 17 job families and composites could not be accomplished in time to
meet the DOD deadline of December 31, 2001. Accordingly, ARI was tasked to develop an interim set
of composites that retained the existing nine operational job families and utilized the remaining
ASVAB subtests after deleting NO and CS. Given the strong preference for predicted performance
composites by ARI and its DCSPER sponsor, ARI estimated least-squares composites (shown in Table
5) for the existing job families using the SQT database previously mentioned. The interim composite
validities (i.e., multiple correlations with SQT scores), along with the composite inter-correlations, are
shown in Appendix C. The existing and interim composites are compared in the charts in Appendix C.

Table 5: Interim AA Composites -- Subtest Weights (Relative)

gs ar as mk mc ei ve|
CL 0.000 1.000 0.110 0.767 0.148 0.110 0.980}
ico 0.313 0.532 0.733 1.000 0.595 0.343 0.529)
EL 0.151 0.818 0.754 0.890 0.469 0.598 1.000
FA 0.249 0.715 0.673 1.000 0.700 0.297 0.586
IGM 0.411 0.828 1.000 0.794 0.503 0.577 0.417
IMM 0.060 0.339 1.000 0.289 0.394 0.340 0.237]
OF 0.251 0.962 1.000 0.600 0.636 0.377 0.714]
SC 0.019 0.685 0.437 1.000 0.386 0.551 0.915
IST 0.187 0.727 0.357 0.697 0.446 0.230 1.000)

With the exception of the clerical composite, the interim composites are noticeably different
from the existing composites. Four groupings of the interim composites are suggested for descriptive
purposes. The CL, EL, SC, and ST composites are constructed around verbal, arithmetic reasoning,
and math knowledge subtests; the CO and FA composites feature math knowledge, with auto / shop
information, arithmetic reasoning, and mechanical comprehension; the GM and OF composites feature
auto / shop information and arithmetic reasoning; and MM features auto / shop information. Keep in
mind that all subtests are used in each composite, and the contribution of two or three of the lesser-
weighted subtests will equal that of one of the stronger subtests. Accordingly, the composites are more
diverse than this description indicates.

The interim composites represent a predicted performance metric with (as it turns out) strong
relationships between the composites and AFQT; its application leads to compositional effects that
favor qualification rates of 1-IIIA’s relative to IIIB’s and IV’s. As a new metric, the interim
composites require (small) changes in cutoff score levels in order to maintain the existing demographic
balance in qualification rates. Because the average predicted performance of those qualifying under
the interim composites is somewhat higher than under the existing composites, cutoff score levels can
be adjusted downward (when necessary) to preserve the demographic balance without reducing
performance standards.



Subtest relative weights

Subtest relative weights

1.200

1.000

0.800

0.600

0.400

0.200

0.000

4.200

1,000

0.800

0.600

0.400

0.200

Hectronics Repair Composites — Existing vs. Interim

gs ar as mk me ei ve no cs
ASVAB subtests
DIEL(SAA) MEL(LSE)
Field Artillery Composites -- Existing vs. Interim
gs ar as mk me ei ve no cs

ASVAB subtests

[FA(9AA) MIFA(CLSE)



95C Internment/resettlement specialist 95] 17
00B [Diver 51
13W ___|Field artillery meteorological crewmember - add 0304 13
13X Field artillery computer systems specialist 13}
27D Paralegal specialist 27|
91Z__|Chief medical NCO 91

B-5


Army Input Population — to arrive at the doubly restricted subtest scores in the MOS
samples. Considering the correlation of the subtest scores with the criterion scores and
the amount of restriction occurring at each stage, we can determine the restriction effect
on the hypothetical criterion scores and then provide a correction extending from the
MOS criterion scores to the less restricted populations where the criterion scores exist
only as a function of the subtest scores (i.e., as predicted criterion scores).

Approach

There is more than one algebraically equivalent way of providing operational u
and k values when criterion scores are only available on the doubly restricted MOS
samples, We will use an approach that utilizes the equality of G-weights computed in the
restricted and the unrestricted population (using Gulliksen’s formulation as described
below). The G-weights computed in the restricted population samples will be used as a
substitute for the unobtainable G-weights in the unrestricted population in Gulliksen’s
formula for computing the criterion variance in the unrestricted population.

1. Consider the matrix of G-weights, G, in each MOS sample. Our use for G is as an
entry value in Gulliksen’s formula (see below). The corrected validity coefficients,
obtained with the use of the formula at either or both the Army Input Population and
Youth Population points, were then employed in computing Beta weights in the
Youth Population. Note that this correction must be made from each MOS sample to
the Youth population to produce validity coefficients corrected for restriction in
range. These corrected MOS validity coefficients are then aggregated into a corrected
validity for each specified family, using acquisition values to weight the MOS
validity coefficients corrected to the Youth Population.

2. Visualize a composite computed for an individual by summing the product of each
subtest standard score (X) and B. The best weighted composite XB will have a SD
equal to the validity of predicted performance (PP) in the Youth Population if the
elements of the V matrix used in computing B are validity coefficients corrected for
restriction in range to represent the Youth Population, and the R matrix consists of the
inter-correlation coefficients among subtests as expected in the Youth Population.
The criterion variables, predicted as least square estimates (LSEs) by the PP
composites, have a SD equal to 1.0 in the restricted MOS samples, while the
hypothetical unrestricted criterion variables would have larger SDs in the less
restricted populations. Compute the Youth Population beta weights as follows:

B=R'V',
where R is the Youth Population matrix of subtest inter-correlation coefficients, V is

the matrix of validity coefficients corrected to the Youth Population, and superscript
T indicates transposition of the matrix. Looking at the formula in more detail,

R= Sx Cx Sx, and VT = Sx Cye Se,

where C represents criterion / subtest variance and co-variances found in Gulliksen’s
formulae, and S represents a diagonal matrix where each diagonal element is equal to
a reciprocal of a SD.

D-2



Chart 1A. Existing AA Composites: OF vs. SC

Subtest weights
&

024
°
mk me ei ve nto
ASVAB subtests
DOF (GAA) MSC (SAA)

Chart 1B. New Composites: OF vs SC

1.20

0.80

0.60 +

0.40

0.20 ap Hate lie
0.00 -

Subtest relative weights

gs ar as mk me ei
ASVAB subtests



References

Brogden, H. (1959). Efficiency of classification as a function of number of jobs, percent rejected, and
the validity and inter-correlation of job performance estimates. Educational and Psychological
Measurement, 19, 181-190.

Campbell, J.P. (1989). Improving the Selection, Classification, and Utilization of Army Enlisted
Personnel: Annual Report, 1987 Fiscal Year (Technical Report 862). Alexandria, VA: U.S.
Army Research Institute for the Behavioral and Social Sciences.

Eitelberg, M. (1988). Manpower for Military Occupations, Alexandria, VA: Human Resources
Research Organization (for Office of the Assistant Secretary of Defense / Force Management
Policy).

Horst, P. (1954). A Technique for the Development of a Differential Prediction Battery.
Psychological Monographs, 68 (9, Whole No. 380).

Johnson C., Zeidner J., and Leaman J. (1992). Improving Classification Efficiency by Restructuring
Army Job Families (Technical Report 947). Alexandria, VA: U.S. Army Research Institute for
the Behavioral and Social Sciences.

Johnson, C., Zeidner, J., Vladimirsky, Y., and Weldon, S. (1999). Development and Evaluation of
Classification-Efficient Job Choice Models (Technical Report, in preparation). Alexandria,
VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

McLaughlin, Rossmeissl, et al. ( 1984). Validation of Current and Alternative ASVAB Area
Composites (Technical Report 651). Alexandria, VA: U.S. Army Research Institute for the
Behavioral and Social Sciences.

Zeidner, J. and Johnson, C. (1993). Is personnel classification a concept whose time has passed? In
M. Rumsey, C. Walker, and J. Harris (eds.), Personnel Selection and Classification. Hillsdale,
NJ: Erlbaum.

Zeidner, J., Johnson, C., and Vladimirsky, Y. (1997). The Substitutability of Criteria in the
Development and Evaluation of ASVAB Classification Procedures (TR1071). Alexandria, VA:
U.S. Army Research Institute for the Behavioral and Social Sciences.

Zeidner, J., Johnson, C., Vladimirsky, Y, and Weldon, S. (2000). Specifications for an Operational
Two-Tiered Classification System for the Army (TR1108). Alexandria, VA: U.S. Army
Research Institute for the Behavioral and Social Sciences.

Zeidner, J., Johnson, C., Vladimirsky, Y, and Weldon, S. (2000). Effect of Reducing the Number of
Tests in the Armed Services Vocational Aptitude Battery (ASVAB) (Study Note 2001-01).
Alexandria, VA: U.S. Army Research Institute for the Behavioral and Social Sciences.

17



General Maintenance Composites ~ Existing vs.interim

1.200

1.000

0.800 ++

0.600 +4

Subtest weights

0.400 ++}

0.200 4+~

0.000
9s ar as mk me ei ve no os

ASVAB subtests
[DGM(SAA) MGM(LSE)

Mechanical Maintenance Composites ~ Existing vs. Interim

4.200

1.000

0.600 + ee |

0.400

Subtest relative weights

0.200

0.000
gs ar as mk me ei ve no cs

ASVAB subtests

TOMMSAA)

(@LSE)



FOREWORD

This report summarizes development of new Army aptitude area (AA) composites at the
Selection and Assignment Research Unit of the U.S. Army Research Institute for the Behavioral and
Social Sciences over the 1995 — 2000 period. This work is one strand of a larger effort to improve
classification (i.e., determination of most appropriate initial training and subsequent job assignment) of
new recruits entering the Army, and has been conducted with the sponsorship of the Enlisted
Accessions Division of the Directorate of Military Personnel Management of the Office of the Deputy
Chief of Staff for Personnel.

The AA composites are derived from the Armed Services Vocational Aptitude Battery
(ASVAB) given to all applicants, and play a central role in defining and measuring recruit aptitudes
required for training and assignment into Army jobs. The report first describes development of 17 new
composites and job families, designed to be more classification-efficient than the existing nine AA
composites. The new 17 composites are being evaluated and tested at the present time, and are under
consideration for implementation in the 2004 — 2005 period. The report then describes the
development of a set of nine interim composites scheduled to replace the existing nine composites in
January 2002. The immediate catalyst for the development and implementation of the interim
composites has been the DOD decision to eliminate two subtests within the ASVAB, thereby creating
aneed to reconfigure the composites. Like the proposed new 17 composites, the interim composites
are based on measures of soldier job performance and use all the information contained in the ASVAB.

The interim composites were in fact implemented on January 2, 2002. Prior to the
implementation decision, briefings and discussions were held with the responsible Army managers
within Total Army Personnel Command, Office of the Deputy Chief of Staff for Personnel, and the
Army Development Systems XXI Task Force.

“Ate. 0) Srectin
TA M. SIMUTIS
Technical Director



55B Ammunition specialist 55} 10)
55D Explosive ordnance disposal specialist 55 10
62E Heavy construction equipment operator 51 10)
62F Crane operator 51 10:
62G__|Quarrying specialist RC 51 10
62H Concrete and asphalt equipment operator 51 10
62J _ |General construction equipment operator 51 10
77W ___|Water treatment specialist 7 10
88H __|Cargo specialist 88 10
92M ___|Mortuary affairs specialist 92) 10
92R Parachute rigger 92 10
92S __|Laundry and textile specialist 92 10
45E __|M1 Abrams tank turret mechanic 63) 11
45N _|M60A1/A3 tank turret mechanic RC 63} 11
62B Construction equipment repairer 63) 11
63A __|M1 Abrams tank system maintainer 63 11
63B Light-wheel vehicle mechanic 63 11
63D _|Self-propelled field artillery system mechanic 63 11
63E M1 Abrams tank system mechanic 63 11
63G __|Fuel and electrical systems repairer 63 11
63H Track vehicle repairer 63) 11
63J Quartermaster and chemical equipment repairer 63) 141
63M __ [Bradley fighting vehicle system maintainer 63) 11
63N M60A1/A3 tank system mechanic RC 63} 1
63S Heavy-wheel vehicle mechanic 63) 11
63T Bradley fighting vehicle system mechanic 63} 11
63W___|Wheel vehicle repairer 63} 11
I63Y__|Track vehicle mechanic 63} 11
88K __|Watercraft operator 88} 11
88L Watercraft engineer 88 11
88P. Railway equipment repairer RC 88 11
88T Railway section repairer RC 88 11
88U__|Railway operations crewmember RC 88 11
67G _|Utility airplane repairer RC 67 12:
67N _|UH-1 helicopter repairer 67 12
67R__|AH-64 attach helicopter repairer 67 12
67S OH-58D helicopter repairer 67 12
67T UH-60 helicopter repairer 67 12
67U \CH-47 helicopter repairer 67 12
67V _|Observation/scout helicopter repairer 67 12
67Y 'AH-1 attack helicopter repairer 67 12
68B _|Aircraft powerplant repairer 67 12
68D _|Aircraft powertrain repairer 67 12
68F [Aircraft electrician 67) 12
68G __ Aircraft structural repairer 67 12)
68H __|Aircraft pneudraulics repairer 67 12
13M —_|Multiple launch rocket system crewmember 13) 13
14D _|Hawk missile system crewmember RC 14 13
14) Air defense command, control, communications, computers, 14 13
and intelligence tactical operations center enhanced

B-3



The compositional effects can be illustrated by looking at the Mechanical Maintenance
composite in the neighborhood of a cutoff score of 100. Table 6 shows the percentage qualifying at
the various line scores in this neighborhood under the existing and interim composite. As can be seen,
the percentage qualifying (male and female) in the IIIB and IV test score categories is somewhat lower
under the interim vis-a-vis the existing composite. This is addressed by lowering the cutoff score for
the affected MOS; we found that the average predicted performance of the qualifying populations
under the existing composite at a cutoff score of 100 and the interim composite at a cutoff score of 98
is approximately equal; thus, on performance grounds alone, a drop of two points is appropriate. The
policy decision in this case to set the new cutoff score at 97 represents a decision to be inclusive and
stretches slightly what can be justified by the increased performance.

Table 6. Percentage Qualifying -- Existing vs. Interim Composites
Mechanical Maintenance

MOS = 14J, 45E, 45N, 63A (M), 63D, 63E, 63G, 63N, 63S, 63Y, 67R, 67S, 68J, 88K, 88P
Line Score

Male Female
All 1-3A 3B VI All 1-3A, 3B )
MM|
100) 65.9 80.4 40.2 25) 26.8 37.8 5.9 2.6]
MM-9
100} 64.3 81.4 33.8 16.2 24.4 36.2 2.0 0.6
99) 66.9 83.7 37.0 18.8} 27.0 39.9 26 0.9)
98) 69.4 85.8 40.5 21.5) 29.9 43.9 3.4 1.2)
97| 71.9 87.8 44.0 24.2) 33.0 48.0 44 1.3)
96 74.3 89.6 475 27.3} 36.1 52.2 5.7 15)

Note: MM is the existing composite, based on 4 subtests, including NO subtest.
MM-9 is the interim composite, based on 7 subtests after deletion of NO and CS from
the battery.

It is also of interest to illustrate the mitigating effect played by the interim composites on
female qualification rates. Consider again the Mechanical Maintenance composite. The deletion of
NO from the ASVAB directly affects the MM composite and female qualification rates for that
composite. Table 7 shows percentage-qualifying rates at different line scores using the existing MM,
MM_3SSS (which is the unit-weighted MM with NO excluded), and MM-9 composites. In the first
stage — the four subtest versus the three subtest unit-weighted comparison, we see that qualifying rates
for males and especially females are lower overall. At a cutoff score of 95, the female percentage-
qualifying overall rate falls from 45.6% to 28.6% with the deletion of the NO subtest. In the second
stage — the three subtest unit-weighted versus the least-squares composite, we see that percentage-
qualifying rates are slightly higher for males and noticeably higher for females, indicating mitigation
by the least-squares composite over the existing composite with NO deleted.



PROPOSED NEW COMPOSITES: 17 LSE

47 LSE COMPOSITE VALIDITIES
cl1 cl2 CO1 Coz Eli El2 EL3 FA GMi GM2 MM1 MM2 OF SC STi ST2  sT3
0.713 0.654 0.525 0.64 0.675 0.682 0.718 0.594 0.767 0.639 0.762 0.748 0.657 0.661 0607 0.719 0.72

17 LSE COMPOSITE CORRELATIONS
Cit Cl2 CO1 CO2 EL1 E12 EL3 FA GMi GM2 MM1 MM2 OF SC STi ST2 873

cu 1.000 0,965 0.930 0.927 0.921 0.936 0.970 0.933 0.898 0.906 0.799 0,922 0,908 0.958 0.949 0.976 0.958
cL2 0.965 1.000 0.980 0.981 0.978 0.985 0.995 0.985 0.970 0.969 0.897 0.979 0.975 0.994 0.991 0.996 0.994
cot 0.930 0.980 1.000 0.998 0.994 0.998 0.977 0.999 0.993 0.997 0.956 0.993 0,994 0.992 0.993 0.975 0.991
co2z 0.927 0.981 0,998 1.000 0.998 0,999 0,982 0.999 0.997 0.998 0.961 0.997 0.998 0.994 0.996 0.979 0.994
eu 0.921 0.978 0,994 0,998 1.000 0.997 0.981 0.995 0.997 0.996 0.966 0.998 0.998 0.992 0.993 0.976 0.992
eL2 0.936 0.985 0,998 0,999 0.997 1,000 0,985 0.999 0.995 0.996 0.953 0.996 0.995 0.997 0.996 0.982 0.995
ELS 0.970 0.995 0.977 0.982 0.981 0.985 1.000 0.982 0.970 0.968 0.901 0.982 0.976 0.994 0.994 0.999 0.996
FA 0.933 0.985 0,999 0.999 0.995 0.999 0,982 1.000 0.993 0.996 0.952 0.995 0.996 0.995 0.996 0.981 0.994
cmt 0.898 0.970 0.993 0.997 0.997 0.995 0.970 0.993 1.000 0.998 0.974 0.995 0.998 0.986 0.988 0.964 0.985
em2 0.906 0.969 0.997 0.998 0,996 0.996 0,968 0,996 0,998 1.000 0.974 0.995 0.997 0.986 0.988 0.964 0.985
MM1 0.799 0.897 0.956 0.961 0.966 0.953 0.901 0.952 0.974 0.974 1.000 0.965 0.972 0.929 0.935 0.889 0.930
MM2 0.922 0.979 0.993 0.997 0.998 0.996 0.982 0.995 0.995 0.995 0.965 1,000 0.999 0.991 0.994 0.977 0.993
oF 0.908 0.975 0.994 0.998 0.998 0.995 0.976 0.996 0.998 0.997 0.972 0.999 1,000 0.988 0.992 0.971 0.989
sc 0.958 0.994 0.992 0.994 0,992 0.997 0.994 0.995 0.986 0.986 0.929 0.991 0.988 1.000 0.997 0.992 0.998
sTt 0.949 0,991 0.993 0,996 0.993 0.996 0.994 0.996 0.988 0.988 0.935 0.994 0.992 0.997 1.000 0,992 0.999
sT2 0.976 0.996 0.975 0.979 0.976 0.982 0.999 0.981 0.964 0.964 0.889 0.977 0.971 0.992 0.992 1.000 0.995
ST3 0.958 0.994 0.991 0.994 0.992 0.995 0.996 0.994 0.985 0.985 0.930 0.993 0.989 0.998 0.999 0.995 1.000

A-2



scores (when all weights are constrained to be positive or zero) and aggregated criterion scores that
represent the MOS of each specified aptitude area. The correlation coefficient to be maximized is
computed using restriction-in-range corrections that make the coefficient a best estimate of what it
would have to be for the measured restriction effects to have the effect of reducing the Youth
Population correlation coefficient to the magnitude observed in the restricted MOS sample. The
validity coefficients and criterion scores aggregated across MOS samples to represent each aptitude
area are doubly restricted since the soldiers in these MOS samples have been selected into the Army
and classified and assigned to the MOS in which each validity is computed. Each such validity
coefficient is corrected to provide an estimate of what the validity coefficient would be if the criterion
were perfectly reliable.

The incorporation of restriction in range corrections into procedures for assuring that best weighted
composites have an expected mean of 100 and standard deviation of 20 in the Youth Population can be
summarized as follows:

1. Obtain the corrected subtest validity coefficients in each MOS sample. This correction is for
the unreliability of the criterion and for the effect of double restriction in range on the
predictors and criterion variables in each MOS sample. This step provides the subtest validity
coefficients corrected to the Youth Population.

2. For each subtest multiply each corrected test validity by the density of that MOS and then sum
these products across the set of MOSs contained in each job family to provide an estimate of
each subtest validity (against a hypothetical job family criterion) within the restricted job
family.

3. Use these job family subtest validity coefficients and the inter-correlation coefficients among
the subtests to compute all positive or zero regression weights that are appropriate for applying
to subtest standard scores (SSSs) to provide LSEs of the criterion. We find and compute those
positive Beta weights (i.e., B-weights) that maximize the prediction of the criterion (as
measured by multiple correlation coefficients) for all possible subsets of B-weights.

4. Separately for each job family, determine expected SDs of the hypothetical job family criterion
variables as expected in the Youth Population. Setting the SDs of the subtest scores to 10, and
using these criterion SDs, compute the raw score regression weights (i.e., b-weights) by
converting each B-weight. Note that the resulting b-weighted composites will have SDs equal
to values less than 10 because of the effects of the positive inter-correlation coefficients among
the subtests.

5. Transform the b-weights into u,k values that provide composite scores with expected SDs equal
to 20 and means equal to 100 when applied to operational subtest scores. (a) To do this we
multiply each b-weight by a composite multiplier (CM) that will convert the composite to have
a SD of 20 without considering the composite mean. (b) In the next step, the constant is
calculated to center the composite at 100.

15



loperator/maintainer

414R Bradley linebacker crewmember 14) 13
14S __|Avenger crewmember 14 13
88M __|Motor transport operator 88} 13
91M ___|Hospital food service specialist 91 13
92G _|Food service specialist 92) 13
13R___|Field artillery firefinder radar operator 13 14)
31C [Radio operator-maintainer 31 14
31R [Multi-channel transmission systems operator/maintainer 31 14
74C _|Telecommunications operator/maintainer 74 14
77L Petroleum laboratory specialist 7 15
91A Medical equipment repairer 91 15
91D __|Operating room specialist 91 15
91E Dental specialist 91 15
91G _|Patient administration specialist 91 15
91H Optical laboratory specialist 91 15)
91d Medical logistics specialist 91 15
91K _|Medical laboratory specialist 91 15
91P. Radiology specialist 91 15
91Q __|Pharmacy specialist 91 15
91R___|Veterinary food inspection specialist 91 15
91S Preventive medicine specialist 91 16
91T ‘Animal care specialist 91 15
91X 91 15
25M __|Multimedia illustrator 25 16
33W __|Intelligence and electronic warfare system repairer 33 16)
37F Psychological operations specialist 37) 16
38A _|Civil affairs specialist RC 38) 16
46Q__|Journalist 46 16
46R Broadcast journalist 46 16'
51T Technical engineer 51 16
74B _|Information systems operator/analyst 74 16
81L Lithographer 81 16
81T Topographic analyst 81 16
82D _|Topographic surveyor 81 16)
96B Intelligence analyst 96 16)
96D Imagery analyst 96 16)
96U Unmanned aerial vehicle operator 96 16)
97B _|Counter-intelligence agent 96 16
97L Transtlator/interpreter 96 16
98C [Signals intelligence analyst 98 16
98H —|Communications locator/interceptor 98 16
98J Electronic intelligence interceptor/analyst 98 16
98K Signal collection/identification analyst 98 16)
98X 98: 16
54B Chemical operations specialist 54 17
82C __|Field artillery surveyor 13 17
93C _|Air traffic control operator 93 17
93P [Aviation operations specialist 93} 17
95B [Military police 95) 17

B-4



Table 3 provides a short occupational description of the 17 job families and highlights the
major differences between the existing (see Table 1) and new composites. Generalizing for a moment,
we can characterize the new composites by the addition of AR, VE, and MK. More detail is provided

in Appendix A. In that appendix, the reader will find: (a) a table providing the new composite
validities (i.e., multiple correlation with the underlying SQT performance criterion) along with
composite inter-correlations; and (b) charts describing the new composites. Each chart depicts the

Table 3: New Composite Changes and Job Family Descriptions

Family Occupation description Highlights of the major changes in
composition of the New Composites vis-a-
vis existing composites

CL-1 CMF 71 clerical VE, MK dominate

CL-2 Other clerical functions Similar to existing clerical family

CoO-1 Infantry Adds MK in predominant role

CO-2 Combat engineer; armor crew Adds MK, VE in predominant roles

EL-1 Combat equipment operator / maintainer | Adds VE, AS; relative balance

EL-2 Equipment operator Adds VE, AS; MK predominates

EL-3 Equipment repairer Adds VE in predominant role

FA Cannon fire Adds VE, AS

GM-1 Mechanic / repairer Adds AR

GM-2 Various Adds AR, MC

MM-1 Land vehicle mechanic / repairer AS predominates

MM-2 Helicopter / aircraft repairer Adds VE, AR, MK

OF Various operators; food service Adds AR.

SC Radar / radio transmission Adds MK, EI

ST-1 Medical Adds AR; relative balance

ST-2 Analyst Adds AR; VE dominates

ST-3 Operations / various Adds AR; relative balance

ASVAB subtests comprising the existing AA and new composites, and allows the reader to compare
the subtest weighting structures across existing and new composites. As an illustration, consider the
shredding of the existing mechanical maintenance (MM) job family into two new families: land
vehicle (MM-1) and aircraft vehicle (MM-2) maintenance (also shown in Chart 2). The existing MM
composite is defined by AS, MC, EI, and NO. The predominant contributor to MM-1 is AS, followed
distantly by MC, EI, and MK. The major contributors to MM-2 are VE, AS, AR, MK, and MC. The
latter shows more balance in the aptitudes identified as necessary to perform in the job family, and as a
consequence female qualification rates across a range of cutoff scores are noticeably higher for MM-2
than for the existing MM composite (see Chart 2).

The predicted performance composites are shown in Table 4 for the 17 job families, each row
depicting relative subtest weights.



Table 1: ASVAB Subtests Comprising the Army’s AA Composites and AFQT

ASVAB SUBTESTS

AR_|MK|VE |AS{EI |GS_ |MC |CS_|NO

AA COMPOSITES

~
~

Electronics Repair xX

General Maintenance xX Xx

Pd] PS]

Mechanical Maintenance

><

Operators / Food

ps

Surveillance / Communications

PA] P| PX] PS] PS

Combat

Pel ial tal

Field Artillery

bead ad al ead ta)

Skilled Technical

Clerical

General Technical

re] >< |PS
>] [P<] Ps] ><
2S] ><] P<

AFQT x

ASVAB is comprised of following subtests: Arithmetic Reasoning (AR), Math Knowledge (MK), Verbal (VE)
= Paragraph Comprehension (PC) + Word Knowledge (WK), Auto & Shop Information (AS), Electronics
Information (EI), General Science (GS), Mechanical Comprehension (MC), Coding Speed (CS), Numerical
Operations (NO).

The classification task is the assignment of recruits into initial job training in order to provide
the best match or fit while meeting accession and training management goals. While the formulation of
the AA composites as shown in the table has a certain intuitive appeal, it does not provide the requisite
classification capability.

Fit pertains to congruence between recruit aptitudes and job requirements, and should be
measured with predicted performance in the assigned job as the criterion. The AA composites were
not developed with a job performance criterion. The training criteria used to validate the AA
composites were the best available criteria at the time, but do not adequately represent soldier
performance.! Moreover, each composite is formulated as a combination of 4 unit-weighted subtests
(except for Clerical). This approach is a carry-over from the 1950s when calculations were kept as
simple as possible, reflecting earlier limitations of computing power. One consequence is that AA
composites do not track performance as well as they might A better approach is full least-squares
estimation, in which all subtests contribute to each composite in proportion to their power to explain
the variation in performance.

Classification efficiency requires the use of predictors that can distinguish how well a recruit is
likely to perform in different jobs. The AA composites have limited ability to do this. Consider that

1 In the mid-1980s representative MOS were examined in a validation of the AA composites using school grades, early
Skill Qualification Test (SQT) performance data, and Project A criterion measures, though the unit-weighted calculation
rocedures were retained (see Campbell (1989)).
Predictive validity estimates in the literature reveal correlations in the 0.55 to 0.60 range between the composites and
measures of training and job performance.
2



APPENDIX B

MOS Membership in 17 Job Family Structure

17 Job Family Structure:

1 = Clerical-1; 2 = Clerical-2

3 = Combat-1; 4 = Combat-2

5 = Electronics Repair-1; 6 = Electronics Repair-2; 7 = Electronics Repair-3

8 = Field Artillery

9 = General Maintenance-1; 10 = General Maintenance-2

11 = Mechanical Maintenance-1; 12 = Mechanical Maintenance-2

13 = Operators / Food

14 = Surveillance / Communications

15 = Skilled Technical-1; 16 = Skilled Technical-2; 17 = Skilled Technical-3

12/20/01
Entry-level MOS List [PERSCOM 11/2/2001]
MOS [TITLE “47
CMF Family
56M ___|Chaplain assistant 56 1
71D Legal specialist - del 0110 71 41
71L Administrative specialist 71 4
73C Finance specialist 71 41
73D __|Accounting specialist 71 41
75B Personnel administration specialist 71 41
75F Personnel information system management specialist 71 1
75H Personnel services specialist 71 1
88N _|Transportation management coordinator 88 1
77F Petroleum supply specialist 77 2
92A [Automated logistical specialist 92 2
92Y __|Unit supply specialist 92] 2
11B Infantryman 11 3
11C _|Indirect fire infantryman 11 3
11H Heavy antiarmor weapons infantryman 14 3
11M ___|Fighting vehicle infantryman 1 3
11X 11 3
12B (Combat engineer 12 4
12C _|Bridge crewmember 12 4
19D Cavalry scout 19} 4
19K _|M1 armor crewman 19 4
14E Patriot fire control enhanced operator/maintainer 14 5
14L AN/TSQ-73 air defense artillery command and control 14 5
system operator/maintainer_ RC.
14M ___|Man portable air defense system crewmember RC 14) 5
14T Patriot launching station enhanced operator/maintainer 14 5
23R __|Hawk missile system mechanic - del 0110 RC 14 5
31L Cable systems installer/maintainer 31 5
51R__|Interior electrician 51 5
52G __|Transmission and distribution specialist_ RC 51 5

B-l



Subtest relative weights

Subtest relative weights

General Maintenance Composites ~ Existing vs. New

1.20

1.00

0.80 +4

0.40 $f}

0.20 t+

0.00

gs

ar

a mk me
ASVAB subtests

GM(9\A)

Mechanical Maintenance Composite - Existing vs. New

ei

0.6

0.2

gs

ar

as mk me ei
ASVAB subtests

[IMM (exstingAA) IMM-1 (new 17) CMM-2 (new 17)

no



be adjusted downward to preserve the demographic balance without reducing average performance

levels.

viii



Executive Summary

The Army currently employs nine Aptitude Area (AA) composites to classify new recruits; they are
derived from the Armed Services Vocational Aptitude Battery (ASVAB) subtests in a manner which
makes them easy to calculate but inefficient for classification. The Office of the Secretary of Defense
(OSD) is eliminating the two timed subtests in the ASVAB by December 31, 2001. This is due to the
expenses involved in administering these tests in an automated environment. When this comes to pass,
the classification efficiency of the existing Army composites will be further reduced and will
necessitate measures to redefine the existing composites. In their place the Army is expected to adopt
new composites, which have been developed by the U.S. Army Research Institute for the Behavioral
and Social Sciences (ARI) based on a job performance criterion.

The Army’s current set of nine composites were formed in 1976 to represent measures of the aptitudes
/ skills required for training and assignment to the corresponding nine Army job families.
Classification efficiency requires the use of composites that can distinguish how well a recruit is likely
to perform in different jobs. The AA composites have limited ability to do this. In addition, the AA
composites were not developed with an on-the-job performance criterion. The training criteria used to
validate the AA composites were the best available criteria at the time, but do not fully represent
soldier job performance. Each composite is formulated as a combination of four unit-weighted
subtests (except for Clerical). This approach is a carry-over from the 1950s when calculations were
kept as simple as possible, reflecting earlier limitations of computing power. One consequence is that
AA composites do not track performance as well as they might. A better approach is full least-squares
estimation, in which all subtests contribute to each composite in proportion to their power to explain
variation in performance.

Based upon extensive research utilizing job performance data, ARI has developed a set of 17
operational classification-efficient job families and corresponding composites which would be used for
administrative, counseling, and school proponent purposes. Equivalent minimum eligibility standards
(cutoff scores) would be established against these new composites. This job family structure is
consistent with the career management fields currently used by the Army and with the current AA
system. The new structure is, in effect, a further shredding of existing families.

In early 2001 the responsible Army agencies indicated that the changes in databases and procedures to
effect a change to 17 job families and composites could not be accomplished in time to meet the OSD
deadline. Accordingly, ARI was tasked to develop an interim set of composites that retained the
existing nine operational job families and utilized the reduced ASVAB. Given the strong preference
for predicted performance composites by ARI and its DCSPER sponsor, ARI estimated least-squares
composites using the 1989 Skill Qualifications Test (SQT) database.

The resulting interim composites represent a predicted performance metric with strong relationships
between the composites and AFQT; its application leads to compositional effects that favor
qualification rates of Category 1-IIIA’s relative to TIIB’s and IV’s. As a new metric, the interim
composites require (small) changes in cutoff score levels in order to maintain the existing demographic
balance in qualification rates. Because the average predicted performance of those qualifying under
the interim composites is somewhat higher than under the existing composites, cutoff score levels can

vil



Table 4: 17 LSE Composites -- Subtest Weights (Relative)

gs ar as mk mc ei ve]

1 CL1 0.000 0.000 0.000 0.985 0.095 0.022 1.000

2 CL2 0.000 1.000 0.261 0.712 0.269 0.175 0.797

301 0.259 0.401 0.658 1.000 0.532 0.262 0.398

4 C02 0.428 0.813 0.893 1.000 0.731 0.518 0.809

5 EL1 0.056 0.842 1.000 0.808 0.507 0.720 0.997

6 EL2 0.235 0.636 0.680 1.000 0.474 0.556 0.675

7 EL3 0.102 0.675 0.206 0.479 0.213 0.239 1.000)

8 FA 0.249 0.715 0.673 1.000 0.700 0.297 0.586

9 GM1 0.335 1.000 0.932 0.614 0.429 0.684 0.414)

10 GM2 0.461 0.593 1.000 0.923 0.545 0.427 0.390}
11 MM1 0.065 0.279 1.000 0.239 0.359 0.344 0.137]
12 MM2 0.020 0.789 0.954 0.662 0.647 0.299 1.000)
13 OF 0.251 0.962 1.000 0.600 0.636 0.377 0.714
14 SC 0.019 0.685 0.437 1.000 0.386 0.551 0.915
15 ST1 0.390 0.886 0.456 0.752 0.659 0.325 1.000)
16 ST2 0.120 0.673 0.177 0.604 0.268 0.118 1.000)
17 ST3 0.112 0.670 0.449 0.741 0.461 0.265 1.000)

Predicted performance composites have also been estimated for 150 job families. These
constitute the “invisible” tier, and would be embedded within the appropriate REQUEST module’® for
classification optimization. Thus, the classification gains embodied by greater delineation of job
families is obtained without the (visible) accompanying administrative complexity. In sum,
differential assignment theory supports the use of a larger number of job families, having more
homogeneity within and greater heterogeneity between families, with stable weights producing higher
MPPs. More stable families of this type produce larger mean validity and lower intercorrelations, thus
providing greater optimization efficiency.

Adoption by the Army

A two-phase adoption and implementation process is envisioned. The purpose of the first
phase is to educate and inform Army management about the issues, and obtain consensus and approval
for implementation of the recommendation. As part of the “educate and inform” management, this
phase could include a one-day workshop sponsored by DCSPER / PERSCOM, school proponent site
visits, and briefings to Army management to synthesize the issues and school site findings. Issues that
would be addressed at school proponents include: placement of MOS within new 17 operational job
family structure; determination of how changes in composites would affect composition of MOS
eligible applicant pool; and derivation of new cutoff scores under new composites / job family
structure. In the second phase, the new composites and job family structure would be installed into
Army recruiting and training management procedures with the coordination and assistance of the
affected agencies.

16 For example, the Enlisted Personnel Allocation System (EPAS) under development is designed to push REQUEST
toward optimal assignments. EPAS field-testing is scheduled for FY 2001-03.
07



Proposed New Army Aptitude Area Composites

The Army Research Institute (ARI) has been conducting research with the objective of
improving the effectiveness of Army classification (i.¢., matching of recruits to initial job training
assignments). Under current Army procedures, applicants are offered initial training in MOS for
which they are eligible and which meet the accession needs of the Army. Eligibility is based on
minimum standards, with applicants’ AA (Aptitude Area) composite scores serving as the scale upon
which MOS eligibility is measured. Two major recommendations have come out of the research
conducted by ARI. The first recommendation is to improve the classification process. The existing
training reservation system (known as REQUEST) should be enhanced so that recruits are assigned to
jobs that they are likely to perform best, while meeting the training management goals of the Army.
The Enlisted Personnel Allocation System enhancement, now under development, is designed to push
REQUEST toward optimized assignments. The second recommendation is to improve the
classification metric by replacing the AA composites with more precise measures of predicted
performance. We have found that the AA composites were not adequately designed for today’s
classification task of distinguishing recruit predicted performance across a variety of jobs.

In this report we summarize the development of new classification-efficient composites and
corresponding job families.

Role of Test Composites in Classification and Training Management

In conducting classification and training management functions, the Services have created
occupational aptitude test composites derived from the Armed Services Vocational Aptitude Battery
(ASVAB). Each Service creates and applies its own aptitude composites in the manner that it sees fit.
Generally, the composites are chosen and applied because they have been shown to predict training
performance for broad classes of occupations within a Service (Eitelberg, 1988, p. 69). Thus, for
purposes of classification, broad classes of occupations have become defined with respect to ASVAB
composite subject areas. We will refer to these broad classes of occupations as job families.

The Army’s current nine AA composites go back to 1976. These composites were formulated
to represent measures of the aptitudes / skills required for training and assignment to the corresponding
nine Army job families: clerical, combat, electronics repair, field artillery, general maintenance,
mechanical maintenance, operators / food, surveillance / communications, and skilled technical. Table
1 depicts the AA composites and their component ASVAB subtests. Eligibility for training in a
particular MOS is determined by the recruit’s AA composite score relative to the minimum or cutoff
score required for training in that MOS. For example, MOS 55B (ammunitions specialist) requires a
minimum score of 95 on the General Maintenance composite to be eligible for job training and
assignment.



Zeidner, J., Johnson, C., and Vladimirsky, Y. (1998). Fairness of Proposed New ASVAB Test
Composites for Restructured Job Families (in preparation). Alexandria, VA: U.S. Army
Research Institute for the Behavioral and Social Sciences.

Zook, L. (1996). Soldier Selection: Past, Present, and Future (Special Report 28). Alexandria, VA:
U.S. Army Research Institute for the Behavioral and Social Sciences.

18



Study
Report
2002-03

Proposed New Army Aptitude Area
Composites: A Summary of Research
Results

Peter M. Greenston
U.S. Army Research Institute

20020206 137

United States Army Research Institute
for the Behavioral and Social Sciences

January 2002

Approved for public release; distribution is unlimited.



covariance matrix of operational test scores (explicitly restricted variables, e.g., sub-tests)
are invariant to the effects of restriction (as defined). Thus it is assumed that

G =(Cx)" Ce)"

can be computed in a restricted population sample and substituted in formulae for use in
the unrestricted population where a G-weight is to be entered. Gulliksen’s formula 42,
used to compute criterion variance in the Youth Population, requires such an entry. This
criterion variance is essential for converting Beta-weights into b-weights and obviously
cannot be directly computed in the Youth Population.

As previously stated, our objective is to have an algorithm replete with valid
formulae that will convert operational test scores into LSEs of the criterion (i.e. PP

composites) in a scale appropriate for use in the indicated population.

Application of Formulae 37 and 42

Applying combined formulae 37 and 42 to one criterion variable at a time, and
making small changes in Gulliksen’s notation, we can compute the squared SD of each
Youth Population criterion variable associated with each job family. This result can be
described as the Youth Population criterion variance, or YPCV:

YPCV = 1.0 + Cec (Cox) (( #Cax) (Co! - 1) Ca)”,

where ( Cx<) is a 9 by 1 vector of co-variances between the criterion variable and each of
the 9 tests, Cy, is a 9 by 9 matrix of co-variances among 9 tests using the operational test
scores, and vectors are denoted by underlining. Note that the asterisk matrix, e.g., *C,
indicates computation in the unrestricted (i.e., Youth Population) sample.”

The R matrix has the following relationship with the C,, matrix:

R=8xCxx Sx,

where S is a diagonal matrix for which the diagonal elements are equal to the reciprocals
of the SDs of either the subtests or the criterion variable in either the MOS sample or the
Youth Population, as indicated.

The *Cyot matrix is derived from the Gulliksen formula as:
(*C x)" = (Can) (G) = (*Crx) (Cx) "(Cae)"

2 Note that YPCV can also be written as follows:
YPCV = 1.0 + (W)(*CxW-(Coo)"),

where W = (C,,)" (Cyo)" , a9 by 1 vector of regression weights for a specified job family. W will also be
recognized as one column of the G matrix.

D-4



and several of the unique factors found in the ASVAB. The empirical implementation is based upon
the following three principles:

¢ One: New composites should utilize defensible criterion data (i.e., soldier performance data) and
all the informational power of the ASVAB battery.

In the estimation and testing of new composites, the research team created and validated a job
performance database of 260,000 Skill Qualification Test (SQT) observations over the 1987-89 period.
A carefully developed composite of hands-on and written job sample tests, known as the Project A job-
specific technical proficiency measure, was used to validate the SQT criterion. The researchers found
relationships between ASVAB and SQT similar to those found between ASVAB and Project A
measures, providing additional confidence in reliance upon ASVAB in the development of the new
composites (Zeidner, Johnson, Vladimirsky, 1997).

For each job family, new composites were developed by estimating the functional relationship
between individual SQT performance and ASVAB subtest scores -- hence the name predicted
performance composites. The resulting composites have good predictive power or what psychologists
call predictive validity, and all subtests contribute to each composite, but not equally. In other words,
the new composites utilize weights that allow each subtest to make its maximum contribution (taking
into account the other subtests in the battery) to predicted performance. This is in marked contrast to
the use of unit weights in the existing composites, in which the contributions of selected subtests are
predetermined and set equal to each other without regard to explanatory power.

¢ Two: Classification efficiency not only depends upon (a) how well the composites predict
performance but how distinctive the composites are, one from another, and also (b) how many
distinct job families can be identified using available criterion data.® In other words, predictive
validity is only one term in the classification efficiency equation and, thus, classification efficiency
cannot be described adequately by predictive validity alone.

According to this formulation, predictive validity and the distinctiveness of the composites (as
described by their intercorrelations) are both affected by increasing the number of job families.
Increasing the number of job families affects validity because it results in more homogeneous jobs
being placed together to be predicted by a single composite. Increasing the number of job families
reduces the intercorrelations among the composites because it results in a greater uniqueness in the job
families. And finally, as the number of job families increases, it is possible to more precisely assign
individuals to job families by capitalizing on intra-individual differences. Using the SQT database and
a clustering algorithm designed to identify classification-efficient job families, two sets of
classification-efficient job families were identified — a detailed set of 150 and a summary set of 17 job
families.

® See Brogden (1959); Johnson, Zeidner, and Leaman (1992, p. 6).

7 We begin with a set of 150 job families that can be identified with the SQT data set. In this initial position, maximum
classification efficiency would obtain with each MOS as its own job family. The clustering procedure entails finding the
two MOS which can be combined into a single job family with the smallest loss in efficiency. This is determined with
simulation experiments, and the selection of that combination which yields the largest MPP of assigned recruits. The steps
are repeated until further clustering of job families is judged to reduce classification efficiency by an unacceptable amount.



five pairs (out of 36 possible pairs) of AA composites have three out of four ASVAB subtests in
common. See Table 1: these are electronics repair and general maintenance; operators / food and
surveillance / communication; operators / food and mechanical maintenance; surveillance /
communication and combat; field artillery and combat. And 19 pairs of AA composites have two out
of four ASVAB subtests in common. The consequence is difficulty in distinguishing predicted recruit
performance across these pairs of job families.’

Developing Classification-Efficient Composites and Job Families

The ASVAB subtests, which underlie the AA composites, do better in tracking job
performance. Indeed, ARI Project A research of the 1980’s found a relatively strong relationship
between ASVAB factors and first-term soldier performance measures and validated the use of ASVAB
as a selection tool.! The AFQT composite is used for selection into the services.

With the success of the ASVAB factors in tracking performance, ARI conducted research
beginning in the early 1990’s to develop classification-efficient composites for use in Army
classification -- that is, to develop better measures for distinguishing recruit capabilities to do different
jobs and to test the efficacy of these measures for making better classification decisions. This research
was founded upon differential assignment theory (DAT), enhanced and applied by Dr. Joseph Zeidner
and Mr. Cecil Johnson of George Washington University.®

This research has grown out of the ongoing debate among psychologists about the nature of
intelligence and how it is measured. Generalizability theory suggests that one general ability factor is
sufficient to predict performance on jobs with a strong cognitive component. Proponents of the theory
believe that the underlying general ability factor is the only stable predictor of job performance as
measured in independent samples. If it is true that specific aptitudes do not add to the prediction of job
performance, then more efficient classification of individuals to jobs based on specific aptitudes is not
achievable. However, if there are two or more common or unique factors that differentially predict
performance in various jobs, then classification efficiency is a relevant issue (Zeidner and Johnson,
1993, p.377).

In developing DAT the research team members postulate that several factors differentially
predict performance in various jobs, and argue that DAT provides a more coherent framework for job
classification, although recognizing the general ability factor as the dominant predictor of performance
in a selection model (Zeidner and Johnson, 1993, p.378). The empirical implementation of DAT has
shown that there are useful factors contributing a nontrivial amount of classification efficiency in
addition to that contributed by the general ability factor. This includes both the three common factors

3 Additional evidence for the absence of differential classification power can be seen by examining, for each composite or
job family, how many times other composites do as well or better than its own composite in explaining performance. The
counts are as follows: CL, 7; SC, 7; FA, 5; EL, 4; GM, 4; MM, 3; OF, 3; CO, 2; and ST, 0. For example, at one extreme, 7
of the 9 composites do as good or better at explaining CL performance than does the CL composite itself. Validity data
source: McLaughlin, Rossmeissl, et al (1984). Note that data refer to CL and SC validities before they were revised in
1984,
* Four ASVAB factors were derived using factor analysis: verbal, quantitative, technical knowledge, and perceptual
speed. This is summarized in Zook (1996).
5 Zeidner and Johnson have built on the earlier work by Brogden (1946, 1959) and Horst (1954).

3



Subtest relative weights

Skilled Technical Composites — Existing vs. Interim

4.200

1,000

0.800 +4 2

0.600 +~| | a

0.400 4]

0.200

0.000
ar as mk me ei ve no cs

ASVAB subtests

EIST(9AA) MEST (SLSE)



Subtest relative weights

1.20

Skilled Technical Composites -- Existing vs. New

1.00

0.80 +}

0.40 4-4

0.20 ++

0.00

gs

ar

as mk me
ASVAB subtests

EXST(9AA) MBST-1(17) OST-2(17) OST-3(17)

A-7

ei



Note that one column of *Cx<) is ( Cue) , a vector used in the computation of YPCV.
The validity matrix (*V" ) required to compute Beta weights in the Youth Population has
the following relationship with the *C,<" vector:

one column of *V" is (*Sx) (*Cxc)" (*Se) 5

and note that *S, is a scalar.

Positively Weighted Composites for the Visible Tier

This section extends the initially professed objectives of this appendix beyond
restriction in range corrections and the conversion of Betas to u and k values. We will
now discuss the methodology for selecting the “best” positively weighted composites
where best is defined in terms of maximizing the multiple correlation coefficient of a set
of tests with the criterion.

The surest way to find this best positively weighted composite from a set of n
tests is to compute the Betas and validity coefficients for every possible combination of n
tests, then successive levels: for n-1 tests, then n-2 tests, ...to 2 tests --- rejecting any
combination of tests that has one or more negative weights. There is no need to actually
consider all of these combinations since there comes a point in this process where all
multiple correlation coefficients (Rs) for succeeding levels are lower than the highest R
in a prior level.

The multiple-correlation coefficient, R, corresponding to each set of Betas is
computed for each combination whether or not all of the weights are positive. Clearly, if
the R for each combination of m-1 tests, negative weights permitted, was less than the
highest R for m positively weighted subtests computed from the combinations considered
at the prior level, the stopping point has been reached. After the stopping criterion has
been reached, the set of subtests with all positively weighted coefficients that provides
the maximum R is selected as the very best set and these weights become the B-weights
for the associated subtests. All other tests are given a weight of zero in the composite
associated with the specified job family.

D-5


Subtest relative weights

Subtest relative weights

Operators / Food Composite - Existing vs. New

1.20

1.00 42

0.80

0.60

0.40

0.20 +=

0.00
gs a as mk me 4 ve no

ASVAB subtests

IOF(9AA) MOF (17)

Surveillance / Communications Composite - Existing vs. New

1.20

1,00 ae —

2

2
2
3
|
|
|

°
2
&

os ar a mk me ei ve
ASVAB subtests



PROPOSED NEW ARMY APTITUDE AREA COMPOSITES

CONTENTS

Role of Test Composites in Classification and Training Management.
Developing Classification-Efficient Composites and Job Families .
Fairness of the New Composites.........s.ssessssseerseseenrenneesees

Recommended New Composites and Job Family Structure .
Adoption by the ArMy.......sssssssecssssssssesssseeses
Interim Composites (beginning January 2002).
Estimation of New / Interim Composite Weights

References
Tables and Ch:

Table 1: ASVAB Subtests Comprising the Army’s AA Composites and AFQT 2
Table 2: Estimated MPP — Results of Simulation Experiments ... 6
Table 3: New Composite Changes and Job Family Descriptions . 10
Table 4: 17 LSE Composites — Subtest Weights (Relative). 11
Table 5: Interim AA Composites — Subtest Weights (Relative) .....s.ssssesecseeeetersseressserenees 12
Table 6: Percentage Qualifying -- Existing vs. Interim Composites: Mechanical Maintenance. 13
Table 7: Percentage Qualifying Rates for Mechanical Maintenance Composites. 14

Chart 1A: Existing Composites: OF vs. SC .
Chart 1B: New Composites: OF vs. SC...
Chart 2: Mechanical Maintenance Composites: Existing and New

Appendices

Appendix A: Existing 9 AA Unit-Weighted Composites vs. 17 LSE Composites ...
Appendix B: MOS Membership in 17 Job Family Structure ........ssessesssesserseeeses

Appendix C: Existing Job Families: AA Composites vs. 9 LSE Interim Composites
Appendix D: Restriction in Range and the Estimation of AA Composites

ix



Table 7. Percentage Qualifying Rates for Mechanical Maintenance Composites

Line Score} Male Female
All 1-3A 3B IV All 1-3A 3B IV
MM
105) 52.3 67.5 25.0 12.9 14.7 21.5 1.8 0.3}
100 65.9 80.4 40.2 25.0) 26.8 37.8 5.9 2.6
95 79.5 90.6 59.9 45.9) 45.6 59.9 18.8 10.7)
90 88.8 95.8 76.5 67.5] 65.2 78.3 40.6 27.8
MM_3SSS
105) 48.8 61.7 25.8 14.7 8.6 12.6 11 0.6
100) 61.4 74.3 38.5 25.0) 16.5 23.6 3.1 1.5)
95) 73.1 84.6 52.8 40.2) 28.6 39.2 8.3 3.9)
90 83.1 91.8 67.7 56.8 44.2 57.2 19.6 12.9}
MM-9
105) 50.8 67.9 19.9 7.3 13.3 20.1 0.5 0.3)
100) 64.3 81.4 33.8 16.2 24.4 36.2 2.0 0.6}
95} 76.6 91.2 51.1 30.2) 39.6 56.6 7.3 1.9)
90] 86.8 96.8 69.7 50.4} 58.3 77.9 21.3 7.2|

Note: MM is the existing composite, based on 4 subtests, including NO subtest.
MM_3SSS is the existing unit-weighted composite, but excluding NO subtest.

MN-9 is the interim composite, based on 7 subtests after deletion of NO and CS from
the battery.

Estimation of the New / Interim Composite Weights

This section summarizes how to obtain the AA composite weights (referred to as “u and k”
values in the source research reports, and incorporated into the subtest weights shown in Tables 4 and
5) for operational use in the applicant Youth Population.!” The validity coefficients we wish to
maximize in the Youth Population actually exist only in doubly restricted MOS samples containing the
Skill Qualifications Test (SQT) criterion in the 1987 - 1989 research data set. Appropriate corrections
have to be made to these restricted validity coefficients to obtain unrestricted validity coefficients that,
if subjected to restriction in range effects, would equal what was obtained in the MOS samples. We
also have to estimate what the criterion standard deviation (SD) would have to be in the unrestricted
population to yield the criterion SDs observed in the MOS samples.'*

Our values of u and k used in conjunction with operational ASVAB subtest scores provide
composite test scores with weights that maximize the correlation coefficients between composite test

1” The methodology summarized here pertains to the development of both interim and new (17 job family) composites.
For a detailed discussion of estimation in the face of restriction in range, see Appendix D.
18 It should be noted that whenever validity coefficients are mentioned, we are assuming that these coefficients have been
corrected for attenuation with respect to criterion unreliability. Even if we should refer to an uncorrected validity coefficient
(for restriction in range), this “uncorrected” coefficient has been corrected for attenuation.

14



INTERIM COMPOSITES
9-LSE COMPOSITE VALIDITIES AND CORRELATIONS

9 LSE COMPOSITE VALIDITIES
CL co EL FA GM MM OF sc st
0.677 0.566 0.684 0.594 0.698 0.756 0.657 0.661 0.679

9 LSE COMPOSITE VALIDITIES

cL co EL FA GM MM OF sc ST

cL 1.000 0.963 0.973 0.969 0.948 0.883 0.954 0.984 0.988
co 0.963 1.000 0.997 0.999 0.997 0.971 0.996 0.993 0.990
EL 0.973 0.997 1.000 0.997 0.994 0.965 0.996 0.997 0.996
FA 0.969 0.999 0.997 1.000 0.995 0.965 0.996 0.995 0.993
GM 0.948 0.997 0.994 0.995 1.000 0.983 0.998 0.986 0.982
MM 0.883 0.971 0.965 0.965 0.983 1.000 0.982 0.946 0.939
OF 0.954 0.996 0.996 0.996 0.998 0.982 1.000 0.988 0.986
sc 0.984 0.993 0.997 0.995 0.986 0.946 0.988 1.000 0.998
ST 0.988 0.990 0.996 0.993 0.982 0.939 0.986 0.998 1.000

C2



Subtest relative weights

1.200

1,000

0.800

0.600

0.400

0.200

0.000

Clerical Composites ~ Existing vs. Interim

4.200

41.000
2
© 0.800 7
§
2 0.600
3 |
3 0.400 —
a

0.200

oceo a | i

9s ar as mk me ei ve no
ASVAB subtests
[ICU 9AA) MECLISLSE)
Combat Composites ~ Existing vs. Interim
os ar as mk me ei ve no cs
ASVAB subtests
EICO(9AA) MICO@LSE)

cs



1.20

1.00

0.60

Subtest relative weights
°
®
8

0.00

1.20

Clerical Composites -- Existing vs. New

0.80 |

gs

ar

as mk me
ASVAB subtests

OCL(9AA) MECL-1(17) CICL-2(17)

Combat Composites — Existing vs. New

a!

ei

°
2
&

3

Subtest relative weights
3
8

0.20

0.00

gs

mk me
ASVAB subtests

TDICO(@AA) wCO-1(17) CICO-2(17)



Study Report 2002-03

Proposed New Army Aptitude Area Composites: A
Summary of Research Results

Peter M. Greenston
U.S. Army Research Institute

Selection and Assignment Research Unit
Michael G. Rumsey, Chief

U.S. Army Research Institute for the Behavioral and Social Sciences
5001 Eisenhower Avenue, Alexandria, Virginia 22333-5600

January 2002

—— a oT Neen ee eer eer worn
Army Project Number Personnel and Training
20465803D730 Analysis Activities

Approved for public release; distribution is unlimited.

iii



3. Compute b-weights by converting the Beta weights computed in step 2. The b-
weights that are appropriate to apply to operational test scores to obtain a least
squares estimate (LSE) of the criterion can be defined in terms of the Beta weights,
the SDs of the subtests, and the SDs of the criterion scores. These b-weights applied
to the operational test scores would provide a composite that, if the appropriate
regression constant were subtracted, would have a mean of 50 and a SD less than 10
(because of the effects of the positive inter-correlation coefficients among the
subtests). The b-weights are computed, ignoring the regression constants, as follows:

b-weight = B-weight * (SD), / (SD),
where t represents a subtest, SD, = 10, and c represents the criterion variable.

4. The composite computed in step 3 will have a SD less than 10. We wish to convert
this composite to have a SD of 20. To do this we will multiply each b-weight by a
composite multiplier (CM) that will convert the composite to have a SD of 20 without
affecting the composite mean. CM can be computed as follows.

CM =20/(10 * (bRb")"”),

where b is a vector of b-weights and R is the Youth Population matrix of subtest
inter-correlation coefficients.

5. We can now compute the u and k values (i.e., the operational subtest weights) for each
composite:

u; = CM * b-weight of the j-th subtest
k= 100- Xu * 50

Key Formulae From Gulliksen

The algorithms we use to correct for restriction in range due to “selection” effects
are developed and described by Gulliksen 950". His development is based on a
model that visualizes the presence of both explicit and implicit selection processes in the
unrestricted population, and the presence of both explicitly and implicitly selected
variables in the restricted population. Thus, both explicit and implicit variables are
present in both the unrestricted and restricted populations. The author shows, in the
context of this model, relationships among the restricted and unrestricted variances/co-
variances without relaxing flexibility as to which population contains the unknowns that
cannot be directly computed but can be determined on the basis of the relationships
defined in his model.

The Gulliksen formulae for correcting variances and/or co-variances for
restriction in range effects are based on Lawley’s (1943) assumptions that include the
following: (1) that the regression of the implicitly restricted variables on the explicitly
restricted predictors is linear; (2) that the co-variance of the restricted variables exhibit
homoscedasticity; and (3) that the G-weights for application to the population variance-

21 See H. Gulliksen, Theory of Mental Tests. New York: John Wiley & Sons, 1950.

D-3



Appendix D
Restriction in Range and the Estimation of AA Composites

Introduction

This appendix will focus on how to obtain the AA composite regression weights
(referred to as “u and k” values) for operational use in the applicant Youth Population. »
The validity coefficients we wish to maximize in the Youth Population actually exist only
in doubly restricted MOS samples containing the Skill Qualifications Test (SQT)
criterion in the 1987 - 1989 research data set. Appropriate corrections have to be made to
these restricted validity coefficients to obtain unrestricted validity coefficients that, if
subjected to restriction in range effects, would equal what was obtained in the MOS
samples. We also have to estimate what the criterion standard deviation (SD) would have
to be in the unrestricted population to yield the criterion SDs observed in the MOS
samples.”°

The Army operational process involves an applicant Youth Population from
which self-selection first occurs, and then the Recruiting Command selects some and
rejects others using tests, medical examinations, security investigations etc. This results
in an Army Input Population from which classification and assignment procedures and
further self selection create the 150 MOS samples, each with its separate SQT criterion
measure. Thus there is a selection stage and a classification and assignment stage, with a
restriction in range effect on both test scores and hypothetical criterion scores occurring
at both stages. If we confined selection effects to the impact of the AFQT screen, the
selection and classification stage effects would have to be corrected in a sequential
manner. However, since we are not restricting ourselves to such a limited selection
effect, and are instead considering all effects on the subtest co-variances at each
restriction stage, we can correct validity coefficients and criterion SDs directly to the
Youth Population.

Our correction process for restriction in range involves contrasting, separately for
each MOS, the within-MOS subtest variance/co-variances against the Youth Population
operational test variance/co-variances. The differences in the variance/co-variances
across the unrestricted and the restricted samples for variables specified as explicitly
selected variables are the measures of the magnitude of the restriction effect. For our
purposes we use all ASVAB subtests as the explicitly restricted variables and we
designate the criterion variables as the implicitly restricted variables that are restricted to
the extent that they are predicted by the explicitly restricted variables.

Using this concept we can calculate the effect selection has on subtest scores and
can then calculate the further effect classification and assignment has on test scores in the

19 This appendix has been prepared by Cecil Johnson, consulting research psychologist.

29 It should be noted that whenever validity coefficients are mentioned, we are assuming that these
coefficients have been corrected for attenuation with respect to criterion unreliability. Even if we should
refer to an uncorrected validity coefficient (for restriction in range), this “uncorrected” coefficient has been
corrected for attenuation.

D-1



The results depicted in Table 2 (with the exception of the last row) are based on a 9-subtest
ASVAB battery — they precede the DoD decision to delete the Numerical Operations and Coding
Speed subtests from the battery. The last row indicates the overall loss in classification efficiency
from deletion of these subtests. We would expect that the MPP estimate of a 9-LSE composite based
on a 7-subtest battery would be reduced in the same proportion.

In the simulation experiments described, the recruit is assumed to choose the job opportunity at
the top of his/her list. In reality, classification gains are affected by the behavior of recruits choosing
among an ordered-list of job training opportunities. To examine the effects of job choice behavior |
upon classification gains, the research team conducted an additional set of simulation experiments.”
They compared choosing from the top-of-the-list job versus a 75% probability of choosing one of the
top five jobs on the ordered list --- deemed to be a reasonable depiction of job choice behavior in an
operational setting --- and found a reduction of .072 in MPP. In this case, the classification gains
obtained (i.e., MPP = .121) are still more than, five times those achievable using optimization
procedures with the existing AA composites.'?

Table 2: Estimated MPP — Results of Simulation Experiments

Family size Unbiased MPP
AA — 9 operational families (unit weights) .023
10 (LSE weights) 123
13 (LSE weights) 138
17 (LSE weights) 145
150 (LSE weights) 195
150 (LSE weights; 7 ASVAB subtests) 183

Source: Zeidner, Johnson, Vladimirsky, and Weldon (2000a, Table 3) and (2000b, Table 3).

2 These experiments are described in Johnson, Zeidner, Vladimirsky, and Weldon (1999).
‘3 The monetary value of these classification gains can be estimated as (what economists call) the opportunity cost of
retaining the current AA system. In the present context of an optimizing assignment procedure, this is the additional cost of
using current AA composites to achieve the same level of performance gains obtainable using the new composites.
Specifically, using current AA composites, how many additional 1-3A recruits, in place of 3B recruits, would be required
to achieve the same gains obtained using the new composites, and what would it cost to acquire them?

The heart of the opportunity cost calculation is determination of the number of additional 1-3A recruits required.
Using the 1997 accession cohort as a baseline, the assignments made using the current procedures are ordered from high to
low by AFQT score. For individuals at each percentile score, average and cumulative average predicted performance
scores for the job assignments actually made are calculated. To meet a predetermined overall average performance target,
individuals from the bottom are successively deleted and replaced with 1-3A recruits (assumed to score at the original 1-3A
average) until the performance target is reached. We find that that the percentage of 1-3A recruits would have to rise 10 -
15 percentage points and would conservatively cost in the neighborhood of $100 million in additional recruiting costs.

Calculations are made for a cohort size of 72,000, with 1-3A recruits comprising about 68%. Average recruiting
costs are $11,660 for high-quality and $6,223 for low-quality recruits. Marginal costs are estimated at $35,555 for high-
quality recruits, and assumed to increase with high-quality share (each one percent increase in share is associated with a one
percent increase in marginal costs). For example, at 80% high-quality share, the average cost has increased to $14,935 for
high-quality recruits. Recruiting costs refer to 1995 (Source: USACEAC Army Manpower Cost System).

7



Fairness of the New Composites

As part of the development of the new composites, the ARI research team has also examined
the fairness of the new composites from a gender and race perspective (Zeidner, Johnson, Vladimirsky,
1998). Fairness is traditionally defined as the absence of under-predictions for minority groups for
which discrimination potentially exists. In other words, a situation in which the predicted performance
of minority group members falls below their actual performance would call into question the fairness
of the prediction instrument -- in this case the test composites.

The fairness of the new vis-a-vis the existing AA composites was examined using data based
on actual assignments of recruits and data from simulation experiments. The sample in both
experiments used ARI’s data set of ASVAB subtest and SQT scores.'* In the actual assignments
experiment, a distinct trend of under-predictions was found for females and blacks for both existing
and new composites. While the prediction errors for females were statistically significant, the errors
for females and blacks were too small to have practical significance. In the simulation experiments,
again there was a consistent trend of under-predictions for minorities, the new vis-a-vis existing
composites resulted in smaller errors, and the small under-predictions were considered of little
practical consequence. In sum, the new composites are at least as fair as the existing composites.!

Recommended New Composites and Job Family Structure

The classification research findings suggest that new (predicted performance) composites are
superior to the existing unit-weighted composites. Furthermore, adoption of new composites and
corresponding job families within an enhanced REQUEST system holds the promise of dramatic
increases in soldier performance. A two-tier structure is recommended for operational use.

A visible tier is comprised of 17 operational families, and would be used for administrative,
counseling, and school proponent purposes just as the existing nine operational job families are used.
(An invisible tier is described on page 9.) Minimum eligibility standards (cutoff scores) would be
established against these new composites. The visible tier is consistent with the CMFs currently used
by the Army in managing the entry-level job structure and is also consistent with the current AA
system. No pair of MOS that are together in the current AA system fails to be together in the 17 job
family system. The visible tier strongly resembles the existing structure, in effect being a further
shredding of certain existing families. Four of the nine existing job families are divided into two
relatively homogeneous sets of clusters, two are divided into three sets of clusters, and three remain as
single clusters. Provisional MOS membership in the set of 17 job families is shown in Appendix B.
Note that recently created MOS remain to be inserted into their appropriate job family set.

‘* The sample was limited to soldiers having the task-based written test, skill level | SQTs for 66 MOS obtained during
FY 1987-89. The sample was further limited to enlistees who had taken the ASVAB in its current format. The total sample
size in the first experiment was 83,000 enlistees and the size of the sample for the second experiment was a stratified
random sample of 30,000 first-term enlistees drawn from the larger sample.
‘5 Additional research into fairness issues is underway in response to recommendations made by an expert panel convened
in September 2001 to review ARI’s classification research.

9



REPORT DOCUMENTATION PAGE

Form Approved
OMB No. 0704-0188

Pubic reporting burden for this collection of information ie estimated to average 1 hour per response, including the time fo
pethcring end maintairing the date needed, and completing end reviewing the collection of information. Send commmerns
a reticd of information. including suggestions for reducing this burden, to Washington Headquerters Services, Director

viewing Instructions, searching existing date sources,
ing this burden estimate or any other aspect of this

ratio
Sree eanway, Sute 1204, Arlington, VA 22202-4302, and to the Office of Manegement and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503.

mation rts, 1215 Jefterson

Oper

1. AGENCY USE ONLY (Leave blank) 2, REPORT DATE

3. REPORT TYPE AND

DATES COVERED

January 2002 Final, January 2001 - December 2001
4. TITLE AND SUBTITLE 5. FUNDING NUMBERS
Proposed New Amy Aptitude Area Composites: A Summary of Research Results 20465803D730
Work Unit 263

6. AUTHOR(S)

Peter M. Greenston (U.S. Army Research Institute)

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

U.S. Amy Research Institute for the Behavioral and Social Sciences
Atta: TAPC-ARI-RS

5001 Eisenhower Avenue

Alexandria, VA 22333-5600

8. PERFORMING ORGANIZATION
REPORT NUMBER

9. SPONSORING / MONITORING AGENCY NAME(S) AND ADDRESS(ES)

U.S. Army Research Institute for the Behavioral and Social Sciences
5001 Eisenhower Avenue
Alexandria, VA 22333-5600

10.SPONSORING / MONITORING
AGENCY REPORT NUMBER

Study Report 2002-03

11. SUPPLEMENTARY NOTES

12a. DISTRIBUTION / AVAILABILITY STATEMENT

[Approved for public release; distribution unlimited.

12b. DISTRIBUTION CODE

13. ABSTRACT (Maximum 200 words)

Armed Services Vocational Aptitude Battery (ASVAB) subtests in
inefficient for classification. At the end of 2001 the Office of the

|job families and corresponding
urposes. The new structure strongly

existing nine operational job families
January 2002 while planning for the new 17 Army

The Army currently employs nine Aptitude Area (AA) composites to classify new recruits; they are derived from the
a manner that makes them easy to calculate but
t Secretary of Defense (OSD) will eliminate the two timed
subtests in the ASVAB . This will reduce the classification efficiency of the existing Army composites and necessitate
redefinition of the existing composites. In their place the Army is considering for adoption
developed by ARI based on a job performance criterion. ARI has developed a set of 17
composites that would be used for administrative, counseling, and school proponent
resembles the existing structure, in effect being a
families. The new composites / job families are undergoing further testing and evaluation,
implementation in the 2004 - 2005 period. In the meantime ARI has developed an interim
but are also based on defensible job performance data. These will be implemented
Aptitude Area composites goes forward.

new composites that have been
operational Classification-efficient

further shredding of existing
and will be considered for
set of composites that retain the

14, SUBJECT TERMS

Armed Services Vocational Aptitude Battery (ASVAB); Amy aptitude area composites;

15. NUMBER OF PAGES
49

personnel classification 16. PRICE CODE
17. SECURITY CLASSIFICATION |18. SECURITY CLASSIFICATION |19. SECURITY CLASSIFICATION |20. LIMITATION OF ABSTRACT
OF REPORT OF THIS PAGE OF ABSTRACT
Unclassified Unclassified Unclassified Unlimited

NSN 7540-01-280-5500

‘Standard Form 298 (Rev. 2-89)
Prescribed by ANSI Std. Z39-18 298-102

USAPPC V1.00



APPENDIX C
9 AA Composites vs. 9 LSE (Interim) Composites

1. 9 LSE composite validities

2. 9 LSE composite correlations

3. Nine bar charts comparing existing 9 AA composites with interim
9 LSE composites

a. Existing composites shown in gray

b. Interim composites shown in black

c. Job families / aptitude areas are as follows: Clerical (CL);
Combat (CO); Electronics Repair (EL); Field Artillery (FA);
General Maintenance (GM); Mechanical Maintenance
(MM); Operators / Food (OF); Surveillance /
Communications (SC); Skilled Technical (ST)

d. ASVAB subtests are as follows: general science (gs);
arithmetic reasoning (ar); auto & shop information (as);
mathematics knowledge (mk); mechanical comprehension
(mc); electronics information (ei); verbal (ve); coding speed
(cs); numerical operations (no)

C-1



=

APPENDIX A
9 AA Composites vs. 17 LSE Composites

17 LSE composite validities
17 LSE composite correlations

. Nine bar charts comparing existing AA composites with new 17 LSE

composites

a. Existing composites shown in gray

b. New composites shown in black and white

c. Job families / aptitude areas are as follows: Clerical (CL); Combat
(CO); Electronics Repair (EL); Field Artillery (FA); General
Maintenance (GM); Mechanical Maintenance (MM); Operators / Food
(OF); Surveillance / Communications (SC); Skilled Technical (ST)

d. ASVAB subtests are as follows: general science (gs); arithmetic
reasoning (ar); auto & shop information (as); mathematics knowledge
(mk); mechanical comprehension (mc); electronics information (ei);
verbal (ve); coding speed (cs); numerical operations (no)


U.S. Army Research Institute
for the Behavioral and Social Sciences

A Directorate of the U.S. Total Army Personnel Command

EDGAR M. JOHNSON
Director

Technical Review by

Peter Legree
Myron A. Fischi

NOTICES

DISTRIBUTION: Primary distribution of this Study Report has been made by ARI.
Please address correspondence concerning distribution of reports to: U.S. Army
Research Institute for the Behavioral and Social Sciences, Attn: TAPC-ARI-PO, 5001
Eisenhower Ave., Alexandria, VA 22333-5600.

FINAL DISPOSITION: This Study Report may be destroyed when it is no longer
needed. Please do not return it to the U.S. Army Research for the Behavioral and Social
Sciences.

NOTE: The findings in this Study Report are not to be construed as an official
Department of the Army position, unless so designated by other authorized documents.



(68X _[AH-64A armament/electrical systems repairer 67) 5
13D Field artillery tactical data systems specialist 13 6
25R__|Visual information equipment operator/maintainer 25 6
25V Combat documentation/production specialist 25) 6
31F Network switching systems operator/maintainer 31 6
31P _|Microwave systems operator/maintainer 314 6
31S __|Satellite communication systems operator/maintainer 31 6}
31U Signal support systems specialist 31 6)
35Y _|Integrated family of test equipment operator/maintainer 35) 6
93F Field artillery meteorological crewmember 13 6
96H Imagery ground station operator 96) 6
96R Ground surveillance systems operator 96) 6
27E Land combat electronic missile system repairer 35) 7
27G__|Chaparral and Redeye repairer 35) 7
27M ___|MLRS repairer 35) 7
27T _|Avenger system repairer 35) 7
35B LCSS test specialist 35] 7
35D __|Air traffic control equipment repairer 35) 7
35E_ _|Radio and communications security repairer 35) 7
35F Special electronic devices repairer 35 7
35H Test, measurement, and diagnostic equipment maintenance 35) 7
support specialist
35d Computer/automation system repairer 35, 7
36L Avionic communication equipment repairer 35 7|
35M__|Radar repairer 35 7\
35N __|Wire systems equipment repairer 35) 7
35R___|Avionics systems repairer 35) 7
39B _|Automatic test equipment operator/maintainer 35 7
45G ___|Fire control repairer 63) 7
68J Aircraft armament/missile systems repairer 67 7
68N Avionic / mechanic 67 7
68S _|OH-58D armament/electrical/avionics systems repairer 67 7
68Y _|AH-64D armament/electrical/avionic systems repairer 67 7
13B__|Cannon crewmember 13 8
13C __|Tactical automated fire control systems specialist 13} 8
13E Cannon fire direction specialist 13} 8
13F Fire support specialist 13} 8
13P___|MLRS operations/fire direction specialist 13 8
44B _|Metal worker 63} 9
44E  |Machinist 63) 9
45B _|Small arms/artillery repairer 63) 9
45D Self-propelled field artillery turret mechanic 63 9
45K —__|Armament repairer 63) 9
45T Bradley fighting vehicle system turret mechanic 63 9
52C Utilities equipment repairer 63 9
52D _|Power-generation equipment repairer 63) 9
43M ___|Fabric repair specialist - del 0110 92) 10
51B _|Carpentry and masonry specialist 51 10
51K |Plumber 51 10
51M ___ {Firefighter 51 10)

B-2


Subtest relative weights

Subtest relative weights

Operators / Food Composites -- Existing vs Interim

1.200

4.000

0.800

0.600 +

0.400

0.200 +

0.000

4,200

mk

‘Surveillance / Communications Com posite ~ Existing vs. Interim

me
ASVAB subtests

TOF(9AA)

F(SLSE)

no

cs

1.000

0.800

8

2
8

0.200 f=

0.000

mk

me
ASVAB subtests

SC(SAA) (SLSE)

os



Electronics Repair Composites - Existing vs. New

04 t.

Subtest relative weights

0.8 ++}

0.6 +

1.20

gs

ar

Field Artillery C omposites ~ Existing vs. New

as mk
ASVAB subtest:

DEL(@AA) WEL-1(17) EL-2(17) DEL-3(17)

ei

ve

1.00

0.80

0.60

0.40

Subtest relative weights

0.20

0.00 4

gs

a as

mk

me

ASVAB subtests

FA(SAA)

17)



We can illustrate how the new (classification-efficient) composites differ from the existing AA
composites. In the earlier discussion we pointed out the difficulty of distinguishing between the
existing Operators / Food and Surveillance / Communications composites because they share three
ASVAB subtests in common (AS, MC, VE). However, when we look at the new composites (see
Charts 1A and 1B) we see notable differences.* In the first place, the four top contributors in new
Operators / Food are AS, AR, MC and VE; now AS, MC, and VE are the core of the existing
composite, while AR has a very important role in new Operators / Food but is not part of the existing
operators composite. Second, the four top contributors in Surveillance / Communications are MK, VE,
AR, and EI; only AR and VE are part of the existing composite. Indeed, MK plays a defining role in
the new Surveillance / Communications composite but is not part of the existing composite.

e Three: Classification efficiency should be evaluated by measuring soldier mean predicted
performance (MPP) in job assignments made within classification optimization experiments.

The simulation experiments utilize Army job performance and ASVAB data circa 1989:
individuals are assigned so as to maximize predicted performance while meeting MOS quotas and
certain other distributional constraints (but stopping short of the complete set of training management
constraints). In these experimental simulations assignments are made on an entity-by-entity basis
while avoiding the assignment of an entity to any job family or MOS for which the quotas have been
filled within a batch or sub-sample. The predicted performance of each entity in his/her assigned job
family is computed, and the average or mean predicted performance is computed.

The experiments were conducted using a “triple cross validation” design to assure unbiased
estimates. The design requires three independent samples: (a) an analysis sample, for estimating the
predicted performance function used during the assignment process; (b) cross samples which serve as
the source of entities that are optimally assigned in the simulation; and (c) an evaluation sample, for
estimating the predicted performance function used in the evaluation of the assignments made. Each
experiment is conducted twice, with the role of analysis and evaluation samples reversed, and the
results averaged.

The results of the classification simulation experiments are expressed in MPP and shown in
Table 2. By comparing the 10 job family? new composite results (MPP = .123) with the existing AA
family results (MPP = .023) we see that the new composites provide more than five times the
classification power as measured by MPP. '° Tn other words, improved accuracy in the estimation of
predicted performance, utilizing only the existing job families, promises large gains in classification.
And when 150 job families are identified, with the greater homogeneity of these families and greater
overall distinction between them, the classification gains increase to more than eight times. Indeed, we
estimate that the performance gains obtainable from improved classification would rival in size those
obtained from existing selection-for-service.!!

5 This illustration is taken from a 17 job family structure proposed as the second tier and described below.
A very small General Technical (GT) family is included; otherwise, structure is same as existing operational families.
© Random assignment would result in an MPP of zero.
| That is, the performance gains due to improved classification are estimated to exceed those obtained from the screening
out of low AFQT applicants. See Zeidner, Johnson, Vladimirsky, and Weldon (1997).
5



‘Subtest relative weights

08

06

04

02 4

Chart 2. Mechanical Maintenance Composite - Existing vs. New

ar

as mk me ei
ASVAB subtests

[BIMM (existing AA) BMN- (new 17) OMM.2 (new 17) |

’

no