bptitude-asse

| ‘circumstan:
| iards.
Relative .
The rar
} 7-Test Cor
i and FAA
| (Sample £
' peven con
aptitude |
Air Force
relatively
| the meant
nonsignifi:
' high scor
fact, the
higher th:
Inasmu
selected f
expected
quite wel
ever, ther
mean cor
tary gro
FAA Sai
1961, mo
from am
military
sumed t]
group i
cerned.
TATC 1

_—— _

Commer. 7-Test 4
Composite Score |

3600 7

3400 7

3200 “

3000 _

2800 “

2600 =“

2400 “

2200 4

2000 7

1800 a”

1600 7 ,
Sample 1 3 4
Type FAA Arny USAF
N Cases 212 169 169
Mean 7-Test Score 2710.9 2881.2 2842.1
SD of 7-Test Sg. 503.0 425.8 444.5

Freure 4, Comparative Ranges and Distributions of Commercial 7-Test Composite Scores for FAA, Army, Air
Force, Navy and Marine Corps ATC Trainees.
18

the enti
schools

personne
improve



P

TABLE III. COMPARISON OF RESULTS OF MULTIPLE REGRESSION ANALYSES WITH VALIDITY COEFFICIENTS OF
COMPOSITE SCORES DERIVED THROUGH APPLICATION OF SIMPLIFIED” FACTOR WEIGHTS

Multiple R for Each Composite Versus Training-Course Grade
(R te the maximum validity theoretically possible for each composite.)
Sample 1 Sample 2 Sample 3 Sample 4 Combined Samples 5 & 6
FAA FAA Aray Air Force Navy and Marines
Type of Composite N22 N=183 162 —Nn169_ in!
Commercial 7-Test Composite 49 +40 57 wks
(3 DAT's & 4 CTMM subtests)
Commercial 4-Test Composite 45 +33 +32 4 43
(3 DAT's + CTMM Analogies)
New Commer. 4-Test Composite 45 234 5S 43
(3 DAT's + CTM Inference
Corresponding Validities of Composite Scores Derived
Sample 1 Sample 2 Sample 3 Sample 4 Sauple 5 Sample 6
FAA FAA Aruy Air Force Navy Marines
Typs_of Composite wNe2h2_ Ne183__ e162 e626 _ +) a
Wed. 7-Test Commer, Comp. 39 37 52 41 41
Wed, 4-Test Coumer. Comp, +32 +52 +27 49 38 al
Wed. New 4-Test Composite +36 +33 52 Al wal

=
* Both the 7Test and 4-Test Composite Scores were derived by factor weighting each test by a two-digit value

based on the inverse of its standard deviation obtained with other and larger samples.

Composite, the weights repri
weights to single digit values.

weights had been utilized, the resulting composite
scores would have yielded validity coefficients
of only slightly-greater ‘magnitude than found
for the composite scores based on the inverse of
the standard deviation of performance measures
for each test.

Intercorrelations and Validities of Composites
and MSC Scores

The interrelationships of the three experi-
mentally-derived composite aptitude measures
and the MSC test scores of each military group
are presented in Table IV. Although previously
presented in Tables II and III,’ the validity
coefficients of the various composites and the
MSC scores are also shown in Table IV. For
the Army and Air Force trainees, the correlations
between the MSC scores and the composites
ranged from .52 to .59. For the Navy sample,
the “GCT-+ARI” score correlated no less than
.64 with any commercial composite, whereas the
Marige Corps MSC correlated .46 with each of
the abbreviated composites and only .51 with the
Commercial 7-Test. nposite Score. Within the
context of the present study, the correlations of
.51 and less should be considered as “moderately
low” and those above .60 as “moderate-to-good.”

12

For the New 4-Test

ted a aimple and proportionate reduction of the original set of two-digit

Two multiple-regression analyses were accom-
plished on each matrix of intercorrelations to
determine the potential with which a summary
measure based on both the MSC and the seven-
test composite measures might be used to enhance
prediction of the two criteria. The results of
these analyses are presented in Table V. For the
Army and the Navy trainees, each “R” (or esti-
mate of validity) for the theoretical summary
variable only slightly exceeded the validity co-
efficient of the MSC measure (alone). Yet, in
each instance, the difference was statistically
significant. In c6ntrast, “R’s” of .52 and .48
(with Course Grade and PI) were obtained for
the Air Force sample whereas the MSC index
had yielded corresponding validity coefficients of
35 and .36. Similarly, the analysis of data for
Sample 6 revealed that Course Grade, which had
correlated .21 with the Marine MSC score, would
correlate .41 with a variable reflecting perform-
ance on both batteries, and that the P-F cri-
terion, for which the MSC score yielded a validity
coefficient of only .12, would correlate 23 with
the combination score.

In conjunction with each multiple-regression
analysis just described, procedures were also
employed to extract from the Commercial 7-Test

TABLE

Sample 3 -
Commer.

New 4-1
MSC -
Course

Sample 4
Commer

Commer
New 4-
MSC -

Course

Sample 5
Comme1

Comme!
New 4
MSC-GI
Cours:

Sample 6
Comme



woyangz3z3e}q Jo Syazzg-rn0g reddp uz ss a-teurtzey A |

peyeum e ‘SMINSLAAL PUBUOFEd OSINOD JUTUVIL-OLVY AAV J9[$eeW JO sioy}pord su so10og wal OLV-OSO
HSH Pus opsoduey yseal-— TepsMMOD ‘eysodu0p ysaz-, TeOrEMUMON ‘xepul ,epndy Te1eUH GOV. OSW exo ITY Jo uosttedmoD -, wHNOLT

aioog 3sel apnatady D1V-9S peIewrIsyT a109g aatsodmoy Asay-y Te TOIaWMMI0D
<8 67Z- 61Z- 60Z- 661- 68I- 64T- 691- > < 8 678T- 67LT- 6791-6YST- 6YVI- 6YET- 6YZT- > 2
O€Z O22 O12 002 O61 O8t OLT O9T 6ST OSBI OSZT OS9T OSST OSHT OSET OSZT OSIT 6¥TT
. oO”
691 - N 691 - N
405
a
a
a10dg aytsodulog 3SaZ-/ TPFOTSMMIOD xapuy epnjzidy, Tezsuag, gov 20107 ITV
< 9 6YSE- GREE- BYIE- 676Z- 6HL7- 67SZ- 6NEZ- > <3. * >R
OSSE OSEE OSTE OS6Z OSZZ OSSz OSEZ OSTZ 64IZ $6 06 S808 SL OL 1 $9 09 ss
T
9 ot
ST
Oz
- o€
9€ } 9€ or
691 - N 69t - N
Os
SIpbAN aS1no0g-Zuyuyzeay, OLY 1918294 JO o saseg [Tey 10/pue
UoFINg FAI SQ JO SYIZFZ-InN0g reddy ut sg Sseq-[eupZIEy .



ite measures, derived
on five, six, or seven
m .35 to 54 (Pear-
ts*). More import-
that about 70 to 80
as training-course
ees, represented in
bution of scores de-
roup of tests, The
ig over 30 per cent
vainees were former

tests for screening
id, when similar re-
itional samples, the
Commission agreed
hould be employed,
lection of some of
ts. However, com-
not used because

to approximate a
ercial tests. Since
t (ATP) had been
FAA and was still
officially adopted
CAMI researchers
other CSC tests
to administer and

961, all incoming
inees were experi-

wailable for each
amination of each
reviously-validated
‘However, time be-
iC tests was avail-
f a portion of the
ion analysis of all
in time indicated
ercial Seven-Test,
with a summary
f the seven. The
later, were there-
CSC tests in the

Follow-up studies of Academy trainees exam-
ined with the revised battery during the next ten
months revealed that composite scores based on
five of the CSC tests and ATP could be used
effectively to predict training outcomes. Com-
posite scores of 190 and higher were attained by
approximately 55 per cent of all the examinees.
Of these, about 70 per cent successfully completed
their training course and were certificated as Air
Traffic Control Specialists (ATCSs). In con-
trast, almost 75 per cent of those with scores of
189 and lower failed to graduate and were elim-
inated from further FAA training. These re-
sults approximated those obtained in earlier
analyses (with other groups) for the Commercial
Seven-Test Composite. Three of the tests in-
volved in the Six-Test CSC Composite were
those which had been selected as “counterparts”
of three commercial tests; these instruments
provided measures of numerical, spatial, and
non-verbal abstract-reasoning abilities. The new
composite also included the ATP Test, an instru-
ment known as “Letter Sequence” which also
measured reasoning ability, and a test of fol-
lowing oral directions.

Although CAMI was requested to continue its
experimental testing program and obtain addi-
tional validation data, the Civil Service Com-
mission and the FAA agreed that the six CSC
aptitude tests should be used to select one-third
of the ATCS trainees from among those appli-
cants who could not qualify on the basis of pre-
vious job-related experience. The procedure was
officially implemented in May 1962. During the
ensuing 17 months several thousand applicants
were examined with the battery, but about half
of them were unable to attain passing scores of
190 or higher. Moreover, at about the time the
battery was adopted for limited use, budgetary
limitation prompted a drastic reduction in the
recruitment of ATC personnel. Consequently,
the FAA continued to select an overwhelming
majority of its trainees from among those having
fully-qualifying amounts of pre-FAA experience.
Only a few hundred of those who passed the
CSC aptitude screening battery were appointed
to training and most of them possessed some,
though not fully-qualifying, ATC-related experi-
ence, .

CAMI continued to administer the CSC test
battery, on an experimental basis, to all incoming
classes of the Academy’s basic-training courses.

The effectiveness of the CSC battery was con-
clusively demonstrated when an analysis involv-
ing 893 cases revealed that 182 (or 67.1 per cent)
of 271 training-course failures were unable to
attain composite scores of 190 or higher whereas
only 222 (or 37.5 per cent) of the 622 pass cases
did so.

These results prompted a revision in the selec-
tion standards. Beginning in January 1964, the
CSC battery was incorporated in the screening
of all applicants—regardless of their pre-FAA
experience. Aside from other factors, eligibility
required a composite CSC score of at least 210.
Retention of a screening score of 190 was con-
sidered, but 210 was eventually adopted because
it was contemplated that a further reduction in
the number of trainee positions would be neces-
sary pending an increase in Congressional ap-
propriations. Personnel selected under these
standards during the next four years and nine
months were insufficient to offset attritions within
the air traffic management system. In the sum-
mer of 1968, however, the President announced
that an expanded program of recruitment and
training was necessary to preclude a critical
shortage of ATC personnel. The program was
implemented in October 1968,

Under the new standards, which are still in
use, individuals with exceptional amounts and
types of pre-FAA ATC experience, and par-
ticularly in radar-control work, can be granted
an exemption of the aptitude requirement and
also be hired at higher-than-normal pay grades.
About one-fourth of those appointed to training
since November 1968 have established their
eligibility in such a manner. Although there are
some other standards whereby education and
certain typés of other experience may warrant
waiver of the aptitude screening requirement, an
overwhelming majority of the trainees are se-
lected from among those applicants who have
attained a score of 210 or higher on the CSC
battery and who, in most instances, also possess
somé ATC-related experience. Consequently, an
applicant pool of former military air traffic con-
trollers has continued to represent the prime
source for the selection of FAA ATC trainees.
Aside from other reasons, these recruiting prac-
tices have prompted the FAA to maintain an
interest in the effectiveness of the ATC selection-
and-training programs of the Air Force, Army,
Navy, and Marine Corps.



P75

Median

P25

Pro

Po2

6
Marines
317
3163.7

382.1

4

aptitude-assessment battery may have been too

On the basis of the data presented in Figures
and 3, it would seem that any effort by the
rmy, Air Force, Navy, or Marine Corps to
prove the aptitude screening of personnel for

whould additional tests, such as represented in
the experimental battery, be considered for
specondary screening purposes. However, the
‘efficacy of any selection program—regardless of
‘jis nature and scope—will depend upon the ap-
‘propriateness of the levels at which the screening
‘standards are established and the degree to which
circumstances permit adherence to those stand-
ards,

: Relative Aptitude Levels of Different Samples
The range and distribution of Commercial
7-Test Composite Scores for each military group
and FAA Sample 1 are presented in Figure 4.
(Sample 2 was administered only four of the
seven commercial tests.) In comparing the
aptitude levels of the various groups, the Army,
Air Force, and Navy trainees were found to be
relatively homogeneous; all differences between
the means and variances of these groups were
nonsignificant. Most of the Marines attained
high scores on the experimental battery. In
fact, the mean for this group was significantly
higher than that obtained for any other.
Inasmuch as the majority of the Marines were
selected from high MSC-score categories, it was
expected that they would also tend to perform
quite well on the experimental battery. How-
ever, there was no basis for anticipating that the
mean commercial composite score of every mili-
tary group would significantly exceed ‘that of
FAA Sample 1. Although recruited in 1960 and
1961, most of these FAA trainees were selected
from among those applicants with experience as
military controllers, and it was therefore as-
sumed that they also represented a rather select
group insofar as certain aptitudes were con-
cerned. The finding that these former FAA
TATC trainees had lower aptitude scores than

for FAA, Army, Air

the entrants Ynto the military ATC training
schools indicates that the screening of military
personnel for ATC training had substantially
improved during the intervening period.

19

Figure 5 is similar to Figure 4 but pertains to
the Commercial 4-Test Composite Scores which
were available for Sample 2, as well as for all
other samples. Differences between the means
and variances of the two FAA groups were not
statistically significant. However, both group
means were significantly lower than that obtained
for any military sample. Other findings gen-
erally paralleled those reflected in Figure 4.

Figures 6, 7, 8 and 9 pertain to the Army, Air
Force, Navy, and Marine samples, respectively.
Each figure consists of four graphs illustrating
the effectiveness with which the MSC score, the
Commercial 7-Test Score, the Commercial 4-Test
Composite Score, and the Estimated CSC-ATC
Aptitude Test Score could have been used to
forecast training course performance. AI find-
ings except those relating to the Estimated CSC-
ATC Aptitude Test Score (which will be
explained shortly) have heretofore been dis-
cussed. The figures are presented for summary
and comparative purposes but without further
comment at this stage of the report.

Estimated Performance of Military ATC
Trainees on the CSC ATC Aptitude Test Battery

Although a number of objectives were set forth
at the outset of the study, the FAA’s primary
interest was to obtain a reliable estimate of the
proportion of each military group that would
be able to qualify on the CSC-ATC Aptitude
Screening Test Battery and the extent to which
the MSC Scores of such personnel might be used
to predict their qualification-failure status on the
CSC battery. Had policy reasons not precluded
experimental use of the CSC battery at the mili-
tary facilities, such information could have been
obtained insa rather simple and straight-forward
manner. However, indirect methods were neces-
sary and consequently there are several issues
concerning the reliability of the predicted CSC
Scores and related findings which warrant con-
sideration.

Sample 2 (183 subjects) represented the only
group of former FAA TATC trainees for which
data were available to establish a relationship
between the CSC battery and any sizable portion
of the commercial battery. The reliability of
the prediction equation is therefore contingent
upon the reliability of the correlation of .81
between the Composite 4-Test Scores of these
subjects and their CSC scores. By applying


Estimated CSC-AT

Commercial 4-Test Composite Score

Figure 9. Comparison of Marine Corps MSC “% (AR+PE+VA)” Score, Commercial 7-Test Composite, Commercial 4Test Composite and Estimated

CSC-ATC ‘Test Scores as Predictors of Glynco NAS Training Course Performance for Marine Corps ATC Trainees.

standardized statistical procedures,’ a standard
error of .025 for the “r” of .81 was obtained. In
other words, if scores of both types had been
available for additional TATC samples, the
chances are about two to one that the resulting
correlation coefficients would range between .785
and .835. By employing somewhat similar pro-
cedures,‘ the standard error of the estimate, or
margin of error in prediction of the CSC scores,
was determined as 23.6. This index indicates
that if the prediction equation were applied to
the sample which provided the data for its deri-
vation, the differences between the predicted
values and the observed (i.e, actual) measures
of performance on the CSC battery would not
exceed 23.6 points for two-thirds of the subjects.
Due to “regression-toward-the-mean” effects,
most errors of greater magnitude would gener-
ally represent either “under-estimates” of very
high CSC scores or “over-estimates” of very low
performance measures. Such effects, however,
should be considered less important when pre-
dicting qualification-fail status on the screening
battery than when attempting to predict each
subject’s level of performance.

After predicting the CSC scores of all subjects
of Samples 1, 3, 4, 5 and 6 from the Composite
4-Test Scores, analyses were undertaken with
each military group to establish the relationship
between the predicted scores and the MSC meas-
ures. Although not presented in any table, the
product-moment correlations were: .52 for the
Army sample; .56 for the Air Force group; .63
for the Navy sample; and .46 for the Marines.
Each coefficient was statistically significant at
the .01 level. In each instance, however, the
relationship was deemed insufficient in magni-
tude to asure an appreciable degree of accuracy
in the prediction of CSC perfornfance data from
the MSC measures of future military trainee
groups. Yet, the relationships should be consid-
ered as very substantial. When supplemented
by the data presented in Figure 10, they illustrate
that subjects with relatively high MSC scores are
much more likely to attain qualifying scores of
210 and higher on the CSC battery than those
haWng relatively low MSC indexes. For ex-
ample, qualifying ycores on the CSC battery
were predicted for 71 per cent of the 62 Army
trainees who possessed “V +A” scores of 125 and
higher, compared to 41.4 per cent of the 99 with

MSC scores of 110 to 124, and only 12.5 per cent
of those represented in the lower MSC category.
Results for the Navy and Marine samples were
even more impressive, and those obtained for the
Air Force group followed a similar trend.

In reviewing Figure 10, it may be noted that:
the Army deviated from its recommended stand-
ard in assigning eight men to ATC Training
who possessed MSC scores of 109 or lower; the
Navy failed to adhere to its similar standard in
selecting 104 trainees; the Marine Corps made
only one exception to its identical standard; and
the Air Force dropped below its recommended
standard of 65 in designating ATC training for
25 airmen. The predicted CSC scores for the
majority of these 138 cases were rather low.
Although not directly shown in Figure 10, the
predicted CSC scores for 91 (or 66 per cent) of
the 138 were lower than 190—a score which the
FAA has considered as non-qualifying since
December 1963.

The predicted CSC scores for the FAA train-
ees of Sample 1 were generally lower than those
of any military sample and the mean of the
predicted values for Sample 1 was also signifi-
cantly below the average score of FAA Sample 2
(trainees who were actually administered the
battery before it was adopted for operational
use). Comparative data in this respect are in
Figure 11. By adding the percentages shown
for the two highest score intervals, it can be
determined that scores of 210 and higher were
estimated for only 29 per cent of trainees in
FAA Sample 1. Seventy-one (about 39 per
cent) of FAA Sample 2 trainees (who had
actually been examined with the battery) at-
tained scores above 209. In contrast, qualifying
scores of 210 anfl higher were predicted for 78.7
per cent of the Marines, 50.8 per cent of the
Army trainees, 46.6 per cent of the Navy sample,
and 40.9 per cent of the Air Force trainees,

Inasmuch as the CSC measures were estimated
from the Composite 4-Test Scores, the relative
differences between the groups parallel those
previously shown in Figure 5. However, the

Composite 4-Test Scores, as well as other sum-
mary measures of performance on the commercial
tests, were found to overlap, or correlate, very
substantially with the MSC measures. Moreover,
those subjects with low MSC scores and low
commercial-test composite measures also tended



‘SOOUIVLL OLY AUIY GOT AOZ SaINSvaFT oOUBUAOJIed BSINOO-SUOILIL-CAV IW1S2Y JO Siow perq Se Ser00y
380L OLV-OSO PelwMS| pus azsodwoD ysez-F [BjLoMUIED ‘eysodmog yseJ-) TereuUIED ‘a100§ WV+An OSA sury ‘SO Jo wosiedmop ‘9 Sano

109g YO] epnatady OL¥-089 PpeIPUTIsA a109g a3;s0dmo) 4S2I-y TeyII2uMOD
<9 6%%- 61Z- 60Z- 661- 68I- G6LT- 69T- > <9} 6HBT- 6YLT-679T- 6YST- 67HT-6ET- 6YZI- > 9 i
Oz O22 O12 002 O6T O8t OLT O9T GST OS8T OSLT OS9T OSST OST OSET OSZI OSTT 6vIT

Zz € Zz
S$ 9 a S og s 4 OT
8 6
mW a eq ott 4

61 . 4
oe
ov

W . ,
os

691 -N 691 - N

‘ 09 a
@102§ azTsodmog ISa]-/ [TP FOTIUMOD 2209 ,,V+A, OSH Ary *s°n

<9 6YSE- GYEE- GYTE- 676Z-67LZ- 6YSZ- 6NEZ- > 9 <9 GET- VEI- 6ZI- %I- 6II- YIT- 60I- > 9 :
OSSE OSEE OSTE OS6Z OSLZ OSSZ OSEZ OSTZ 6YIZ Ovl SET OFT SZI OZT STT OTT S@T 0T

691 - N 691 - N
5 os
Saperg asrnog-Bupuzezl OLY 291 s98y JO o seer) [BI 10/pue g
uoFINgFAIsTG JO SyIZTA-3nog aaddq ut ss esbg-[euTs2ey

>.

—
3 5

s 3 a g 2

ty 2 i om i)

245.7
FAA, Army, Air

6
Marines
317

1730.0



Marginal-Pass
and/or Fail Cases

Sa in Upper Four-Fifths of Distribution
Oe Military ATC Training-Course Grades

Sample 3
ad Aray ATC Trainees
N= 169

2149 2150 2350 2550 2750 2950 3150 3350 3550
& < -2349 -2549-2749 -2949 -3149 -3349 -3549 & >
Score on Experimental ly-Administered
Battery of Seven Commercially-Published Tests

Sample 4
Air Force ATC Trainees
N- 169

2149 2150 2350 2550 2750 2950 3150 3350 3550
& < -2349 -2549-2749 -2949 -3149 -3349 =3549 & >
Score on Experimental ly-Administered
Battery of Seven Commercially-Published Tests

100 | Sample 5

Navy ATC Trainees 92
N- 628
80 80
60
40
20

2149 2150 2350 2550 2750 2950 3150 3350 3550
& <_ +2349 -2550-2749-2949 -3149 -3349 -3549 & >
Score on Experimentally-Administered

Battery of Seven Commercially-Published Tests

Sample 6
Marine Corps ATC Trainees
N- 313

2149 2150 2350 2550 2750 2950 3150 3350 3550
& < -2349 -2549 -2749 -2949 -3149-3349 -3549 & >
Score oa Experimental ly-Adminiatered
Battery of Seven Coumercially-Published Tests

10

&

8

100 4

8
80
60
40

21

° 1 |
1
6

Fieure 2. Potential of Commercial 7-Test Composite Score for Prediction of ATC-Training-Course Performance
of U.S. Army and Air Force Students at Keesler AFB and of Navy and Marine Corps ATC Trainees at the

Glynco NAS ATC School.

and/or Fail” and all others as representing the
“upper four-fifths”. Similarly, the groups
trained at the Glynco facility were merged and
all cases not represented in the upper 80 per cent
of the distribution were appropriately designated
either as “Fails” or as “Marginal Passes”. The
Composite 7-Test Scores were then plotted for
the dichotomized groups and cqarse grouping
procedures were subsequently applied to obtain
the data shown in Figure 2 for each of che
samples.

An examination of Figure 2 will reveal that
the “Marginal-Pass or Fail” subjects in Santples
3, 4, and 5 tended to experience considerably
more difficulty with the experimental battery
than, did those who performed more satisfactorily
in the training course. Although less accentu-
ated, a similar tregd characterized Sample 6.
In other words, each graph suggests that tests
such as represented in the experimental battery
could have been used to improve substantially

16

the selection process. For example, approxi-
mately 40 to 50 per cent of the “Marginal-Pass
or Fail” subjects in Samples 3, 4, and 5 failed
to attain a composite score of at least 2550 on
the experimental battery, whereas only 14 to 18
per cent of the upper-category students were
unable to do so, Similarly, about 20 per cent of
the Marines who failed or performed marginally
in training scored less than 2550 on the battery,
compared to 3.4 per cent of all other trainees in
the sample.

However, any recommendation concerning the
possible u&e of such tests for operational purposes
would be contingent upon the degree to which
the screening potential of each existing MSC
aptitude test measure is being realized. Results
bearing upon this issue are presented in Figure 3.
In reviewing the results, it should first be noted
that most of the Army, Navy, and Air Force
students who marginally passed or failed their
ATC course also tended to have low-to-moderate

Figure 3.

MSC sco
fail to re
respect.
ever, the:
ing pote
measure
Adher:
MSC sec
65 for |
selection
failed o:
Even gr
been ack
stances
herence
For exa
per cen
margina
of less
cent) ¢
similar
preclud:



mercial 4-Test Composite Scores. The CSC
6-Test Composite Scores for Sample 2 were ob-
tained in the same manner as prescribed by the
Civil Service Commission. In accordance with
that procedure, the raw scores on four of the
CSC tests were unit-weighted whereas a factor
weight of “2” was applied to Spatial Patterns,
Abstract Reasoning, and Letter Sequence.

TIL. Results and Discussion.

Empirical Validities of the Aptitude Tests

The validities of the various aptitude tests,
when used separately and in combination for
prediction of Training Course Grade, Pass-Fail
Status and/or the Progress Index (“PI”) are
presented in Table II. Similar data relating to
the MSC scores are also shown. In every in-
stance where Pass-Fail (P/F) Status served as
the criterion, a point-biserial correlation coefli-
cient was obtained; for the other criteria, Pear-
son product-moment correlation coefficients were
computed.’ The same is true regarding the cor-
relations presented in all succeeding tables unless
otherwise specified.

Validities of Each Commercial Test

The validity coefficients established with the
various samples for each of the commercially-
published aptitude tests appear in the upper
portion of Table IJ. An appreciable degree of
variation is apparent in the validities of the
respective tests from sample to sample. Yet, all
coefficients except a few of those pertaining to
CTMM Analogies and CTMM NQ Coins proved
to be statistically significant and, within the
context of the present study, most of them should
be considered of substantial and practical value.
The validities established with Sample 4 (Air
Force) appear, in general, to be somewhat higher
than those obtained with the other military
groups. It may also be noted that the scores on
the commercial tests tended to correlate more
highly with Training Course Grade than ‘with
either the PI or P-F criterion. As mentioned

earlier, however, the training course attritions
represented only eight per cent of Samples 1 and
6 afid 12.2 per cent of Sample 5; consequently,
the point-biserial c@efficients obtained with these
samples cannot be considered very reliable.
Although the battery of tests administered to
Sample 2 included only four of the commercially

published instruments, the correlations between
Course Grade and three of the four tests were
higher for this sample than for any other. The
performance scores on each of the four tests by
these 183 TATC trainees (among whom there
were 40 training-course failures) also correlated
at a substantial and statistically-significant level
with the P-F criterion.

The author can offer no explanation as to why
the commercial tests validated better with FAA
Sample 2 than with Sample 1. A selection bias
was not involved because the combined samples
represented the total input for the TATC train-
ing course for a 19-month period. All other
participants in the ATC research program con-
ducted during this period were ARTCC trainees.
Although not presented in this report, the va-
lidities obtained with the ARTCC groups for
the seven commercial instruments were generally
better than those established on Sample 1.

Comparison of Validities of CSC and
Non-CSC Tests for Sample 2

Sample 2 comprised the total number of TATC
trainees examined with the entire group of six
CSC tests. The battery with which they were
assessed also included four of the commercial
tests. As may be noted in Table II, the validi-
ties of the six CSC tests ranged from .23 to .55
for Course Grade and from .16 to 45 for the
P-F criterion whereas those of the four non-CSC
instruments ranged from 27 to 44 for Course
Grade and .25 to .87 for Pass-Fail Status. If
the coefficients in each array were averaged, the
means would slightly favor the group of com-
mercially-published tests.

Although not presented in any table, inter-
correlations of atl CSC and non-CSC test meas-
ures for Sample 2 were computed. A correlation
of .72 between DAT Space Relations and CSC
Spatial Patterns was deemed sufficiently high to
support the hypothesis that these two similarly-
named tests were also similar in factor content.
A. coefficient of 51 was obtained between DAT
Numerical Ability and CSC-24 Computations;
this was lower than had been anticipated and the
latter correlated no higher than .28 with any of
the other three commercial instruments. While
DAT Abstract Reasoning was found to correlate
only .48 with Part I (Abstract Reasoning) of
CSC Booklet 157, it correlated .71 with Part IT
(Letter Sequence).

* Ally
coeffiet

and the
¢ with

Vario
cation 0
describe
derived
Two su
rived f
Sample
posite,”
Analogi
Compos
the vali
as 52:
Corresp
tained
compar:
rate an:
group
betweer
indicate
tween
reasona

compos



saansteyy
epuyady
uonaatag
jauuosteg
saaq[orq10,)
owRay Ty
8407011989

“Ha ‘aq09 1

‘ISaI, UORDATAS
OLV-VVE SuoIss}MIOD adladeg AID ‘Ss’ 9u3 Uo Syrrenb
0} a1qe oq PMOK setoseya apnynde Ogiy seddn esey] uy
sjalqns ay} JO Spaly]-O} JNoqU Jey} pareUyIsa sea IT “OL
uBy} Ss9] JO Xopuy [BIOUIH ¥ Zulawy soyEpIpuBD aos0q Ty JO
PUR CTT Aiojaq sexapmy aphyyde OgIy Suravy jautosiod sdxo;y
DUEL pue ‘CaBy ‘Auury JO WONDe[s ay] Suypnpord sTwapyeds
isprypurs (OSI) UOTBOyISssR]D-puv-Suyuee.s-<IB I oy] UT
aBuvyo efdans v 0} parwdurod sesodind Huyuaas.s S1UpUoDes OF
ANIBA aT Sayer JO oq PINON sysa}y YoRs yvyI peyerysNt

sZu]puy aeyjo ‘WaaemoyY “soUBUTIOJIed SuyuyeIA-OLY Arey
qoypaid 0} ssau@alpaya o[quiepisuod YA posh weeq eavy
PINod sys} ey} Jo 1OJ Stu SurAfoauy ag00s aysodurod v yvUy
payeorpul syMsey (VV) vopersTUTUpY UoHeIAY Teepe
ay} JO SPOUTRI] OLY WIA yoauasas snorserd ur paureyqo vywEp
UOTIEPIIVA JO SISeq oy) UO APTQS 9] AOJ pazoaTas azo $]Sa}
udaas dyg, ‘swBIZ0Id UONPEPS-OLY ATUITW oy} esordur oF
pasn aq ISU sansvaur apnqyde yous yorys YA TeAUaz0d
ay} JO JualUssasse WE pUB ‘sjUapNIs SBN puE ‘sdIog eULTETY
‘sorog ITV ‘SUY Jo somsvour aouemozied asanos-duruye.g
PUB Sa1oss opnyTde oy uaanjeq sdrysuoyejer Jo uoTRUTT
-19}9p OY} ‘Spooysy SuyuIBIL (OLY) lodjuoy owerL ay SYN
ooudTD PUR GAY J9[S9aY ay 18 Sjsaq apngnde paysiqnd Aye
-J9UTUIOD U@AS Jo ash [BJUaU[Zedxe ey} sUIeDUCD SpNys SIT

‘O-U-WY-VVul ON Jodo “dd og
‘6I 300010 ‘qq0D “f Jv Sq SHANIVUL Yq
“IOULNOD VVd GNV AUVLITIW JO SAMASVaN
ISHL AGALILIVY OlddVaL IV “PwoyRMO ‘AND
RMOTETYO ‘ayNIMIsUT [wopemoray Tal ‘OUI
UONBIAY JO aBWO ‘UONvIIS|UIIPY UOVIAY [repag

saansvaly
apnyndy
uoraateg
yeuuosied
S19T101910,)
WRAL, ITV
81030).4989(7

“a a ‘aq00 “1

WAL WoRPEPS
OLV-VV S.UoIss}WMOD’ aIAIeg AID “Sau? UO Anrenb
O} a1qu aq PINOM saTsoSa}vo apnzde Hgiy seddn aseyy uy
sjafqns oy} JO SPIIYI-OM] yNoqR IY] pazwUNse. ses ITO,
ubYy} sso JO XOpUy [eAVIH wv SUABY So|UpIPUBD 10g ALY JO
PUB CTE Aojaq saxepuy apninde Ogi sujsvy jeuuosrad sd1oy
DUTARIY PUB ‘SAB ‘ULTY Jo UOTOd[s oy) Suypnqooad sTRoBIoads
{sprvpurys (OS) LoVoyissep-puR-suruders-LALIT[L oy} Uy
asuByD a[duuls B 0} pareduo sasodind Zujuseds L1epuoses 20y
ATYBA BIL ATANBPEL JO aq pros ssa} YRS Tey) paxwxysnqt

sSulpug topo ‘TeseMoF] ‘aouRWAoJIed SuyuyEI-OLY Arey
joypad 0] ssauaanoaya e[quaapjsuod yy} past ueaq aary
PIM Syso 94} JO AMOZ suo Supajoruy aaoos oy1soduroo wv wy
peworpul SIMs (VV) UoHErs|uIpY uoNwAy Tereped
ay} JO saauIET] OLY WIEN yoawasar snoraoid ul paureyyo wyEp
UONTEPITRA JO SISBq ay} UO APHIS ayI JOJ pazooTes ated s]so}
UaAVs BY ‘stuVsZ0Id uorpefes-QLY AABY[U ay] aaordury 0}
pasn aq 1ySur sarnsvaur apuzyAde yous yor YEA TenUE}od
auj JO JUaTUSSasSe UR PUB ‘s}llapNIs sAUN pue ‘sdiog auTTeTy
‘so10q ay ‘<uUY Jo satusvour douvuLosied asi0d-Buyurery
pue sezoss apnzyde ayy uaaajeq sdywkuoyeer Jo uoreurm
-A9}9P OY} ‘Sjooyay SuyuTBAL (OLY) TOUWO) MBIT ATV SV:
ooUsTO PUB TAY JeSeay oy] 18 S}saq epninde paystand se
-AaWUOD UBAEs JO aS TRIUAUN|Ledxe oY} SUAADUOD APNIS STL

“OU-NV-VVd ‘ON Woda “dd og
‘TL6L 10907200 ‘qq0p ‘dq weg 4q SAENIVUL UAT
“1OULNOD V¥d GNV AUVLITIN JO SAXNSVAW
USAL AGALILdVY OdVYL UIV “Bwoyepio “A119
BUOY ‘aNqVSUT [wPeMoLy AI) ‘auIO1payT
uoTRay JO YO ‘UoTeNsINIMpY uopeAy Terapeq



the military ATC training schools consistently
failed to correlate at statistically-significant
levels with the training-performance measures of
every group. Both were subtests of the California
Test of Mental Maturity. One (“Analogies”),
validated rather well with the Air Force group
and FAA Sample 2, but only moderately with
the Marines and negligibly with the remaining
four samples. Scores on the other subtest
(CTMM Coins) yielded significant validities for
the prediction of the Academic-+Laboratory
(A+L) Grade Averages of every group but
failed to correlate appreciably with the “PI”
criterion measures of the Air Force trainees and
the “Pass-Fail” status of the Marines. Despite
those instances in which the two CTMM subtests
yielded rather conservative validities, analyses
revealed that composite scores based on the entire
seven-test battery correlated with the A+L
Grades of every group at somewhat higher levels
than any of the measures based on combinations
of test scores. Yet, in every instance, the validity
of the Commercial 7-Test Composite Score was
closely approximated by that of a composite
measure reflecting performance on DAT Space
Relations, DAT Numerical Ability, DAT Ab-
stract Reasoning, and CTMM Inference.
Composite scores of both types correlated with
the training-course performance measures signifi-
cantly better than did the Air Force’s General
Aptitude Index (“GI”) and the Marine Corps’
MSC score and at about the same level as the
aptitude screening measures used by the Army
and Navy. Despite the high degree of com-
parability between the validities of each experi-
mental composite and the MSC scores of the
Army and Navy subjects, the results of multiple
correlational analyses indicated thgt the experi-
mental measures for these trainees, as well as the
Airmen and Marines, could have been used in
combination with the MSC scores to attain some
degree of improvement in the respective ATC
selection programs. However, the benefits stem-
ming from such a procedure would have been
considerably less for the Army and Navy than
for the Air Force and Marine Corps. Moreover,
mucl of the improvement theoretically possible
could have been attaiged through use of the MSC
scores only. For example, the initial screenout,
or disqualification, of every subject having either
an MSC score of less than 115 or a GI under 70
would have rendered the commercial composite

measures virtually useless for secondary screen-
ing purposes.

It is understood that the MSC aptitude-test,
batteries currently used in the selection of mili-
tary ATC personnel are much the same as those
with which the subjects in this study were as-
sessed. If so, an upward revision in the MSC
test screening standards would, in the opinion
of the author, represent a more practical ap-
proach toward improving selection procedures
than would the use of additional tests. Although
military classification-and-training officials must
cope with the demands for high-aptitude per-
sonnel for a diversity of other specialties, the
results obtained in this study provide a sound
basis for recommending that eligibility for ATC
training require either an MSC score of at least
115 or a GI score no lower than 70. Should such
a qualification standard be adopted, it would be
advisable to undertake a number of followup
studies in order to: (a) further validate the new
procedure, (b) determine the appropriateness
and feasibility of implementing even higher
MSC aptitude-screening standards, (¢) re-
evaluate the predictive potential of the various
tests or subtests of each respective MSC battery,
and (d) determine, through multiple-regression
analyses, whether the performance scores on the
MSC subtests could be combined and factor-
weighted in a specific manner for derivation of a
“special” ATC-selection index having greater
yalidity than the conventional MSC summary
scores. Succinctly, it may be concluded that
tests similar to those used on an experimental
basis in the present study would be of very lim-
ited value in improving the military-ATC screen-
ing process since the predictive potential of
existing MSC medsures, if fully exploited, could
effect a substantial improvement.

Although in quest of further improving ATC-
selection methods, officials of the military train-
ing schools may also experience no small degree
of satisfaction from the finding that both samples
of FAA TATC trainees performed at. signifi-
cantly lower mean levels on the experimental
test battery than any military group. However,
these differences are important in a number of
respects. The subjects comprising each FAA
sample were recruited during 1961 and 1962,
prior to implementation of the CSC ATC Apti-
tude Screening Battery, under standards whereby
an assessment of types and amounts of previous

28

job-rel:
work, ¢
As pal
aimed
dures,
tude te
entry i
Alth
from t
nated ;
which
Test a
lished
at the
The mr
trainee,
amined
tests b
battery
the Co
CSC 4
provid
tion w
ance §
subject
posite
Only
experi?
Test a
Osc °
per ce
Based.
dicted
only 2
Samph
In con
of the
Force :
and 68
survey
of 210
jority
qualific
ing on
tary ¢
differe:
FAA

conclu:



ach item, which
properly carries
pment exhibited
vides a measure

relationship be-
cts) in order to
choices as being

t designated as
‘ithmetic” meas-
ted arithmetic

mbers of coins.
ipplement with
ublished instru-
‘he two samples
in the present
tterns; CSC-24

battery which
1 screening of
training since
atial Patterns,
ing are “coun-
1 tests bearing
5 items of the

development
In “Following

Operationally,
ferred to as

Samples

FAA Samples 1 and 2. Two groups of FAA
ATC trainees who participated in CAMI’s pre-
vious validation research were selected for in-
clusion in the study. All trainees within each
group had established their eligibility for selec-
tion and appointment to FAA training under
standards which involved no aptitude screening.
These 395 subjects represented all entrants into
the Academy’s Terminal-Area-Traffic-Control
(TATC) Training Course during the period
September 1960 through June 1962. It was as-
sumed that the basic TATC training course was
more like that provided at Glynco NAS and
Keesler AFB than the Academy’s ARTCC (or
“Enroute”) course. Of the 395 trainees, 212
entered training prior to July 1961 and were
experimentally examined with a battery which
included all seven commercial tests ultimately
chosen for use at the Glynco and Keesler training
facilities but no CSC instruments. This group
was designated as Sample 1. The remaining 183
arrived at the Academy during the period August
1961 through June 1962. They were the only
FAA TATC trainees which CAMI had assessed
with a battery which included not only the six
CSC tests but also four of the commercially-
published instruments. This group was desig-
nated as Sample 2. CAMI’s previous research
had revealed a correlation of .81 between the
Commercial Four-Test Composite scores of these
subjects and their composite CSC measures. The
sample is of considerable importance in the pres-
ent study because the correlation coefficient of
81 provided the basis for development of an
equation whereby the CSC 6-Test Composite
Scores of the military trainees could be estimated
from their performance on the four commercial
tests. ,

Samples 3 and 4: Army and Air Force. Be-
ginning in August 1968, the seven commercially-
published tests were administered, on an experi-
mental basis, to incoming classes of the Keesler
AFB ATC Training School. In planning the
study, it had been anticipated that performance
data for 200 Army trainees and a similar number
ofyAir Force students could be collected within
about three months, However, the training

course inputs wel underestimated and it was
necessary to conduct assessment sessions through
December 1968 in order to obtain data for two
groups of 169 each. The Army group was desig-

nated as Sample 3 and the Air Force group as
Sample 4.

Samples 5 and 6: Navy and Marine Corps.
As mentioned earlier, CAMI researchers received
the request to undertake the Keesler study fol-
lowing their report of findings for the Glynco
study.+ A minor portion of the results presented
in the Glynco report stemmed from multiple re-
gression analyses which (for reasons explained
in that report) were accomplished upon correla-
tion coefficients (i.e. intercorrelations) based
upon the combined data of the Navy and Marine
trainees. Similar analyses on the data of the
respective groups were not undertaken due to
monetary considerations. Consequently there are
a few instances in the present report where the
data relate to the combined Navy and Marine
Corps groups. In general, however, separateness
of the data was maintained and the results are
presented for the respective Glynco groups in
the same manner as for each FAA, Army, and
Air Force sample.

All but four of the 963 students who entered
the Glynco NAS training course during March
1966 through February 1967 were administered
the uniform battery of seven commercial tests
on an experimental basis soon after enrollment.
Of those examined, 642 were Navy personnel;
this group was designated as Sample 5. The
remaining 317 were Marines who entered train-
ing during the earlier two-thirds of the indicated
time period. These subjects comprised the sixth
sample.

Table I indicates the number of subjects com-
prising each respective sample, the number who
passed their respective training courses, the spe-
cific groups of tests they were experimentally
administered, ahd the types of MSC scores avail-
able on each sample.

Military-Screening-and-Classification
Test Scores

Three, types of MSC scores were forwarded
to CAMI for each Army trainee. One reflected
performance on a test of verbal abilities (“V”),
another pertained to a test of arithmetical
reasoning (“A”), and the third, referred to as
“MSC V+ A,” represented an average of the
two, Data for each Air Force trainee included
the AFQT (Armed Forces Qualification Test)
score and four “aptitude indexes” derived from
the Airman Classification Battery (ACB). The



II. Procedure.

Both ‘the Glynco and Keesler studies were
undertaken on a cooperative basis with officials
of the respective military ATC training schools.
It was mutually agreed that CAMI researchers
would assume responsibility for: formulation of
the test battery, supplies of test booklets, the
design and accomplishment of all data analyses,
and preparation of a report of findings. Glynco
and Keesler representatives assumed responsibili-
ties for the experimental administration and
scoring of the aptitude tests and the collection
of various types of correlative data such as
military-screening-and-classification (MSC) test
scores, chronological ages, and training-course
performance data for all examinees.

Aptitude Tests

The selection of tests for validation at the two
training facilities was predicated upon the find-
ings which CAMI had obtained in previous re-
search with FAA ATC trainees. As mentioned
earlier, a total of eight non-CSC instruments
(seven commercially-published tests and the Air
Traffic Problems Test), had been identified as
yielding a number of different and valid compo-
site measures for prediction of FAA ATC per-
formance criteria‘. Each composite score—
whether based on four, five, six or seven tests
(no group had been given all eight tests)—
validated at a statistically significant level for
prediction of the training measures. Inasmuch
as the Air Traffic Problems Test (ATP) was
subsequently designated as a CSC test, existing
policies precluded its release for experimental
use with military trainees. It was assumed that
the remaining seven tests would constitute an
appropriate battery for experimental studies in-
volving military ATC trainee,

Although the seven instruments of the experi-
mental battery are referred to in the present
report as “tests” they are actually parts, or sub-
tests, of rather lengthy and comprehensive
aptitude-measuring devices. Three of the seven
are subtests of the Psychological Corporation’s
reputable Differential Aptitude Test (DAT),

gnamely “DAT Space Relations,” “DAT Numeri-
cal Ability,” and “DAT Abstract Reasoning.”
One of these n¥asures the ability of a subject to
visualize objects and forms in two or three di-
mensions and one is a test of arithmetical or
computational skill, The task in DAT Abstract

Reasoning is to determine, for each item, which
of a series of choices (figures) properly carries
out a principle of logical development exhibited
by a sequence of figures; it provides a measure
of non-verbal reasoning. The remaining four are
subtests of the California Test Bureau’s Test of
Mental Maturity (CTMM, Advanced Form A
Edition). In each item of CTMM Analogies,
the subject must recognize the relationship be-
tween a pair of drawings (objects) in order to
identify, by analogy, one of four choices as being
similarly related to a third. CTMM Inference
involves the comprehension of statements which
present premises underlying the derivation of
logical conclusions. The subtest designated as
“CTMM Numerical Quantity-Arithmetic” meas-
ures ability to solve word-presented arithmetic
problems, while CTMM Numerical Quantity-
Coins involves the mental manipulation of inter-
related amounts of money and numbers of coins.

The six CSC tests used in supplement with
four of the seven commercially-published instra-
ments to obtain data for one of the two samples
of FAA ATCS trainees involved in the present
study were: CSC-51 Spatial Patterns; CSC-24
Computations; CSC-157 Abstract Reasoning;
CSC-157 Letter Sequence; CSC-135 Following
Oral Directions; and CSC-540 Air Traffic Prob-
lems. The six tests comprise the battery which
has been used in the operational screening of
most applicants for FAA ATCS training since
January 1964. Of the six, Spatial Patterns,
Computations and Abstract Reasoning are “coun-
terparts” of the three commercial tests bearing
similar titles. In each of the 25 items of the
Letter Sequence Test, the examinee’s task is to
indicate which of a series of letters properly
carries out @ principle of logical development
exhibited by a sequence of letters. In “Following
Oral Directions,” the subject must make rapid
and accurate decisions while he is orally presented
a diversity of both irrelevant and pertinent in-
formation. The ATP Test presents a flight-data
display (ie, the altitudes and ETA’s of several
aircraft) and the examinee’s task is to determine
whether the aircraft may be permitted to make
certain changes in altitude without violating o
specified time-separation rule. Operationally,
the CSC battery is commonly referred to as
“The CSC ATC Aptitude Screening Test” and
the six elements as “subtests.”

_

FA.
ATC
vious
clusio:
group
tion +
stand:
These
the

(TAS
Septe
sume:
more
Kees:
“Enr
enter
expel
inclu
chost
facil
was
arriy
1961
FA:
with
csc
publ
nate
had
Con
subj
sam
ent
81
equi
Sco
fror
test
Ss
gin:
puk
mer
AF
stuc
dat
of
abo
cou
nec
De

gre



tribution Marginal-Pass oO $e in Upper Four-Fifths of Distribution
oe Grades and/or Fail Cases of Military ATC Training-Course Grades
H Sample 3 Semple 4
x” Aray ATC Trainees Air Force ATC Trainees
Ne 169 N- 169
39
na
30
20 15
19 4 6 6
2950 3150 3350 "3550 104 105 110 MS 120 225 130 135 140 3 «660 65) C7075 OCS DSC*OS
Pie 73349 73549 & > & < -109 +114 -119 -124 -129 134 -139 & > &< &>
y-Administered tA" re ACB "General" Aptitude Index
ly-Published Tests Army HSC Score Air Force eral” Aptitu
Sample 5 102 Sample 6
100 Navy ATC Trainees Marine Corps ATC Trainees
N- 626 8 a M - 259
5
80
61
6
ro)
25
20 L 12
3
549 104 105 110 M5 120 125 130 135 140 204 105 110 115 120 125 130 135 140
Administered &< -109 -114 -119 -124 -129 +134 -139 & > &< -109 -114 -119 -124 -129 -134 -139 & >
jy-Published Tests Navy “GCTARI" Score Marine Corps "'1/3(ABHPEVA)" Score

Course Performance
ATC Trainees at the

xample, approxi-
e “Marginal-Pass
3, 4, and 5 failed
at least 2550 on
only 14 to 18
ry students were
ut 20 per cent of
‘ormed marginally
0 on the battery,
other trainees in

m concerning the
rational purposes
degree to which

and Air Force
or failed their
Jow-to-moderate

Ficure 3, Comparisori of Military-Screening-and-Classification (MSC) Scores of the U.S. Army, Air Force, Navy
and Marine Corps as Predictors of Military ATC-Training-Course Performance.

MSC scores. In contrast, the data for Marines
fail to reflect any significant relationship in this
respect. For each sample except the latter, how-
ever, there is evidence indicating that the screen-
ing potential of the respective MSC aptitude
measure has not been fully realized.

Adherence to the recommended standards (i.e.,
MSC scores of 110 for the Army and Navy and
65 for the Air Force) would have precluded
selection of significant proportions of those who
failed or performed rather poorly in training.
Even greater effectiveness, however, could have
been achieved by the Army and Navy if circum-
stances had permitted the adoption of, and ad-
herence to, slightly higher selection standards.
For example, within the Army group, 15 (38.4

pPer cent) of the 39 students who failed or
marginally passed training possessed MSC scores
of less than 11¥ compared to only 18 (13.8 per
cent) of the 130 non-marginal students, A
similar screening score by the Navy would have
precluded the entry of 80 (57.6 per cent) of the

189 students who performed rather poorly in
training and 118 (24.2 per cent) of the remaining
487. The Air Force deviated from its normal
selection standard in the selection of 25 men
who had MSC-GI scores of less than 65. Twelve
of the 25 either failed or marginally passed the
Keesler training course. The 12 represented
exactly one-third of the lower performance group
whereas the 13 represented only 9.8 per cent of
those in the upper four-fifths of the grade
distribution.

Virtually all of the Marines possessed MSC
Scores of 110 or better and they tended to achieve
somewhat higher training grades than their
Navy classmates. Only 16.2 per cent of the
Marines either failed or completed the course
with only marginally-passing grades and almost
half of these were men with MSC Scores of less
than 125. It should also be noted that the MSC
Scores are rather normally distributed. This
finding warrants dismissal of the speculation
that the difficulty index of the Marine Corps’



/stuby
Sample 6
Marines

Ne=317

7 _Np=288 Ne=2s

rere

2
—_———

scores were avail+
"Y" refer to scores
Meary ATC training.)

IMSC index of 110
dard for selection
fainees. However,
this policy was
316 (of the 317)
d 106 (about 16.5
pm lower-aptitude
Ma similar stand-
en represented in
bving MSC scores
r “General Apti-
ughly equivalent
Navy MSC score

of the military
1 lower-aptitude
er, a number of

involved. For
itations, quotas,
many different
nd, inasmuch as
and assigned to

|

some type of training, deviations from normally-
prescribed standards are sometimes necessary.
Moreover, the minimum MSC “screening score”
recommended by each respective service for se-
lection of ATC personnel should not be consid-
ered “low”; each is roughly equivalent to a
percentile score of at least 65, and the vast ma-
jority of the military ATC trainees comprising
the current samples were found to possess scores
well above this level.
Criteria y
The basic criterion variable against which all
test measures were evaluated was the “Overall
Training Course Grade Average.” For the FAA
groups, as well as the Glynco and Keesler train-
ees, this measure represented an arithmetical
mean of two averages, one of which was based
on all tests relating to the course lectures and
academic materials, while the other was a com-
prehensive measure of laboratory performance.
“Pass-Fail Status” (“P-F”) for each respective
training course was also employed as a uniform
criterion but the correlations and other statistics
based on this criterion were ultimately considered
as rather meaningless for Samples 1, 8, and 4
because the attrition rates were so low.

However, CAMIZ researchers were able to
formulate an additional criterion, referred to as
the “Progress Index” (“PI”), for each Army
and Air Force Trainee (Samples 3 and 4). The
first step in the derivation of the PI involved a
review of each individual training record. A
negative weight of 4 (ie, “—4”) was assigned
to the subject in each instance where the record
indicated he was a “washback” or “holdover” to
a succeeding class. Similarly, a weight of —3
was assigned for each “retake” of an exam or
“block failure,” a —2 for each occasion where
the trainee was “counseled for slow progress,”
and —1 for each instance where the record indi-
cated “remedial instruction” was necessary. If
the subject was designated as an “honor grad-
uate,” a positive weight of 2 (ie, +2) was as-
signed. The algebraic sum of all such assigned
values was computed for each subject; the result
was algebraically combined with a basic value

¥ of 100 to derive an adjusted score, and an average
of the latter atd the Overall Training Course
Grade was then obtained which, in raw-score
form, was presumed to reflect overall progress.
For purposes of convenience, the raw scores were

then converted, on the basis of separate frequency
distributions for the Army and Air Force groups,
to stanines. Thus each subject’s PI (or “Progress
Index”) is expressed on a 9-point scale, with 9
indicating that the subject was represented in
the approximate upper four per cent of the
original distribution. (Similarly, stanines of 8,
7, 6, 5, 4, 3, 2 and 1 roughly correspond to per-
centages of 7, 12, 17, 20, 17, 12, 7 and 4, respec-
tively.)

Factor Weights Used in Computing
Composite Aptitude Measures

With the exception of the estimated perform-
ance scores for the CSC battery, all composite
aptitude measures for the subjects were derived
through application of sets of factor weights
developed in previous research?* with perform-
ance data of several hundred FAA ATC trainees,
many of whom were “Enroute” trainees. In
fact FAA ATC samples included in the present
study represent only a minor portion of the total
group. For each test, the inverse of the standard
deviation of scores (1/SD) obtained for the
larger group was adopted as the basic factor
weight. Decimals were omitted and the resulting
sets of two-digit weights were applied in the
present study to derive the Commercial 7-Test
Composite Score and the Commercial 4-Test
Composite Score for each subject. (The appli-
cation of such weights is essentially equivalent
to a summation of test performance scores ren-
dered in standard-score form.) Earlier research,?
including the cross-validation aspects of the
Glynco investigation,’ indicated that weights of
this type were about as effective as beta weights
(derived through regression analyses) for ob-
taining summary correlate measures.

Moreover, single-digit factor weights utilized
in the Glynco study were also used in the present
study to compute the New 4-Test Commercial
Composite Score for each Army and Air Force
ATC trainee. However, these merely represented
a simple and proportionate reduction of the
original set of two-digit weights to single-digit
values, As stated earlier, the correlation of .81
between the Commercial 4-Test Composite Scores
and the CSC 6-Test Composite Scores of the
subjects comprising Sample 2 provided the basis
for development of an equation whereby the
scores of all other subjects on the complete CSC
battery were estimated directly from their Com-



“SOOMIWAT, AAUN “§'D ZO log OoUBUTLOZIEg esINOD SUIMIBIT, OLY OOUATH JO SIOWIPOLA SB SeI0DS

789L OLV-OSO peeUSsT DUB oySodMD ysoL-p TeIoAeMIMIOD ‘eysodm0Q ysaz-2 THIOMMOD ‘aIODg (IUV+LOD) OSW AABN 30 wospedmog “g THAeKT
aroog 389] epnazady OLV-OS) pereuyIsA 22095 e3;80dmoy 38aL-y TeTOIIIMOD
<9 6Z2- 61Z- 60Z- 661- 68I- 6LT- 69I- >¥ < 8 6YBI- 67LT- 679T- 6YST- 677T- 6YET- 6yZI- > 9 b
O€Z O72 OTZ 002 O61 O8f OT O9T EST OSST OSAT OS9T OSST OS¥I OSET OSZI OSTI 6¥%IT
oz
ov
09
+ 08
Oot
879 --N eo - R ozt
Szt &
@209g aQ780dm0D 38e]-/ [PFII0NMIOD e025 (TaV + 109) OSH LAeN
< 9 GYSE- GEE- GYIC- 6762- 67LZ- 67S7- B7EZ- > 9 <9 6€I- YEI- 6ZI- %2I- 6IT- YII- 601- >

OSSE OSEE OSTE OS6Z OSLZ OSSZ OSEZ OSTZ 6YIZ Ov SET O€T SZI OZT SIT OIT Sot 0T

Lj
ras

SZ
rau
- 979 - N
879 --N ozt
Gapeag oeaino)-Suyupery OLY ODUATH JO sae" TF¥Z 10/pue T]
woranqyz3eya 30 8423 Fa-2n0g 30ddq uy eg [_] eeug-[eUT81eK .

‘ROINSVOPY VOUBUIOJFIed BSINOQ SUUTVIL-OLY PAV Jessy JO StOyIporg sv Salog Wal OLV-OSO

PAWS pu sysodmoy yoL-7 TeOIOMMOD ‘oyFodMIOg ysaT-Z TEPTEMIMOD ‘xePUT ,PPRIGY TBeUa FOV» OSH ex0q ITV Jo uosEdMOg “2 FHAONT

azoog 383] apnazady OLV-9S) pezeUtIs|a az0og aqysoduog 3S9]-y TE FIASWMOD
< B 6YBT- 6YLT- 679T- 6ST- 6YT- 6YET- 677T-
eas a aro A are oD a oO

<9 6@Z- 61Z- 60%- 661- 681- 64I- 69T- >”

>



[———

T. Report No. 2. Government Accession No. S Recipient's Catalog No.

TECHNICAL REPORT STANDARD TITLE PAGE

PAA-AM-71-40
‘4 Title ond Subtitle ‘5. Report Date

AIR TRAFFIC APTITUDE TEST MEASURES OF October 1971
MILITARY AND FAA CONTROLLER TRAINEES 6, Performing Organization Code

: 7. Author(s) 8. Performing Organization Report No.

Bart B. Cobb, M.S.
9. Performing Organization Nome and Address
FAA Civil Aeromedical Institute
P. 0. Box 25082 TT. Contract or Grant No.
H Oklahoma City, Oklahoma 73125

10. Work Unit No.

13. Type of Report and Period Covered

12. Sponsoring Agency Nome ond Address

H Office of Aviation Medicine OAM Report
i Federal Aviation Administration
800 Independence Avenue, S.W. 14. Sponsoring Agency Code

Washington, D.C. 20591

i 15. Supplementary Notes

This research was conducted under Tasks No, AM-D-67-PSY-2, AM-B-69-PSY-2,
AM-A-70-PSY-23 and AM-B-71-PSY-23.

16. Abstract
This study concerns the experimental use of seven commercially published aptitude
tests at the Keesler AFB and Glynco NAS Air Traffic Control (ATC) Training Schools,
the determination of relationships between the aptitude scores and training-course
performance measures of Army, Air Force, Marine Corps, and Navy students, and an
assessment. of the potential with which such aptitude measures might be used to
improve the military ATC selection programs. The seven tests were selected for

i the study on the basis of validation data obtained in previous research with ATC

H trainees of the Federal Aviation Administration (FAA). Results indicated that a

H composite score involving only four of the tests could have been used with con-

siderable effectiveness to predict military ATC training performance. However,
H other findings illustrated that such tests would be of relatively little value

H for secondary screening purposes compared to a simple change in the military-
screening-and-classification (MSC) standards; specifically precluding the selec-
H tion of Army, Navy, and Marine Corps personnel having MSC aptitude indexes below
i 115 and of Air Force candidates having a General Index of less than 70. It was
: estimated that about two-thirds of the subjects #n these upper MSC aptitude
categories would We able to qualify on the U. S, Civil Service Commission's
FAA-ATC Selection Test.

* .
17. Key Words 18. Distribution Statement
Air Traffic Controllers Availability is unlimited. Document may
Personnel Selection be released to National Technical Informa-
Aptitude Measures tion Service, Springfield, Virginia 22151,
g for sale to the public.

H +

H T9. Security Classil, (of thia report) 2, Security Classif, (of this page) Gl. No. of Pages | 22. Price

i Unclassified Unclassified ag $3.00

Form DOT F 1700.7 (s-s97



these validities will reveal a consistent trend
indicating that the Commercial 7-Test Composite
was somewhat more effective than the Commer-
cial 4-Test Composite for the prediction of the
various criteria. With Course Grade serving as
the criterion variable, the seven-test and four-
test summary scores yielded validity coefficients
of 39 and .32, respectively, for FAA Sample 1.
Corresponding coefficients of .37 and .27 were
obtained for Sample 3 (Army), .52 and .49 for
the Air Force trainees, and 41 and .38 for
(Navy) Sample 5, while each composite correlated
41 with the training grades of the Marines.
Although the correlations of both summary
measures with either P-F or PI were generally
lower than obtained with Course Grade, there
was no instance (i.e, sample) in which the va-
lidity of abbreviated composite exceeded that of
the Commercial 7-Test Composite. A point
which warrants emphasis, however, is the fact
that none of the differences between the validities
of the two composites for Course Grade, P-F,
or PI was statistically significant. Although it
had been presumed that a composite score based
on the complete battery would exhibit predictive
potential beyond that of a measure relating to
only four of the seven tests, it had not been
anticipated that the validities of the latter would
so closely approach those of the more compre-
hensive measure.

There were a number of instances, as alluded
to above, where each composite differed marxedty
in its correlations with the two criteria. For the
Glynco samples, neither summary score correlated
less than .88 with Course Grade nor higher than
.26 with the P-F criterion. For Sample, 2, the
Commercial 4-Test Composite yielded a validity
coefficient of .52 for Course Grade and .40 for
P-F. All such differences pertaining to Samples
2, 5, and 6 were statistically significant. For
Sample 8 (Army), each composite correlated
slightly better with the PI criterion than with
Course Grade while the inverse was true avith
regard to Sample 4, All differences pertaining
to Samples 3 and 4, however, were quite small
and nonsignificant.

Development of the “New 4-Test
Comme cial Composite”
Prior to the Glynco study, validities of sig-
nificant and substantial magnitude had been
established for the CTMM Analogies Test with

=

several different groups of FAA ARTCC train-
ees as well as with the 183 TATC trainees of
Sample 2. However, the low and questionable
validities obtained in the Glynco study on the
212 TATC trainees of Sample 1 and on the
Navy and Marine Corps samples prompted the
development of an additional summary measure.
In that study, a multiple-regression analysis was
accomplished on the matrix of test intercorrela-
tions and validities which had been established
for sample 1. The results indicated that the max-
imum validity possible with a composite based on
the complete battery could be approached with a
composite score based on the three DAT tests and
CTMM Inference. Consequently, a new summary
score, designated as the “New 4-Test Commercial
Composite” and based on single-digit factor
weights (as described earlier) was computed for
each Glynco subject. The validation data were
better than previously obtained for the original
four-test composite. Therefore, a similar score
was computed in the present study for each Army
and Air Force trainee. The resulting validities
(see Table II) were highly comparable to, and
in some instances equaled, those of the seven-test
composite.

Estimated CSC Composite Scores

No subjects other than those in FAA Sample 2
were administered the CSC battery. Their CSC
6-Test Composite Scores correlated .54 with
Course Grade and .39 with the P-F criterion.
Similar scores for all remaining subjects were
estimated, or predicted, directly from their Com-
mercial 4-Test Composite Scores through use of
the previously-described equation. Inasmuch as
the predicted CSC scores represented “perfect”
correlates of the*Commercial 4-Test Scores, they
correlated with Course Grade and P-F in the
same degree as the latter. Thus, the only mean-
ingful validities presented in Table II for the
CSC 6-Test Composite are those obtained with
Sample 2,

Empirical Validities of MSO Test Scores

The validities of the Military-Screening-and-
Classification Test Scores are presented in the
lower portion of Table II. Since the MSC scores
were used in the selection of the trainees, it
should be presumed that the samples are quite
restricted in range on this variable, and that
each validity coefficient consequently represents

@ gross U
over, unl
lationshiy
MSC tes
formance
each of
attenuate
tenuation
the MSC
The A
with Co
Index.
of 35 a
index ve
the Navy
of .44 an
aptitude
Tnasmuel
jority o:
high M§
should b
ever, it
experime
consider:
Corps 3
later inc
isted bet
experime
Of th
that bas
yielded
Army M
composit
cantly |
ACB-GI
also yie.
diction
and the
but, in
significa:
score.
Based
one mig
tests sin
battery
and CT:
ment wi
moderat:
process.
several 1
of imprc



lations between
four tests were
any other. The
the four tests by
jong whom there
) also correlated
-significant level

umber of TATC
group of six
hich they were
the commercial
je II, the validi-
from .28 to .55
to 45 for the
je four non-CSC
44 for Course
‘ail Status, If
e averaged, the
group of com-

y table, inter-
CSC test: meas-
A correlation

two similarly-
factor content.
between DAT
Computations;

Reasoning) of
with Part IT

COMPARATIVE VALIDITIES* OF APTITUDE TESTS AND COMPOSITES

‘TABLE IT.
FOR FAA, ARMY, AIR FORCE, NAVY, AND MARINE CORPS ATC TRAINEES
lel Sample 2 Sample 3 Sample 4 Sample 5 Sample 6
m= neAA Ary Air Force Navy Marines
Course: Course Course Course Course
cae -F . =] Grade "PI" Grade "Pi" Grade P-F Grade P-F
E Ne Nt Np-Ne
We “Ne Nt Mp NE Ne. Ne Ne Ne ‘Ne Np
C@ORRCIAL TESTS WRIT T9GAT NATGS T63-40 WoL) NIST NIGH W169 NCGZL 552577 NSIT 758-75
DAT Space Relations Mh 20h TR : : : :
DAT Nonerical AbLLity 1330 191362429386 3825368
DAT Abstract Reasoning MO 63847322827 7 BL.
CT Analogies (2009.27.25 0B 2 2070 618A
CTHH Inference 27.25 122.21 33.28 1B 19D
CIMA. N.Q. Coins ri 25 +21 +21 pty a4 3 3
CTH N.Q. Arithmetic 38.16 36.40 +40 : : :
CIVIL SERVICE COMM, TESTS W183 143-40
CSC S1- Spatial Patterns 37.27
CSC 24- Computations 28.16
aC 157- Abstract Reas. 2818
(SC 157- Letter Sequence 55 4S.
CSC 135- Oral Directions +23 +23
SC 540- ATP I+1I 41.29
COMPOSITE SCORES N=211 193-17 Nel83 143-40 N=169 N=169 N=169 Ne169 NM621 (552-77 Ne312 288-25
Commercial 7-Test Comp. 39 7 40 «52 AT AL +26 a 3
Commercial 4-Test Comp. 32 229 +52 40 3 4B 4 “é 3 3 nt 2
New 4-Test Comerc. Comp. 36 34 33 : : : : : :
t csc 6-Test Composite - 4.32) (29) 4 39 €27) (34) 649) 646) 38) C23) G4) (21)
MILITARY (MSC) TESTS Ne169 N=169 Nel69 N=169 9 N=619 550-77 Ne258 235-24
“y + AN Score 35 2 35 6
ACB- "GI" Score . . ab a

GCT + ARI Score
1/3

21 12

F-All validity coefficients are etatistically significant at the .05 level or better except the following:

five

coefficients obtained for CTMM Analogies (ives, -12 and .09 for Sample 1, .08 and .12 for Sample 3, and .07 for Sample 5)

and the .14 for Sample

and the .09 for Sample 6 for the CTMM N.Q. Coins Teat.

¢ With the exception of Sample 2, all CSC 6-Test Composite scores were predicted from the Commercial 4-Test Composite.

Various composite scores (based on the appli-
cation of previously-established factor weights as
described in earlier portions of this report) were
derived for the subjects of the respective samples.
Two such global, or summary, scores were de-
rived for every FAA subject represented in
Sample 2. One, the “Commercial 4-Test Com-
posite,” involved the three DAT tests and CTMM
Analogies. The other was the “CSC 6-Test
Composite.” With Course Grade as the criterion,
the validity of the first composite was ascertained
as .52 and that of the CSC Composite as .54.
Corresponding validities of .40 and .89 were ob-
tained when P-F served as the criterion. The
comparability of the validity data for the sepa-
rate and combined tests of each group, the inter-
goup correlations, and the correlation of .81
between the two composite scores for Sample 2
indicated a high”degree of correspondence be-
tween the two batteries, Moreover, it seemed

reasonable to assume that, had the commercial
composite for Sample 2 been extended to include

the CTMM subtests of Inference, NQ Coins and
NQ Arithmetic, the resulting summary measures
would have correlated with the CSC 6-Test
Composite Scores at a level somewhat higher
than that (ie. the .81) obtained with the ab-
breviated commercial test battery.

Empirical Y alidities of Commercial 7-Test
and Commercial 4-Test Composites for
Samples 1, 3, 4,5 and 6

In the Glyneo study, the first two of several
summary measures ultimately computed for the
FAA trainees of Sample 1, and for the Navy
and Marine Corps groups, were the “Commercial
7-Test Composite” and the “Commercial 4-Test
Composite.” Adhering to the same procedure,
no summary scores other than these two were
obtained in the initial phases of the present
study for the Army and Air Force trainees,

The correlations of the two composite scores
with Course Grade and either P-F or PI appear
in the lower portion of Table II. A review of



w

REFERENCES

. Cobb, B, B.: A Comparative Study of Air Traffic

Trainee Aptitude-Test Measures Involving Navy,
Marine Corps and FAA Controllers. FAA Office of
Aviation Medicine Report No, AM-68-14, 1968.

. Cobb, B, B.: Problems in Air Traffic Management:

II. Prediction of Success in Air Traffie Controller
School. Civil Aeromedical Research Institute Report
No. 62-2, FAA, Oklahoma City, Oklahoma, 1962.

» Cobb, B. B.: Problems in Air Traffic Management.

V. Identification and Potential of Aptitude Test
Measures for Selection of Tower Air Traffic Controlier

a

Trainees. Civil Aeromedical Research Institute Re-
port No. AM-65-19, FAA, Oklahoma City, Oklahoma,
1965.

. Trites, D. K., and B. B. Cobb: CARI Research on

Air Traffic Control Specialists: Age, Aptitude, and
Experience as Predictors of Performance. Civil
Aeromedical Research Institute, Unnumbered Report,
FAA, Oklahoma City, Oklahoma, 1964.

. Guilford, J. P.: Fundamental Statistics in Psychology
and Education, York, Pennsylvania, Maple Press
Company, 1956.



dary screen-

aptitude-test,
tion of mili-
same as those
ludy were as-
in the MSC
p the opinion
practical ap-

officials must,
aptitude per-
pecialties, the
bvide a sound
ity for ATC
bre of at least

Should such
, it would be
of followup

ppropriateness
even higher
5 ( ce) re-

MSC battery,
ple-regression
scores on the
and factor-
rivation of a

oncluded that
experimental
e of very lim-

potential of
ploited, could

proving ATC-
military train-

experimental
ip. However,
a number of

job-related experience, particularly military ATC
work, constituted the primary basis for selection.
As participants in a CAMI research program
aimed at developing improved selection proce-
dures, they were administered a variety of apti-
tude tests, on an experimental basis only, upon
entry into Academy training.

Although the batteries were purposely varied
from time to time, the 183 FAA subjects desig-
nated as Sample 2 were assessed with a battery
which included all subtests of the CSC ATC
Test and four of the seven commercially-pub-
lished tests subsequently chosen for validation
at the Keesler and Glynco training facilities.
The military subjects and the former TATC
trainees of Sample 1 were experimentally ex-
amined with the entire group of seven commercial
tests but were administered no part of the CSC
battery. However, a correlation of .81 between
the Commercial 4-Test Composite Scores and the
CSC ATC Test Scores of subjects in Sample 2
provided the basis for development of an equa-
tion which permitted the estimation of perform-
ance scores on the CSC battery for all other
subjects (from their Commercial 4-Test Com-
posite Scores).

Only 71 of the 183 TATC trainees who were
experimentally examined with the CSC ATC
Test attained scores of 210 or higher. Had the
CSC Test already been operational, only 38.8
per cent would have entered into training.
Based on CSC Test performance measures pre-
dicted from their Commercial 4-Test Scores,
only 29 per cent of the FAA subjects comprising
Sample 1 would have been accepted for training.
In contrast, it was estimated that 50.8 per cent
of the Army trainees, 40.9 per cent of the Air
Force subjects, 46.7 per cent of the Navy students,
and 68.7 per cent of the Marines involved in the
survey would pass the screening test with scores
of 210 and higher. Inasmuch as the vast ma-
jority of the FAA subjects of both samples
qualified for appointment to civilian-ATC train-
ing on the basis of their past experience in mili-
tary air traffic control, the finding of such
differences between the aptitude levels of the

AA and military groups suggests two general
conclusions: (1) She or more of the older (pre-

1960) MSC programs (from which the men who
subsequently entered the FAA were originally
selected for military ATC training) must have
used selection criteria or standards considerably
different from those presently used, and (2) that
significant improvements in the MSC methods
or standards had been initiated by the time the
Keesler and Glynco students who participated
in the present study were selectively assigned to
military ATC training.

On the basis of the results obtained in this
investigation, the Army, Navy, Air Force, and
Marine Corps have the capability, if desired or
required, of upgrading their screening of ATC
personnel without new or additional tests, by
establishing higher minimum MSC requirements.
Any changes in the military requirements will
be of direct interest to the FAA since the agency
will probably continue to select the majority of
its controller trainees from among those appli-
cants who have pre-FAA air traffic control ex-
perience and who also are able to qualify on an
ATC-aptitude-test screening battery. At the
present time, 210 is the minimum passing score
on the FAA CSC-ATC Aptitude Screening
Battery. Such a screening standard (roughly
equivalent to an MSC of 125 and a General
Index between 75 and 80) reflects a high degree
of selectivity, such that numerous unpublished
studies by CAMI have demonstrated that a
downward revision in the aptitude-test screening
standard of 10 to 15 score points (but no more
than 20) is feasible and should be considered if
the recruitment of personnel for FAA training
ever reaches a critical stage. On the basis of
the results obtained in this study and those of
previous investigations, it seems reasonable to
predict that approximately one-half of the mili-
tary trainees selected under recent military ATC
selection standards would be able to meet current
FAA aptitude screening requirements; only
about one-third of the FAA’s applicant popula-
tion prjor to 1964 would have been able to do so.
Moreover, if military cutoff scores are raised,
for any reason, from present levels to MSC scores
of 115 and to General Index Scores of 70, ap-
proximately two-thirds of the military control-
lers who qualify at those levels will qualify on
the FAA’s aptitude screening battery.



>

TABLE I, DESIGNATION OF APTITUDE TEST SCORES AVAILABLE FOR EACH SAMPLE OF THE STUDY
Sample 1 Sample 2 Sample 3 Sample 4 Sample 5 Sample 6
FAA FAA Army Air Force Navy Marines
Ne=212 Nera Ne=169 Np=169 Ne=642 Nee317

Np=193 Ngel7 _Np=ha3 Ng=4o

Np=160 Need Npw161 Ng=8 __Nps552.Nes77__ Npe288 Nen25

COMMERCIAL TESTS
DAT Space Relations

x x x x x x
DAT Numerical Ability x x x x x x
DAT Abstract Reasoning x x x x x x
CT Anatogies x x x x x x
CTH Inference x x x x x
TM N,Q. Coins x x x x x
CIM NG. Arithmetic x x x x x

CIVIL SERVICE COMMISSION TESTS
CSC 51- Spatial Patterns y
CSC 24- Computations y
SC 157- Abstract Reasoning y
SC 157+ Letter Sequence y
SC 135+ Oral Directions y
(SC 540- AIP I+1T y

TYPES OF COMPOSITE SCORES DERIVED
Commercial T-test Composite x x x x x
Commercial G-test Composite xX x x x x x
New 4-test Commercial Comp, —— X x x x x
CSC 6-test Composite Y

MILITARY (MSC) SCORES
"YW + A" score z
ACB ="GI" Score 2
1/3 (AR + VE + PA) Score 2
Ut + ARI Score z

(Notes: The lower-case letter, "x", indicates each conmerciatly-poblished test for which performance scores were avail-

able, while "KX" refers to composite scores based of specific groups of such tests. Similarly, "y” and "Y" refer to scores

for the separate and combined CSC-ATC subtests and "Z" denotes each MSC score used in screening for military ATC training.)

four are referred to ds the “GI” (General), “MI”
(Mechanical), “AT” (Administrative) and “EI”
(Electronics) indices. The Navy provided an
MSC index known as the “GCT+ARI” for each
of its trainees. In each instance, the score re-
flected performance on the Navy’s General
Classification Test (which is omnibus in factor
content) and a test known as “Arithmetical
Reasoning.” A similar index, referred to as
“Y% (AR+VE+PA),” was forwarded for each
Marine. The latter represented an average of
performance scores on Mariné Corps tests of
arithmetical reasoning, verbal ability, and per-
ceptual ability.

The aptitude-evaluation procedures employed
by the Navy, Marine Corps, and Army in the
selection of personnel involved in this study were
designed to yield a mean of 100 and an SD of 20
for a military population; for practical purposes,
tye procedures may therefore be considered as
directly comparable. Inasmuch as the Air Force
employs a different scale, extrapolative proce-
dures are necessary when an aptitude index of a
given magnitude is to be compared with an
Army, Marine Corps, or Navy MSC index. Ac-

cording to Glynco officials, an MSC index of 110
constituted a recommended standard for selection
of the Navy and Marine trainees. However,
subsequent findings indicated this policy was
adhered to in the selection of 316 (of the 317)
Marines while the Navy selected 106 (about 16.5
percent) of its 642 trainees from lower-aptitude
categories. The Army followed a similar stand-
ard and selected 161 of 169 men represented in
Sample 3 from among those having MSC scores
of 110 and higher. A “GI” (or “General Apti-
tude Index”) of 65, which is roughly equivalent
to an Army, Marine Corps, or Navy MSC score
of 110, constituted the Air Force’s recommended
standard. About 85° percent of the 169 cases
comprising Sample 4 met this standard; 25 did
not.

The reasons why each branch of the military
selected some trainees from the lower-aptitude
categories are unknown. However, a number of
mitigative factors may have been involved. For
example, training-facility limitations, quotas,
and priorities regarding the many different
specialty areas frequently vary and, inasmuch as
every recruit must be classified and assigned to

some ty
prescri
Moreov
recomn
lection
ered “
percen'
jority

the cu:
well at

The
test m
Traini
group.
ees, t
mean
on all
acade!
prehe:
“Pass
traini
criter
based.
as ra
becau

Ho
form
the “
and .
first
revie’
negal
to th
indic
a suc
was
“bloc
the |
and
cated
the «
uate,
signe
valu
was
of 1(
of t
Grac
forn
For



rr ee

4-Test Est. of
Comp. C8C-aTC P
Score Toat Sc. 38
2000 250.32:
¢ » P59
Ps
1800 (228.12)
Median
1600 (205.92) Pos
|
Pio
1400 (183.72)
,
1200 (161.52)
1000 (139.32)
Po2
0800 (117.12)
Sample 1 2 3 4 5 6
Type FAA FAA Arny USAF Navy Marines
N Cases 212 183 169 169 642 317
Mean 4-Test Score 1447.3 1496.0 1613.7 1553.6 1583.2 1730.0
SD of 4-Test Score 311.6 295.1 269.0 274.7 284.4 245.7

Force, Navy and Marine Corps ATC Trainees.

20

Frovre 5. Comparative Ranges and Distributions of Commercial 4-Test Composite Scores for FAA, Army, Air

Ss in Upper Four-Fifths of Distribution
of Keesler ATC Training-Course Grades

Oo

Marginal-Pase
and/or Fail Cases

L



189 & Bel 190 - 209 lifyi
Bw & se1oe ebGrens teers
Sample 3, Aray
"vta" Score NL
125 & > 62
110 - 124 99
109 & < 8
Total 169
Sample 4, Air Force
"General" Apt. Index
80 & > 79
65-75 65
60 & < 25
Total 169
Sample 5, Navy
“GCT + ARI" Score
125 & > 203
110 - 124 38.7% 331
109 & < 21.7% 11.3 | 106
299 640
Total 46.7%
Sample 6, Marines
"1/3(ARHVE4PA) Sc.
125 & > 7.20/14 4% 78.3% 180
110 - 124 19.7% 38.3% 42.0% 81
109 & < 100.0% 1
9 37 176 262
Total re) 21.8% 67.1%

Frieure 10. Proportion of Army, Air Force, Navy and Marine Corps ATC Trainees Within Specific Military-Screen-
ing-and-Classification (MSC) Test Score Ranges With Predicted CSC-ATC Screening Yest Scores of 189 and

Below, 190 to 209, and 210 and Above. »

to perform less well in ATC training than did
those of high aptitude levels. These findings
suggest ways in which one or more branches of
the"military establishment might wish to change
their MSC-aptituderequirements for ATC train-
ing. The implications of any such change upon
the future recruitment of personnel for FAA
ATC training are rather obvious.

IV. Summary and Conclusions.

Officials of the Glynco NAS and Keesler AFB
Air Traffic Control Training Schools requested
that studies be undertaken to determine the ef-
fectiveness with which various experimentally-
derived aptitude test measures might be used to
improve military ATC-trainee selection proce-
dures. However, the most important finding of

26

Sang

Samp
FAA

Same
FAA

Samp
Army

Samp
Air

Samp
Navy

Samp
Mari

Fiov

(Note
183 ¥
a cor
sump!

these
exist
tituc
to a
perh
of t'
Fore
subj
trair
low
was
by t
in te
tude
selec



vant experience, particularly in military air
traffic control, has always been heavily weighted
in the selection process. Experience as a pilot
and various types of work in communications
and air surveillance have also been consistently
viewed as important assets. In general, however,
the selection programs prior to 1962 involved no
formal assessment of mental abilities or aptitudes.

Beginning in July 1962 and for eighteen
months thereafter, an aptitude index, reflecting
performance on a CSC battery of six tests, served
as a major determinant in the selection of a
limited number of trainees who possessed little
or no pre-FAA ATC-related experience. The
aptitude screening index was an outgrowth of
extensive research conducted by CAMI. This
research, which was begun in August 1960, in-
cluded the administration of heterogeneous bat-
teries of aptitude tests, on an experimental basis,
to groups of newly-hired personnel as they ar-
rived at the Aeronautical Center Academy for
enrollment in either a nine-week basic-training
course in Terminal Area Traffic Control (TATC)
procedures or a somewhat similar course of the
same duration in which the instruction and lab-
oratory problems were oriented toward work at
an Air Route Trafic Control Center (ARTCC).
Specialists in the latter are sometimes referred
to as “Enroute” or “Center” controllers,

The CAMI experimental testing program ulti-
mately involved a total of 44 different tests,
many of which were either commercially pub-
lished instruments or aptitude assessment devices
developed under contractual arrangement for the
FAA. In fact, no CSC tests were included in
the research prior to July 1961. At that time a
series of multiple-regression analyses, accom-
plished in connection with follow-up studies of
several hundred men,?¢ identified a total of eight
tests (from a group of 27) with which a variety
of summary measures having validity for pre-
diction of ATC trainee performance might be
derived. Seven of the eight were commegcially
published instruments and one, “Air Traffic
Problems,” was a contractually-developed test.

Although no single group was ever adminis-
teed all eight tests, one class was examined with
seven, and several successive classes were admin-
istered either six or five, of the eight instruments.
When an average of the Academic and Labora-
tory Grades (“A+L”) was computed for each
Academy trainee and employed as the criterion,

Demme

the validities of the composite measures, derived
from the performance scores on five, six, or seven
tests, were found to range from .35 to .54 (Pear-
son product-moment coefficients*). More import-
antly, an analysis revealed that about 70 to 80
per cent of the cases classified as training-course
failures were, in most instances, represented in
the lower half of the distribution of scores de-
rived with each respective group of tests. The
attrition rates were averaging over 30 per cent
despite the fact that most trainees were former
military controllers.

The potential value of the tests for screening
purposes was recognized and, when similar re-
sults were obtained with additional samples, the
FAA and the Civil Service Commission agreed
that aptitude-test measures should be employed,
on a tentative basis, in the selection of some of
the non-experienced applicants. However, com-
mercially-published tests are not used because
such instruments may be more susceptible to
compromise than those subjected to rigid CSC
control procedures. Commission officials there-
fore examined their extensive file of CSC tests
and selected several instruments which, in terms
of factor content, appeared to approximate a
number of the validated commercial tests. Since
the Air Traffic Problems Test (ATP) had been
developed specifically for the FAA and was still
completely controlled, it was officially adopted
as a CSC test. In addition, CAMI researchers
were provided a number of other OSC tests
which they were also asked to administer and
evaluate.

Commencing in August 1961, all incoming
classes of Academy ATC trainees were experi-
mentally assessed with the entire group of tests
extracted from’the CSC files and with the ATP
Test. The restricted time available for each
testing session precluded an examination of each
class with the complete and previously-validated
battery of commercial tests. However, time be-
yond that required for the CSC tests was avail-
able to permit administration of a portion of the
commercial battery. A regression analysis of all
data collected up to this point in time indicated
that the validity of the Commercial Seven-Test
Composite could be approached with a summary
measure based on only four of the seven. The
four, which will be discussed later, were there-
fore used to supplement the CSC tests in the
next phase of research.

Follow
ined witk
months 1
five of t
effectivel
posite sc:
approxin
Of these
their tra
Traffic (
trast, al:
189 and
inated 4
sults a]
analyses
Seven-T
volved
those w
of thre

provide
non-ver
compos.
ment k
measur
lowing
Alth:
experin
tional
missior
aptituc
of the

cants v

vious j

official

ensuin;
were €
of the

190 or

batter:

limitat
recruit

the F.

major:

fully-«

Only

CSC

to tra

thoug.
ence.
CA
batter
classe:


4h Al

a two-digit value
the New 4-Test
of two-digit

lyses were accom-
ntercorrelations to
which a summary
C and the seven-
be used to enhance
. The results of
Table V. For the
each “R” (or esti-
oretical summary
the validity co-
(alone). Yet, in
was statistically
” of 52 and 48
were obtained for
the MSC index
ity coefficients of
alysis of data for
Grade, which had
MISC score, would
flecting perform-
hat the P-F cri-
yielded a validity
orrelate 23 with

ultiple-regression
dures were also
‘ommercial 7-Test,

TABLE IV, INTERCORRELATIONS AND VALIDITIES* OF COMPOSITE APTITUDE MEASURES
FOR ARMY, AIR FORCE, NAVY AND MARINE CORPS TRAINEE GROUPS

4-Test New 4- MSC Course Course Pass
Comp, Test Sc. Score Grade _"Py” Fail
Mean oN N N N N Np-N¢
S.D. xr r r xr r Toh
Sample 3 - Army
Commer. 7-test Comp. 2881.2 169 169 169 169 169
425.8 90 92 59 237. 40.
Commer. 4-test Comp. 1613.7 169 169 169 169
269.0 88 52 227 34
New 4-test Comm. Sc. 300.3 169 169 169
41,8 55. 33
MSC - "V+A" Score 122.2 169 169
8.3 35 42
Course Grade 86.8 169
4.4 187.
Sample 4 - Air Force
Commer. 7-test Comp. 2842.1 169 169 169 169 169
444.5. 291 91 59 52 47
Commer. 4-test Comp. 1553.6 169 169 169 169
274.7. 92 56. 49 46,
New 4-test Comm. Sc. 294.0 169 169 169
42.1 258. 252. 47
MSC - ACB "GI" Sc. 74.7 169 169
10,2 35 36
Course Grade «. 87.1 169
4.5 88
Sample 5 - Navy
Commer. 7-test Comp. 2859.2 642 642 640 621 552-77
442.5 92 293 re 41 26
Commer. 4-test Comp. 1583.2 642 640 621 552-77
q 284 4 2} 64 38 23
New 4-test Comm. Sc. 296.9 640 621 552-77
44.7 64 AL 26
MSC-GCT+ARI Score 119.4 619 550-77
10.1 4h 28
Course Grade 79.3 ‘ 551-70
+ 8.3 . 64
Sample 6 - Marine Corps
Commer. 7-test Comp. 3163.7 317 317 262 312 288-25
382.1 90 90 51 41 23
Commer. 4-test Comp. 1730.0 317 262 312 288-25
245.7 88 46 241 21
New 4-test Comm, Sc. 325.5 262 312 288-25
38.0 46 41 22
MSC-1/3(AR+VE4PA) Sc. 127.7 258 235-24
636 21 12
Course Grade 80.7 288-24
—+ 6.8 252

All r's are statistically significant at .05 level or better.



OF A MEASURE SUCH

21

34

209

vel or less,

iewpoint, however,
Army and Navy
such @ major revi-
In contrast, the

ACB subtests in some specific manner. In other
words, had the ATC specialty been deemed so
critical as to justify such action (with acceptance
of concomitant adverse effects upon selection of
personnel for other types of training), Air Force
researchers could undoubtedly have formulated
a “customized” global ACB measure surpassing
the validity of the experimental seven-test
composite,

Since the majority of the 169 Marines were
known to have possessed relatively high MSC
scores, rather low validity coefficients for the
operational index had been expected (due to
restriction-of-range effects). However, only three
relatively factor-pure aptitude tests constituted
the operational screening battery, whereas the
experimental composite was based on seven in-
struments. Moreover, the seven-test composite
measure correlated with .51 with the MSC score
but yielded substantially higher validity co-
efficients than the latter (see Table V). The
multiple and partial correlations further illus-

| trate the superiority of the commercial composite.

This superiority is most probably due to the
greater diversity of aptitudes and abilities as-
sessed with the experimental battery.

‘ Range and Distribution of

Training Course Grades

The bar diagrams of Figure 1 reflect the range
and distribution of the training-course grades
(ie, the overall “A+L” grades) of the FAA,
Army, Air Force, Navy and Marine ATC train-
ees. It may be noted that the means, medians,
and variances of the criterion variable are re-

‘orce trainees are
if circumstances
reening and selec-
have taken a dif-
findings do not
ional tests would
an Classification
perational at the
consisted of many
riety of aptitude
ndex” (GI) rep-

mable to assume
inherent in the
would have been
of the various

markably similar for the two groups trained at
Keesler AFB. Although the distributions per-
taining to the Navy and Marines appear to be
much alike, the differences between the means
and variances of the two groups proved to be
statistically significant. However, even greater
differences characterized the two FAA samples.
In fact, the distribution of grades for Sample 1
resembles those of the Army and Air Force
trainees much more than that of Sample 2, and
the latter is more like that obtained for the Navy
trainees.

It is difficult to ascertain’ the reasons for such
disparities. It®is improbable that the training-
performance evaluation standards were entirely
uniform from facility to facility but the differ-
ences between the FAA samples, and between

’

course on
tase
%
"0
* ms
H LH rcstan
0 Ly
P25
»
"10
6 Fo
«
Tete 7 7 =
vee "4 a sem IAF Meer arin
Yeh case in we See iH
ean Grade 85:0 ao Py no
Soror Grades * 3:6 a3 oe ws 83 ry
Fracre 1. Comparative Ranges and Distributions of

ATC Training Course Grades for FAA, Army, Air
Foree, Navy and Marine Corps ATC Trainees.

the Glynco groups, provide the basis for suspect-
ing that other factors might have been involved.
Nonetheless, the differences do not necessarily
imply a lack of reliability in the criterion meas-
ure. The differential levels of performance, in
terms of the most reliable criteria, may have
stemmed from possible differences in the instruc-
tional materials, lectures, or laboratory problems
presented to the various groups. Policies affect-
ing motivation may have varied and it is also
possible that the groups were unequal in terms
of learning capacity.

Predictive Effectiveness of Experimental 7-Test
Composite Versus Each MSC Index

The effectiveness with which the Composite
7-Te&t: Scores might be used to forecast perform-
ance in the military ATC training courses is
illustrated in Figure 2. In deriving the data
reflected in the four graphs, a frequency distri-
bution was first prepared of the “Overall Train-
ing Grades for the combined Army and Air
Force groups. Those cases comprising the ap-
proximate lower one-fifth (22.1 per cent) of the
distribution were designated as “Marginal-Pass


AIR TRAFFIC APTITUDE TEST MEASURES OF MILITARY AND FAA
CONTROLLER TRAINEES

I. Introduction.

The present report presents a summary and
comparison of the results obtained in two similar,
yet separate, studies. In each of the two investi-
gations, aptitude tests were administered, on an
experimental basis, to groups of men as they
entered military Air Traffic Control (ATC)
training. In both instances, the prime objective
was to determine and compare the validities of
the respective and combined test measures with
the validities of military-screening-and-classifica-
tion (MSC) aptitude test scores as predictors of
performance in military ATC training schools.
One of the studies involved the assessment of
Navy and Marine Corps personnel in the ATC
Training School at the Glynco Naval Air Station
(NAS), Georgia; the other pertained to ATC
trainees of the Air Force and Army who were
examined as they eptered a basic ATC training
program at Keesler AFB, Mississippi. A report
of the Glynco NAS study was published pre-
viously’ and the results obtained in the Keesler
AFB study, though not heretofore published,
were provided to the USAF Chief of Staff and
to officials of the Federal Aviation Administra-
tion (FAA).

Inasmuch as the seven tests chosen for experi-
mental use at both military training schools had
been validated with civilian ATC trainees in
previous research by the FAAys Civil Aeromedi-
cal Institute (CAMI), the availability of cor-
responding test performance data for the various
groups presented an opportunity for a compara-
tive study of the aptitude levels characterizing
the Air Force, Army, Navy, Marine Corps, and
FAA ATC trainee samples. Moreover, CAMI’s
previous researcht had included a regression
analysis and development of an equation whereby

4%, composite measure of performance on four of
the seven commercial tests could be used to esti-
mate an individual’s score on a U.S. Civil Service
Commission (CSC) battery of aptitude tests
with which most applicants for FAA ATC train-

ing are currently screened. Since the FAA has
traditionally selected the majority of its trainees
from an applicant pool of former military con-
trollers, a second major objective of the study
was to estimate the military ATC trainees’ per-
formance on the operational screening battery.
It was presumed that such information would
be of considerable value to FAA officials in the
development of future selection-and-qualification
standards.

Plans for the earliest of the two investigations
were first conceived in 1965 when representatives
of the Glyneo NAS visited CAMI for further
indoctrination regarding the underlying research,
development, and effectiveness of aptitude testing
and other procedures used in .the screening and
selection of applicants for FAA controller train-
ing. Naval officials subsequently discussed the
possibility of being permitted the use of the
operational battery of CSC tests for experimental
administration and validation at Glynco. Al-
though a number of policy reasons precluded
this approach, CAMI scientists suggested an
alternate plan, subsequently accepted, involving
seven commercially-published tests which had
been validated in previous research with FAA
trainees. After completion of the Glynco study,
Keesler AFB officials asked that a parallel study
be conducted on samples of Air Force and Army
ATC traineeg. Inasmuch as the research design,
test. battery, and procedures developed for both
studies were largely predicated upon the findings
obtained in CAMI’s research with FAA trainees,
a brief review of that research?‘ and the FAA’s
selection and recruiting practices is deemed
pertinent.

FAA Selection and Recruiting History

Eligibility for controller training with the
FAA has traditionally included consideration of
an applicant’s pre-employment experience, his
educational background, the outcomes of an
interview with management officials, and the re-
sults of a medical examination. Previous rele-



stan
erro
othe
aval

SEVSCS Sete T SSS hPMeEsegese
SS Ss ages axe =
SSESRFSSSESCSAESERGSEE

poyeuysg

4Tr

of £

eack
bety

ures

proc
Arn

for
Eac.

the

ewe, DY RY

2g 223 #3
>

LESESHERSE

210
havi
amp

wert
trai
higt

“se0ureay, OLV Sd10Q aujABP Joy oouvMAOJzeg OSINOD FUjNTBIY, SVN OOUATH JO sIOJIPaIg se SoI0Ig ISL OLV-OSO
DUE aTSodMOD IALF [OTOMMMOD ‘osoduOD yweI-L TETOMMMIOD ‘2100§ u(VA+HA+UV) %» OSW Shop eujrey_ Jo wostredmog

6 san

<2
0€z

a209§ 389] apnzzady OLV-OS) perseuyasy
62Z- 61Z- 60Z- 661- 68T- 6LT- 691- 79
Oz7Z «OTZ = OOZ_—s«*OHT-«sOBT)=—OLT SCOT.

OS8T

@109g§ aI; sodmoD Ysal-y [eFIIeWMIOD

< 9% 67BT- 6YLT- 679T- 6YST- 6Y7I- BYET- 6YZI- > %
OST OS9T OSST OSHT OSE OSZI OSTI 64IT

oor ele - N
0z1 oz1
2109g aqysoduoD 382j-/ [eyII9NMOD 91098 ,,(WAFId+AV) €/T,, ISH 84209 ‘puyzey
<9 G6NSE- GEE- GYTE- 6762- GHLZ- GHSZ- BYEZT- > 9 <9 GET- HET- 6ZI- MI- GII- YIT- 6OI- 779
OSSE OSEE OSTE OS6Z OS/2 OSsz OSEZ OSTZ “ErIZ Ovl SET O€t SZI OZT SIT OTL SOT 0T
0z
04
09
og

Bapery a8inoy-BuzupeIry JLY CoUATD jo
uoFINgFAIstq JO syIzTA-Inog reddy uz ss

sesey [Tey 30/pue
SSUq-TEUTZ IE

24



ACKNOWLEDGMENT

The author is indebted to Lieutenant Commander F. J. Gallagher
(Glynco NAS), to Lieutenant Colonel F. E. Sayer (DOD-FAA ATC Train-
ing Command), to Mr. M. L. Jay and Mr. C. J. Williams (Keesler AFB),
and to Mr. R. Klotz and Mr. R. F. Jordan (FAA AT representatives) for
arranging the studies and for the collection of data at the Glynco and Keesler
facilities. The assistance of Sharon Bowles, Carolyn Harshaw, Carolyn Lay,
John Mathews, Mary Ann Nelson, and Alice Weikel in analyzing the data is

gratefully acknowledged.



$s.
Higher

79
65

169

180
81

262

ilitary-Sereen-
s of 189 and

eesler AFB
Is requested
ine the ef-
rimentally-
t be used to
tion proce-
t finding of

| | 189 & < 190 - 209 210 - 229 230 6 >
Sample ¢
and Type ae
Sample 1 96 33 37 bry 210
FAA 45.7%, 25.2% 17.6% 11.4%
Sample 2 82 0 ZY 36 35 183
FAA 4b 8h 16.4. G4 19.7% 19.1%
Sample 3 49 34 43 a3 169
army 29.0% 20.1% V3. 25.6% 2 25.6%
Sample 4 427% 397 7 30 169
Air Force 24.9%, 23.1% 17.8%
Sample 5 188 1467 1547 139 628
Navy 29.9% 23.0% 245% 22.1%
Sample 6 81 134 313
Marines 25.9% 42.8%
Fiavre 11. Percentages of ATC Trainees Within Each Sample Having Actual or Predicted OSC-ATG Aptitude

Screening Test Scores in Non-qualifying, Qualifying and High Ranges.

(Note: The minimum qualifying score on the CSC-ATC Test is currently 210; 190 represents a former screening standard. The

183 FAA Ss comprising Sample 2 took both the CSC Test and four conmercially-published aptitude tests.

An equation, based on

A evrrelation of .81 between the two composite measures, was used to estimate CSC Test performance for the Ss of all other

samples.)

these studies was that the screening potential of
existing military-screening-and-tlassification ap-
titude-test scores could be more fully exploited
to achieve that end. For reasons unknown, but
perhaps due to recruiting demands, 16.5 per cent
of the Navy trainees, 14.8 per cent of the Air
Force subjects, about 5 per cent of the Army
subjects, and one Marine were assigned to ATC
training even though their MSC scores were be-
lgw the normally-prescribed minimum (which
was 110 on the different test batteries employed
by the Army, Na¥y, and Marine Corps, and 65
in terms of the Air Force’s ACB General Apti-
tude Index). About 42 per cent of the 138 men
selected from these lower MSC-score categories

27

either failed of marginally passed their training
courses compared to 18.2 per cent of the other
1085 military ATC trainees involved in the
study. Moreover, other findings suggested that
the recommended aptitude-screening standards
could be raised by several points to increase
their effectiveness. For example, MSC scores of
less than 115 reflected the aptitude levels of 234
Army, Navy, and Marine trainees, and 51 airmen
possessed a General Index of less than 70. Over
40 per cent of these 285 students either failed or
marginally passed training whereas only 15 per
cent of the remaining 938 did so.

Only two of the seven commercially-published
tests which were experimentally administered at


A ARTCC train-
TATC trainees of
and questionable
neo study on the
le 1 and on the
ples prompted the
summary measure.
sion analysis was
f test intercorrela-
been established

approached with a
ree DAT tests and
ly, a new summary
-Test Commercial
ingle-digit factor
was computed for
idation data were
for the original
, 2 similar score

lated .54 with
e P-F criterion.
ig subjects were
from their Com-
s through use of
n. Inasmuch as
sented “perfect”
Test Scores, they
and P-F in the
js, the only mean-
‘able II for the
¢ obtained with

Test Scores

-y-Screening-and-
‘presented in the
the MSC scores
the trainees, it
mples are quite
riable, and that
ently represents

& gross underestimate of the true validity. More-
over, unless complete independence (ie., no re-
lationship or correlation) exists between the
MSC test scores and the commercial test per-
formance measures, the validity coefficients for
each of the latter should also be considered as
attenuated. (Procedures for correction of at-
tenuation effects were not employed, for either
the MSC or commercial test: scores.)

The Army “MSC V+A” score correlated .35
with Course Grade and 42 with the Progress
Index. For the Air Force sample, correlations
of .35 and .36 were obtained for the ACB-GI
index versus Course Grade and PI. Similarly,
the Navy’s MSC score yielded validity coefficients
of .44 and 27, whereas those of the Marine Corps’
aptitude screening measure were 21 and .12.
Inasmuch as an analysis revealed that the ma-
jority of the Marines possessed exceptionally-
high MSC scores, each of the latter coefficients
should be regarded as grossly attenuated. How-
ever, it is also interesting to note that every
experimentally-derived composite validated at
considerably-higher levels than did the Marine
Corps MSC score. Evidence to be presented
later indicates moderate-to-low correlations ex-
isted between the MSC score and each of the
experimental composites.

Of the various experimental composites, only
that based on the entire group of seven tests
yielded higher validities than obtained with the
Army MSC score. In contrast, every commercial
composite validated with Sample 4 at a signifi-
cantly higher level than did the Air Force’s
ACB-GI Index. The experimental measures
also yielded appreciable validities for the pre-
diction of the training-course grade averages
and the Pass-Fail status of the Navy trainees
but, in every instance, at a bower (though not
significantly-lower) level than the CGT+ARI
score,

Based on the results of all such comparisons,
one might suspect that performance measures on
tests similar to those comprising the commercial
battery (and particularly the three DAT tests
and CTMM Inference) could be used in supple-
gnent with each type of MSC score to achieve a
moderate degreegot improvement in the selection
process. As will be discussed later, however,
several analyses indicated that, while the degree
of improvement attainable with such a procedure

would indeed be slight, the adoption and use of
a higher minimum MSC score (alone) would
serve the purpose almost equally as well.

Multiple R's Versus Validities of Composites

A series of multiple-regression analyses were
accomplished on the correlational data obtained
for Samples 1, 2, 3 and 4 and for the combined
Samples 5 and 6. (Intercorrelations for the
Navy and Marine trainees were available from
the Glynco Study for the combined groups only.)
For each of the five groups, the first analysis
was aimed at determining the maximum possible
validity attainable with the complete seven-test
commercial battery for prediction of Course
Grade and the respective “beta weight” for each
test needed to obtain the indicated “R” (or opti-
mum validity coefficient). Using the same cri-
terion, two additional regression analyses were
undertaken; one focused upon the three DAT
tests and CTMM Analogies and the other upon
the three DAT’s and CTMM Inference. Inas-
much as simplified factor weights rather than
regression-derived weights, had been used in
computing each composite score, a comparison
between each “IR” and the corresponding validity
coefficient for each composite should indicate the
amount of loss or “shrinkage” resulting from use
of the simplified weights. The data for such a
comparative study are presented in Table IIT.

For Sample 1, the first analysis revealed that
if the performance scores on the seven commer-
cial tests had been weighted according to a spe-
cific set of values (as indicated by the regression
analysis), the resulting summary score would
have correlated 49 with Course Grade. In con-
trast, the Commercial 7-Test Composite Score
based on the,simplified factor weights yielded a
validity coefficient of .39. For the same sample,
an “R” of .45 was established for the three DAT
tests and CTMM Analogies, whereas the validity
coefficient for the Commercial 4-Test Composite
Score was 32. An “R” of 45 was also obtained
for the three DAT tests and CTMM Inference
and this, too, was considered substantially above
the .36 for the New 4-Test Composite Score. In
every instance, however, the difference between
the corresponding coefficients failed to be sta-
tistically significant. For each remaining sample,
the validity of each composite score compared
more favorably with the corresponding “R.” In
other words, even if the regression-derived factor



TABLE V.

AS THE COMMERCIAL 7-TEST COMPOSITE FOR IMPROVEMENT OF MILITARY ATC SELECTION PROGRAMS

Course Criterion r with MSC Score
Grade Pi P-F Theoretically Held Constant
r Yr v Course
Variables R R R Grade PL P-F
Sample 3 - Army
MSC "V+A" Score 35 «42
Comm. 7-Test Composite 37 +40 222 221
Combined 7-Test + MSC +40 46
Sample 4 - Air Force
MSC "ACB GI” Index 35 +36
Comm. 7-Test Composite 52 47 41 +34
Combined 7-Test + MSC 52 48
Sample 5 - Navy
MSC "GCT+ARI" Score 44 +28
Comm. 7-Test Composite AL «26 15 09
Combined 7-Test + MSC +46 229
Sample 6 - Marine Corps
MSC "1/3(ARHVE+PA)" 221 «12
Comm, 7-Test Composite Al «23 36 +20
Combined 7-Test + MSC 41 +23

*A11 coefficients except the .09 are statistically significant at the .05 level or less.

variable that variance associated with the MSC
score and thus estimate the “residual validity”
of the experimental composite. In other words,
the objective was to assess the composite’s pre-
dictive validity exclusive of that which it held
in common with the MSC score (ie., its validity
with the MSC scores theoretically nullified or
held constant). The resulting secoitd-order cor-
relations are shown in Table V. With Course
Grade serving as the criterion, coefficients of .22,
41, 15 and .36 were obtained for Samples 3, 4,
5 and 6, respectively. With PI or P-F as the
criterion, the composite yielded residual validittes
of 21 for FAA Sample 1, .34 for the Air Force,
.09 for the Navy, and .20 for the Marines.

Th data, together with the results of the
multiple regression analyses, illustrate the po-
tential with which th® Commercial 7-Test Com-
posite Score or a similar measure could be used
in supplement with the MSC score to improve
each military ATC-selection program. When

14

considered from a practical viewpoint, however,
results as obtained with the Army and Navy
groups would fail to warrant such a major revi-
sion in the selection procedures. In contrast, the
findings relating to the Air Force trainees are
impressive and indicate that, if circumstances
had permitted, the-aptitude screening and selec-
tion of ATC personnel could have taken a dif-
ferent form. However, the findings do not
necessarily imply that additional tests would
have been needed. The Airman Classification
Battery (ACB), which was operational at the
time this sttidy was conducted, consisted of many
different tests, covering a variety of aptitude
factor areas and the “General Index” (GI) rep-
resented but one of four summary measures de-
rived through differential weighting of the
numerous subtests. It is reasonable to assume
that literally all the variance inherent in the
Commercial 7-Test Composite would have been
attainable by combining the scores of the various

MULTIPLE AND PARTIAL CORRELATIONS* ILLUSTRATING THE POTENTIAL OF A MEASURE SUCH}ACB subtests

words, had t!
leritical as to ;
‘of concomitai
| personnel for
‘researchers c:
a “customize
the validity
composite.
Since the
known to ht
scores, rathe
operational
restriction-of
relatively fa
the operatio
experimenta.
struments.
measure cor.
but yielded
efficients th:
multiple an
trate the su}
This superi
greater div
sessed with

The bar «
and distrit
(i.e, the o
Army, Air
ees. It mi
and yarian
markably +
Keesler A.
taining to
much alik:
and yariar
statisticall:
differences
In fact, tl
resembles
trainees 17
the latter
trainees.

Tt is di
disparities
performar
uniform {
ences bet