PROBLEMS IN AIR TRAFFIC MANAGEMENT 

IV. COMPARISON OF PRE-EMPLOYMENT. 

JOB-RELATED EXPERIENCE WITH APTITUDE 

TESTS AS PREDICTORS OF TRAINING AND 

JOB PERFORMANCE OF AIR TRAFFIC 

CONTROL SPECIALISTS 

DAVID K. TRITES, Ph.D. 

Chief, Selection Section -

BART B. COBB, M.S. 

Selection Section 

Psychology Branch 

63-31 

FEDERAL AVIATION AGESCY 

CIVIL AEROMEDICAL RESEARCH INSTITUTE 

AERONAUTICAL CENTER 

OKLAHOMA CITY, OKLAHOMA 

DECEMBER 1963 
PROBLEMS IN AIR TRAFFIC MANAGEMENT 

IV. COMPARISONS OF PRE-EMPLOYMENT, JOB-RELATED 

EXPERIENCE WITH APTITUDE TESTS AS 

PREDICTORS OF TRAINING AND JOB PERFORMANCE 

OF AIR TRAFFIC CONTROL SPECIALISTS* 

David K. Trites, Ph.D. 

Bart B. Cobb, M.S. 

In addition to passing a second class aviation 

medical examination, individuals applying for 

training as an Air Traffic Control Specialist 

(ATCS) with the Federal Aviation Agency 

have, in the past, been required to have previous 

experience which was considered relevant to air 

traffic control work. This would seem to be a 

reasonable selection procedure, but other re· 

search' 2 3 has demonstrated that aptitude 

tests effectively predict training and job per· 

formance of individuals selected in this manner. Thus, assuming the value of the medical 

requirements, the predictive worth of jobrelated experience can be questioned. 

To evaluate the effectiveness of experience 

as a predictor, the experience backgrounds of 

several hundred ATCS trainees were examined. 

All of the trainees had been selected on the 

basis of the medical and experience requirements only, for training in either Enroute ( Center) or Terminal (Tower) air traffic control 

procedures. These two training courses have 

been described in more detail in an earlier report." In addition, approximately 94 percent of 

the Enroute trainees and 88 percent of the Terminal trainees had participated in an experimental program of aptitude testing. The present 

report summarizes the findings with respect to 

the prediction of training and job performanc~ 

by various types of pre-employment, jobrelated experience, selected demographic variables and an aptitude test battery. 

PROCEDURE 

Samples - Trainees entering the ATCS 

Enroute or ATCS Terminal training courses at 

the FAA Academy, Oklahoma City, in September, 1960, through July, 1961, are to be referred 

to as the Enroute Sample (E) and the Terminal 

Sample (T), respectively. Of 503 Enroute 

Sample trainees, 471 participated in the experimental aptitude testing program; among the 

503, 129 (25.6%) failed to complete the training course, and among the 471, 118 (25.1%) 

failed to complete the course. Of 242 Terminal 

Sample trainees, 212 participated in the testing program; among the 242, 19 (7.9%) failed to complete the course, and among the 212, 

17 ( 8.0% ) failed to complete the course. Minor 

variations in the number of cases entering into 

various calculations and distributions are due 

to missing data or to a few individuals who 

withdrew from training for reasons other than 

failure. These latter individuals, whose withdrawal was necessitated by reasons beyond their 

control, such as illness, de2th in the family, etc., 

were considered as neither successes nor failures in training and in most instances had no 

criterion gata available. 

For both samples, Table 1 contains means 

and standard deviations of age and four of the 

criterion variables to be described subsequently. 

The data revealed no apparent differences, or 

biases, between the total sample and that portion which was tested. 

-1-
TABLE 1 

Compa1'ison cf Certain Characteristics of the Total and the 

Aptitude-Tested Parts of the Enroute and Terminal Samples 

Enroute 

Tested Total 

Variables x S.D. N. x S.D. 

Age 28.5 6.3 470 28.5 6.4 

Academic Gd. 83.6 7.4 470 83.6 7.3 

Lab Gd. 78.1 13.0 467 78.0 13.0 

A+LGd. 80.& 9.3 468 80.8 9.2 

Supervis. Rating 2.45 .54 301 2.45 

Experience and Demographic V ariables - S;:•nce of these data was the "ATCS 

Registration Sheet" which each student completed immediately after arrival at the FAA 

Aeronautical Center. For those who were tested, 

the form was completed on the Saturday immediately preceding the start of their training 

course on the following Monday. For those not 

tested, the form was completed on the day the 

course began. Table 2 contains descriptions of 

the 12 specific types of experience recorded on 

the form, 3 variables representing a summation 

of the individual types of experience, the demographic variables of age and education, and the 

aptitude test variables. In coding the variables 

no attempt was made to achieve normalized 

distributions of the data. 

.53 

Aptitude Test Variables- The aptitude 

test variables represented only a small part of 

a much larger test battery described by Cobb. 2 

The five tests described in Table 2 are those 

which Cobb found most valid as a composite 

for prediction of Enroute training course criteria. The composite score was obtained by 

multiplying the scores on each of the five tests 

by weights which were proportional to the inverse of the standard deviation of each test and 

summing the weighted test scores. Selection of 

the five tests and determination of the weights 

given in Table 2, were based on Cobb's analysis 

of data for 124 ATCS trainees who form part 

of the present Enroute Sample. Thus, relationships between the composite score and training 

level criterion measures were somewhat inflated 

in that sample. However, since the composite 

score was subsequently validated and since the 

sample ;:,f 124 subjects comprises · approximately one fourth of the presen~ sample, the 

spurious inflation of relationships should not 

be too serious. 

Terminal 

Tested _Total 

N. x S.D. N. X S.D. N. 

501 27.8 5.9 212 27.9 5.9 241 

502 87.0 5.3 211 87.1 5.3 241 

499 83.3 5.2 207 83.3 5.4 237 

499 85.0 5.6 211 85.0 5.6 241 

318 2.42 .61 180 2.43 .66 204 

Training-Level Criterion Variables - Four criterion measures were developed from 

information entered by FAA Academy personnel on each trainee's final "Evaluation of Performance" record. These were; Academic Grade 

Average (Acad), Laboratory Grade Average 

(Lab), Combined Academic plus Laboratory 

Grade Average (A+L), and Pass or Fail (P-F) 

in the training course. 

The Academic Grade Average was obtained 

by a -,raging the grades obtained on academic 

examinati<>ns of seven areas administered at an 

intermediate stage and at the conclusion of the 

training course. In the latter instance, the seven 

individual subtests constituted the Air Traffic 

Control Specialist Certification Examination 

(ATCSCE). Normally, the Academic Grade 

Average was based upon the 7 intermediate 

grades plus the 7 final grades; but a failing 

grade of 70 or less at the intermediate stage of 

training for any of the 7 academic areas usually 

entailed a "retake" of the examination for the 

specific areas involved. To be graduated from 

training no more than two failing grades could 

be obtained by a trainee on the first administration of the ATCS Certification Examination; 

and he had to successfully pass the "retake" 

examination for these areas. A trainee was automatically eliminated if he had three or more 

failing grades on the ATCSCE. Thus, theoretically an Academic Grade Average could be 

based on considerably more than 14 grades. In 

practice, the number infrequently exceeded 15 

or 16. 

The Laboratory Grade Average for Enroute 

Sample trainees was based on three final laboratory grades reflecting perfonnance in Strip 

Writing, Control Procedures as an Assistant 

Controller, and Control Procedures as a Journeyman Controller. These grades reflected per-

-2-
Variable Name & 

Abbreviation• 

Experience Variab]es 

1. Pilot Experience CPil) 

2. Radio Operator: Air to 

Ground Communications 

(Radio:Air/Grnd) 

3. Ground Control 

Intercept ! CCI) 

4. Station lStat) 

5. Ground to Air Communica· 

tions (Grnd/ Air) 

6. Point to Point Conununications (p ro P) 

7. VFR Tower tVFR Tow) 

8. Approach Control: Tower 

lApp Con Tow) 

9. Radar Approach Control: 

Tower {Rad App Con Tow) 

10. Center (Cent) 

ll. Ground Controlled Approach 

IGCA) 

12. Radar Approach Control 

Center IRAPCON) 

13. Sum of Communications 

Experience ("l Comm) 

14. Sum of Air Traffic 

Experience (SA Tl 

15. Sum of Re!evant E~perience ( S Rei) 

Demographic Variables 

16. Age 

17. Education lEduc) 

l.'ABLE 2 

Description of Variables 

Descriptit>n and Coding 

Coding for Variables 1 & 2 

Amount of E;,:perience 

No experience reported 

Less than 1 year 

12 months through 23 months. 

2 years through 4 years 

5 years through 6 years 

7 years through 8 years 

9 years through 10 years 

II years through 15 years 

16 years or more 

Codes 

I 

2 

3 

4 

5 

6 

7 

8 

9 

Long ~l.ange. hi~h dti1:ude interception 

military aircraft control and warning 

technique US"'...d by 

units 

:\ unit primarily engaged in ~round to air communication."' 

and pilot hrie{in~& 

Communcations from one fixed ground po~nt to auother 

fixed ground point 

A tower controlling air traffic under Visual Flight Rules 

lVFR) 

A tower capable of controlling air traffic under Instrument 

Flight Ru1es { IFR) but without access to radar 

A rower with access tC~ radar as an aid in controlling air 

traffic 

An air route traffic control center 

A ground radar system us-ed to assist aircraft during landing 

USAF radar system used fvr approach control at Air Forceairfields; similar system in use by the Navy is called a 

Radar Air Traffic Control Center eRA TCC) 

Coding for Variables 3 through 

Amount of Experience 

No experience reported 

Through 3 months 

4 through 6 months 

7 months through l year 

13 months through 2 years 

25 months through 3 years 

37 months through 5 years 

6 years through 10 years 

11 years or more 

12 

Sum of individual experience variables Nos. 4, 5, & 6 

·C<>des 

1 

2 

3 

4 

5 

6 

7 

8 

9 

Sum of individual experience variables Nos. 7 through 12 

Sum of experience variables Nos. 3. 13, and 14 

Chronological age to nearest birthday on date of entry into 

A TCS training 

Coding for Education 

Amount of Education 

None reported 

Less than High Schoo1 Graduate 

High S"hool Graduate 

Less than l year of college 

1 year of college 

2 years of college 

3 Years of college 

4 year~ of college 

5 years of college 

6 or more years of college 

-3-

Codes 

Blank 

I 

2 

3 

4 

5 

6 

7 

8 

9 
TABLE 2 (continued) 

Aptitude Test Variables 

#18. DAT Space Relations 

(Sp) (Weight = 6) 

#19. OAT Numerical Ability 

(Num) (Weight = 13) 

#20. DAT Abstract Reasoning 

(Abs) (Weight = 14) 

Identify solid figures that can be made from an unfolded 

pattern (40 items; scored: Rights minus Wrongs) 

Test of arithmetic or computational skill (40 items; scored: 

Rights minus 1,4 Wrongs) 

Indicate which of a series of choices (figures) ~roperly carries out a principle of logical development exhibited by a 

sequence of figures (50 items; scored: Rights minus 1,4 

Wrongs) 

# #21. CT\IM Analogies 

(Analog) 

(Weight = 49) 

Seven drawings of different objects are presented for each 

item. The first object has a definite relationship to the 

second which must be recognized in order to identify~ by 

analogy, the drawing among the last four which is similarly 

related to the third drawing (15 items; scored: Rights) 

* *22. Air Traffic Problems Determine whether aircraft may be permitted to change alti-

(ATP) (Weight = 18) tude without violating a specified time-separation rule 

(30 items; scored: Rights minus Wrongs) 

23. CompQsite Aptitude Sum of aptitude test variables Nos. 18 through 22, each 

Test Score (Comp) weighted as indicated. 

~ Abbreviations are given in parenthesis following the names of the variables. 

#Part of the Differential Aptitude Test <OAT) Battery, Form A, 194i, published by the 

Psychological Corporation, New York, N. Y. 

##Part of the California Test of Mental Maturity CCTMM)~ Advanced. Form A. 1957 published 

by the California Test Bureau, Los Angeles, California. 

•• Originally develo;Jed by the American Institute for Research. Pittsburgh, Pa .• under con· 

tract with the Civil Aeronautics Administration in 1950. Form used in the present research 

was an extensive revision of the original test. Revision prepared by tb~ Selection Section. 

Psychology Branch, Civil Aeromedical Research Institute. 

formance in simulated air traffic control work. 

An unsatisfactory grade of 70 or less for any 

one of the three areas normally resulted in 

elimination of the trainee from the course. The 

Laboratory Grade Average for the Terminal 

Sample trairiees was based upon a single grade 

reflecting performance in terminal approach 

·control operationS: If the grade were less than 

70, the trainee was eliminated from the course. 

The Combined Academic plus Laboratory 

Grade Average was the simple arithmetic mean 

of the Academic and Laboratory Grade Aver· 

· ages of each trainee. Data from incomplete records of trainees who were eliminated or withdrawn from training were treated simibrly; 

but the averages were based on fewer grades. 

All trainees successfully completing the 

training course were considered as Pass cases; 

those unsuccessful, as Fail cases. Eight trainees 

in the Enroute Sample and t·do in the Terminal 

Sample were not considered as· either Pass or 

Fail and did not enter into any calculations 

involving the Pass-Fail criterion. 

fob-Performance Criterion Variable 

-Approximately 10 months after trainees had 

graduated from the FAA Academy, a letter 

describing the research project and data collection forms were sent to the Chiefs of the 

facilities to which the trainees had been assigned. For each member of the sample at his facili· 

ty, the Chief was asked to have four supervisors 

rate each subject using a 15 item checklist performance evaluation form. The form contained 

' ~ms related to work habits, ability, judgment, 

_ad reasoning, emotional stability, and relationships with others. The possible ratings were: 

Excellent, Vel'y Good, Good, Fair, and Unsatisfactory. the Average Supervisory Rating 

(Super) was computed by assigning weights of 

4 through 0 to ratings of Excellent through Un· 

satisfactory, respectiveiy, summing all of the 

items rated by all supervisors, and dividing the 

sum by the total number of items rated. Unfortunately, it was not possible to obtain four 

supervisory ratings on all subjects, but the 

average number of forms completed per subject was 3.8. In an earlier study a cc-rrected 

split-half reliability of .75 was obtained for 

Average Supervisory Ratings computed from a 

rating form containing 12 of the 15 items used 

in the present form. 3 

Statistical Methodology - To indicate 

the degree of relationship between eaeh criterion variable and the experience, demographic, and aptitude test variables, productmoment or point-biserial correlation coefficients 

were computed for the Enroute and Terminal 

-4-
Samples, separately. In every instance, correlations were computed using the maximum possible nu.'llber of cases. 

Since many of the variables had an appreciable relationship with age, the partial correlation technique wes employed to eliminate, or 

nullify statistically, the influence of age. Both 

the original, or first order, and the partial cortables were then established for the experience 

dichotomy vs. above and below the approximate 

median of the Academic plus Laboratory Grade_ 

Average, and the experience dichotomy vs. Pass 

or Fail in training. The significance of each 

contingency ta!:ole was teste!! by chi square. 

RESULTS 

relations are reported; but in the following dis- Means and standard deviations of all varicussion reference usually is made only to the abies listed in Table 2, except age, (see Table 

partial correlations. l), are given in Table 3. It is readily apparent 

Although the means and standard deviations that many trainees had little, or none, of the 

of some of the variables revealed an extreme individual types of experience. This difficulty 

skewness in their distributions, conventional was overcome to a large extent by pooling inditests of statistical significance were applied to vidual types of experience to . form the three 

the first order correlations. In testing signifi- summary variables: Sum of .t\,ir Traffic Expericance of the partial correlations the smallest N ence, Sum of Communications Experience, and 

involved in the computation of a particular Sum of Relevant Experience. 

partial was used. This resulted in a conservative Experience Variables vs. Criteria -

test of significance. First order and partial correlations of the five 

As a partial check on the appropriateness of criterion variables with the experience variables 

the conventional test of the first order correla- and age are presented in Tables 4 and 5: The 

tions, joint frequency distributions were com- partial correlations indicated that many of the 

piled for the 12 individual experience vari- experience variables had either no relationship 

abies vs. the Academic plus Laboratory Grade to training and job performance or had a nega~ 

Average, a11d the 12 experience variables vs. tit>e relationship; i.e., the greater the amount 

the Pass-Fail criterion. To test the significance of the bivariate distributions all the experience of some specific types of experience the less 

variables were dichotomized as: 1w experience well the trainee did. This was particularly true 

vs. some experience. Two by two contingency , of GCI, and communications type experience. 

TABLE 3 

Means and Standard Deviations of the Experience Variables, Test Variables, and Education 

Based on the Maximum Possible Number of Trainees in the Enroute and Terminal 

Samples Separately 

Mean Std. Dev. N 

Variables E T E T E T 

L Pilot 2.38 1.92 2.64 . 2.16 503 242 

2. Radio: Air/ Grnd 1.38 1.28 1.23 Ll3 503 242 

3. GCI 1.39 1.23 1.42 l.D7 503 242 

4. Station 1.39 1.10 1.38 .68 503 242 

5. Grnd/ Air 2.12 1.72 2.22 1.88 503 242 

6. P to p 1.39 1.23 1.34 1.08 503 242 

7. VFR Tower 2.68 4.19 2.30 2.51 503 242 

8. App Con Tow 1.80 2.53 1.68 2.16 503 242 

9. Rad App Con Tow 1.14 l.l4 .77 .72 503 242 

10. Center 1.44 1.20 1.28 .78 503 242 

li. GCA ~.21 2.26 2.14 2.25 503 242 

12. RAPCON 1.75 1.46 1.68 1.35 503 242 

13. ::S Comm 4.89 4.07 3.51 2.57 503 242 

14. ::SAT 11.0! 12.81 4.83 4.21 503 242 

15. ::S Rei 17.28 18.10 5.06 4.46 501 242 

17. Educ 2.90 2.79 1.59 1.49 489 241 

18. Space 60.79 58.98 18.93 19.62 472 212 

19. Numerical 22.!7 21.68 8,!3 7.52 474 212 

20. Abstract 33.92 34.48 7.14 7.1.6 474 212 

21 Analogies 6.80 6.71 1.99 2.03 475 212 

22. ATP 13.46 13.34 5.84 6.06 475 2!2 

23. Composite 1703. 1688. 363. 375, 471 212 

-5-
V;riables 

1. Pilot 

2. Radio: Air/Grnd 

3. GCI 

4. Station 

5. Grnd/Air 

6. P toP 

7. VFR Tower 

8. App Con Tow 

9. Rad App Con Tow 

10. Center 

ll. GCA 

·12. RAPCON 

13. ::£ Comm 

14.:;: AT 

15. :;: Rei t 

16. Age t 

Total N 

1. Pilot 

Variables 

2. Radio: Air/Grnd 

3. CCI 

4. Station 

5. Grnd/ Air 

6. P to P 

7. VFR Tow 

8. App Con Tow 

9. Rad App Con Tow 

10. Center 

11. GCA 

12. RAPCON 

13. ::S Comm 

14. ::SAT 

15. :;: Rei 

16. Age t 

Total N 

TABLE 4 

Enroute Sample: First-Order Correlations and Partial Correlations {Age Held Constant 

Statistically) between Experience Variables and Criterion Measures 

A±L 

r Partia! r 

-12*"' 

--23** 

-14** 

-11* 

-15** 

-12** 

14** 

17** 

06 

14** 

13** 

I6** 

07 

-I9*"' 

-13** 

-11· 

-Is•• 

-12** 

09* 

I6·· 

06 

12** 

-IS* 

24** 

06 

-18** 

29** 

II 

-27** 

499 

P - F' Acad Lab Super r Partial r r Partial r r Part!a! r r Partial r 

-17** 

-15** 

-12** 

-11* 

-13** 

-15** 

16"'* 

16** 

06 

08 

18** 

15** 

-18** 

30** 

13** 

-28** 

02 01 14** -18** 02 -17** -07 

-11* -20** -18** -20** -16** 03 07 

-11* -16** -16** -10* -·09* -01 00 

-11* -17** -17** .....{)6 -06 -02 .....{)! 

-13** -23** -22** -09* -09* -()5 --05 

-15** -21** -21** -os ---04 -Is~.:t --Ist-• 

n• 21.. 18'* o9• os u• oa 

14** 19** 1s•• 13** n• 03 o1 

07 08 08 04 04 02 02 

07 08 07 15°* 14** 09 08 

16'* 14'* 13** II* 09 09 08 

11* 14** n• 15** 1o• os 02 

-18* -29** -29** -10* -10* -10 ~ -09 

2s•• 3I*"' 29** 23** Is=~• 16** I2** 

08 05 02 12** 07 09 06 

-Is•• ~3o*• ~I8** 

495 502 499 318 

64** 

18** 

0~ 

02 

03 

03 

-21** 

.....{)9 

01 

.....{)5 

-10* 

-I7** 

03 

501 

1 Point~biserial correlations; all other correlations are product moment. Decimal points omitted 

from all correlations. 

*Significant at less than the .05 leveL 

• 41 Significant at less than the .01 level. 

fFor ~ Rei and Age the Ns used in the correlation~ with Acad and Lab were 500 and 498. 

respectively. For the other correlations in these columns. the Ns were as indicated in the 

Total N row. 

TABLE 5 

Terminal Sample: First-Order Correlations and Parti1\l Correlations (Age Held Constant 

Statistically) between Experience Variables and Criterion Measures 

A±L p Fl A cad Lab 

Partial r- r Partial r r Partial r r Partial r r r 

SuJ!er 

Partial r ~ 

r 

-12 -02 -13* .....{)! 04 10 -32** -18** -17* -I4* SF* 

-07 -04 03 07 .....{)7 -06 --09 --05 -17* -16• 15* 

-06 .....{)] -04 01 -03 ---{)I -II .....{)4 01 03 21*"' 

-10 -10 02 03 -15* -I5.:. .....{)7 -.Q7 01 02 03 

.....{)7 .....{)8 03 02 -13* -14" .....{)6 .....{)8 -01 .....{)2 -04 

.....{)8 -10 -02 -04 -09 ··-10 -!0 -" 1''· 06 05 -04 

08 03 13* 09 -01 -03 26** 21""* 02 00 -20"'" 

20** zo•• 06 07 lO 10 25** 27** 02 02 uo 

06 09 06 09 04 05 06 II 03 04. 12 

05 04 08 05 07 06 01 ·-02 01 00 .....{)9 

04 06 01 03 Ol 02 03 06 -01 .....{)1 09 

II ll 03 03 09 08 11 11 06 05 -03 

-11 -12 02 01 -I7** ·-17..;.* -10 -11 02 02 .....{)3 

22** 22*"' 15* 14* 10 10 35** 35** 05 04 -07 

13* 13• 14* 14* ---{)I -01 25** 26** 06 06 .....{)3 

-2I** -25*• -09 -34** -09 

241 240 241 237 204 24! 

1 Point-biserial correlations; all other correlations are product moment. Decimal points omitted 

from all correlations. 

* Significant at less than the .05 level. 

• * Significant at less than the .01 level. 

t Correlations between Age and the first four criteria were computed with an N which was 

one less than the Ns given in the Total N row. 

-6-
On the other hand, those types of exfierience 

related more dire:;tly to air trafiic control tended to have positive relationships with the criteria; i.e., the greater the amount of such airtraffic-control related experience the better the 

trainee tended to do. The data for the Sum of 

Air Traffic Experience (Variable 14) and the 

Sum of Communications Experience (Variable 

13) illustrate these contrasting relationships. 

Each statistically significant correlation of I 

AT with one of the criteria had a positive 

sign, whereas each statistically significant correlation of I Comm with one of the criteria had 

a negative sign. Combining the Sum of Air 

Traffic, the Sum of Communications, and GCI 

Experience to form the total Sum of Relevant 

Experience was, therefore, inappropriate. As 

revealed by the attenuated relationships with 

the criterion measures, this particular combination of experience failed to improve the preoiiction of training or job performance. 

Of all the individual experience variables, 

Pilot Experience correlated most highly with 

age resulting in relatively large changes in the 

correlations when the effect of age was statistically controlled. Before age was statistically con· 

trolled, seven of the ten correlations between 

Pilot Experience and the Criterion variables in 

Tables .\ and 5 were significant at less than the 

.05 level and were negative in sign. With age 

controlled, the correlations with Laboratory 

Grade 1o1.nd Average Supervisory Rating in the 

Terminal Sample remained significantly negative; but the non-significant and positive correlation of .01 between Pilot Experience and 

Academic Grade in the Enroute Sample was 

increased to a statistically significant positive 

va:i.ue. Considering the contradictory findings 

in the two samples and inasmuch as 7 of the 

partial correlations were not statistically significant, it seems that Pilot Experience can not 

be <:onsidered as an effective predictor of training or job performance measures. 

A comparison of Tables 4 and 5 revealed 

many more significant <:orrelations in the Enroute Sample. ·Out of 75 experience vs. criteria 

correlations in each sample, 45 were significant 

for the Enroute and 15 were significant for the 

Terminal. The magnitudes of the correlations 

also tended to be greater in the Enroute Sample. 

Noteworthy exceptions to this occurred in the 

case of the VFR Tower and Approach Control 

Tower Experience. Both of these individual 

types of experience were more highly related 

to the Laboratory Grade Average in the Terminal Sample than in the Enroute Sample. A 

similar difference in favor of the Terminal 

Sample was also found with the Sum of Air 

Traffic Experience. The results seemed logical 

since these types of experience might be expected to generalize more readily to Terminal air 

traffic control than to Enroute and he of particular importan<:e in the laboratory phase of 

training. 

With the job performance criterion, two of 

the experience variables in the Enroute Sample 

had significant correlations, but only one of 

these, the Sum of Air Traffic Experience, was 

positive and this at a relatively low level, .12. 

In the Terminal Sample, the only two significant correlations with the Average Supervisory 

Rating were negative in sign. 

As fa~ as the magnitudes of the correlations 

were concerned, only one of the experience vs. 

criteria coefficients exceeded a value of .3 and 

only 11 exceeded a value of .2. Considering 

that there were 60 significant correlations in 

the tables, 29 of which were negative, and that 

of the 11 correlations greater than .2, 3 were 

negative, it can be concluded that previous jobrelated experience \';as a poor predictor of subsequent training and job performance. In fact, 

as the large number of negative correlations 

indicat~, some types of experience which have 

been considered as qualifying should not have 

been. 

The distribl!tion ,Q(lta and the significance of 

the chi squares for the 12 individual experience 

variables vs. criterion measures given in Table 

6 are in dose agreement with the correlations 

of Tables 4 and 5. In every instance where the 

observed and expected values differed, tetrachoric correlation coefficients would have the 

same algebraic sign as the <:orresponding correlations in Tables 4 and 5. 

Table 6 does not contain data for the PassFail criterion in the Terminal Sample. An examination of that sample revealed that in only 

two instances did more than 4 of the 19 failures 

indicate that they had had a particular type of 

-7-
TABLE 6 

Frequencies of Trainees Reporting Various Types of Experience and Chi Square Tests 

of the Experience-No Experience Dichotomy vs. Pass-Fail and Above-Below the 

Approximate Median of Academic plus i.aboratory Grade Average 

EomJJte. :E;nrQute: A ± L Grade Termittal: A ± L Grade 

Variables Fail Pass Total :::::~ ,;s3 >84 Total x• <85 >86 Tota! X2 

I. Pilot 45 80 ( 92) # 125 8.57* 80 45 (58) 125 7.!! 25 21 (25} 4Q NS• 

2. Radio: Air/Grnd 23 27 f 37) 50 11.47 40 II t24J 51 13.97 13 4 ( 9) 17 7.00 

3. CCI 14 22 ' 27) 36 NS 27 11 08) 38 4.9R 9 3 ( 7) !2 4.39 

4. Statjnn 17 23 • 30) 40 6.10 25 15 {!9) 40 NS 5 1 ( 3) 6 NS 

5. Grnd/ Air 39 71 f 81) llO 6.47 71 39 (51) 110 6.67 22 ll (18) 33 6.81 

6. P to P 21 22 f 32) 43 12.67 34 !0 (20) 44 10.78 9 3 { 7) 12 4.39 

7. VFR Tower 33 :66 {!47) 199 15.51 &8 Ill t92) 199 11.98 7l 92 (89) 163 NS 

8. App Cm1 Tower 11 95 ( 78) 106 17.21 41 66 (50) 107 12.97 30 6! {49) 91 ~.47 

9. Rad App Con Tow 2 16 ' 13) 18 NS 9 9 ( 8) 18 NS 4 7 ( lj) 11 NS 

10. Center 9 50 f 44) 59 4.06 20 39 <27) 59 10.56 8 9 ( 9) 17 NS 

II. GCA 17 130 (!09) 147 22.79 61 87 <69) 148 13.20 33 39 (39) 72 NS 

12. RAPCON !0 86 ( 71) 96 15.12 29 68 (45) 97 27.45 10 21 117) 31 NS 

•• TOTAL N 129 366 495 268 231 499 IJO 131 241 

# Number"' in parenthe-"e"' are expected valu!!s. for Pass or Above Median trainees computed from 

a 2 x 2 contingency table. If the expected va:ue is less than the observed value, the sign of the 

relationship is positive; if greater, negativ~. 

'* With I degree of freedom, a X2 value of 3.84 is significant at the .05 level, 6.64 at the .01 

level, and 10.83 at the .001 level. NS replaces X2 values not reaching the .05 significance 

leveL Yates correction wa!'< not app!ied. 

•• Total N is the maximum p[)ssihle frequency which could occur as an entry in each column. 

From the Total N and the table entries. the 2 x 2 contingency tables can be reconstructed. 

experience. Consequently, chi squares were 

computed for just these two variables - Pilot 

Experience and VFR Tower Experience. Neither 

was significant. Because of the small number of 

failures in the Terminal Sample and the extreme skewness o~ the ini:lividual experience 

variables, the relationships with the Pass-Fail 

criterion must he interpreted cautiously. 

Age vs. Criteria - Detailed studies of the 

implications of training entry age for training 

and job performance have been reported elsewhere! " However, it should he noted in Tables 

4 and 5 that the age-criteria correlations were 

all negative and that the largest ccefficients o<:-

curred with the Laboratory Grade Average. 

This suggests that the older trainees had the 

most difficulty in acquiring performance skills 

as contrasted with academic knowledge. Since 

it is performance skill which is most critical in 

the control of air traffic, the finding is of some 

import. 

Aptitude Tests and Education vs. 

Criteria.-Tables 7 and 8 contain the correlations of the aptitude tests and Education with 

the training and job performance measures. 

None of the Education vs. criteria correlations 

were statistically significant. 

Out of 30 correlations in each sample be· 

tween the tests and criteria, 27 were significant 

for the Enroute and 19 were significant for the 

Terminal. All significant correlations were 

positive in sign. With respect to the magnitude 

of the correlations in the Enroute Sample, 6 

exceeded .4, 14 exceeded .3, and 22 exceeded 

.2. In the Terminal Sample, 3 exceeded .4, 9 

exceeded .3, and 15 exceeded .2. A comparison 

of these magnitudes and levels of significance 

with those for the experience variablei leaves 

no doubt as, to the superiority of the individual 

aptitude tests and the composite test score for 

the prediction of the Academic Grade Average, 

the Combined Academic plus Laboratory Grade 

Average, and Pass or Fail in both Enroute and 

Terminal Samples and for the prediction of 

the Laboratory Grade Average in the Enroute 

Sample. 

For the prediction of the Laboratory Grade 

Average in the Terminal Sample, experience 

was superior to the aptitude tests. This is most 

apparent in the correlation between Lab and 

~AT. 

In the area of job performance, the aptitude 

tests were not very effective predictors of the 

Average Supervisory Rating. Although two of 

the indi·1idual tests and the composite test score 

were significantly and positively related to the 

Average Supervisory ltating in the Enroute 

Sample, none were significantly related in the 

Terminal Sample. 

-8-
TABLE 1 

Er.route Sample; First-Ordel" Correlations and Partial Correlations (Age Held Constant 

Statistically) of Test Variables and Education with the Criterion Measures 

Variables r 

A±L 

Partial r r 

p 

Partial 

Fl 

r r 

A cad 

Partial r r 

Lab 

Partial r r 

Su~r 

Partial r ~ r 

17. Educ -03 05 -08 00 03 07 -06 03 -08 -01 28** 

18. Space 34"* 31** 30*t' 26** 37** 35*" 28** 24** 04 00 -18** 

19. NumerEcal 36** 39** 26** 30** 43** 45** 27** 30** 09 11 08 

20. Abstract 45** 43** 36** 33** 44** 43** 39*"' 36** 12* 09 -17** 

21. Analogies 28** 25** 18** 15** 30** 28** 23** 20** 13 .. 11* -14** 

22. ATP 37** 34*¢ 28** 25** 32** 30** 3sc.• 31** 15* 12* -17** 

23. Composite 52** 49** 39** 37** 53** 52** 44** 41 .. '17** 14* -17*• 

Max. 487 483 488 486 310 489 

Range of N 

Min. 468 464 470 467 301 470 

1 Point-biserial correlations; aU other correlations are producl-moment. Decimal points omitted 

from all correlations. 

• Significant at less than the .05 level 

• • Significant at less than the .01 level. 

TABLE 8 

Tenninal Sample: First-Order Correlations and Partial Correlations (Age Held Constant 

Statistically) of '!est Variables and EUucation with the Criterion Measures 

A±L f 

Variables r Partial r r Partial 

[1 

r r 

A cad 

Partial r r 

Lab 

Partial f. r 

Su~r 

Partial r ~ r 

]7. Educ 00 04 -03 Ol 05 06 -08 -03 OJ 00 16. 

18. Space 14· 12 2o•• 17* 26·· 25•• 06 02 01 -OJ -II 

19. Numerical 33"'"' 34"'"' 19*"' 20"'"' 45*"' 45"'"' ]4• 15* 10 lO Ol 

20. Abstract 40** 37"'"' 38** 35** 45*"' 44** 3o•• 24"'"' -03 -06 -2s•e 

21. Analogies 12 lO 09 08 22** 22"""' 02 -01 OJ 00 -08 

22. ATP 37** 34** 33** 29** 3a•• 37•• 31"'* -~ ?-•· 07 05 -22"'* 

23. Composite 38"'"' 35*"' 33"'"' 3Q*It: 49** 48** 23** 18** 05 03 -18*" 

Max. 240 239 240 236 203 240 

Range of N 

Min. 2ll 210 2ll 207 180 212 

1 Point-biserial correlations; all other correlations are product-moment. Decimal points omitted 

from all correlations. 

• Significant at less than the .05 level . 

• • Significant at less than the .01 level. 

DISCUSSION 

Previously reported research and the present 

investigation have confirmed the value of aptitude tests and knowledge of chronological age 

for the prediction of training performance of 

Enroute and Terminal types of Air Traffic Control Specialists. In contrast to this, pre-employment, job-related experience, which has been 

used as a selection standard, must be fractionated into particular kinds of experience to obtain a clear evaluation of its worth as a predictor of training performance. 

More specifically, communications, GCI, and 

Pilot experience had little value as predictors 

of training performance. Indeed, the more communications experience a trainee had, the less 

well he tended to perform. On the positive side, 

a trainee with experience more directly related 

to air traffic control work, such as Tower, Center, GCA, or RAPCON experience, tended to 

perform better in training. 

Significant and positive predictions of the 

job performance measure by both aptitude tests 

and experience variables were obtained only in 

the Enroute Sample, witi1 the aptitude tests being superior; but neither the tests nor experience can be considered highly effective predict· 

ors. The reasons for the lack of predictability 

of the Average Supervisory Ratings, particularly in the Terminal Sample, are not clear, but 

s~veral possibilities exist. 

First, with respect to the aptitude tests, it 

should be recalled that the present test battery 

was selected on the basis of prediction of Enroute training performance and was validated 

independently on additional samples of Enroute 

and Terminal trainees. Even though the tests 

do an effective job of predicting Tenninal training performance, it is likely that more discriminating tests can be found. The lack of a significant relationship in the Tenninal Sample between the Laboratory Grade Average and the 

-9-
Space and Analogies tests ·lends some support 

to this likelihood. If tests were found which improved the prediction of Terminal training performance, it may be inferred that predictable 

elements of training performance exist which 

are different for Enroute and Terminal trainees. 

. The different, but predictable, elements might 

also exist to some degree in the job situation. 

If so, the prediction of job performance measures by a test battery developed through analysis of Terminal training performance measures 

would be improved. Research is currently underway to investigate this possibility. 

Another factor is related to the preceding. 

Enroute trainees are assigned to less than 

thirty Air Route Traffic Control Centers; whereas Terminal trainees can be assigned to one of 

several hundred Terminals. The diversity of activity between Terminals is greater than between 

Centers. This being so, it may be that for Enroute trainees there is more similarity ·between 

what is done on the job and what is taught in 

training. A greater overlap between behavioral · 

elements required in the training course and on 

the job could account for the superior prediction of Enroute job performance by tests 

selected so as to maximize the prediction of 

training performance. 

A third factor, having relevance to this problem, concerns the ratings by job supervisors. If 

indeed, there is more similarity between activities in Centers than in Terminals, then the job 

supervisors in different Centers would be making their evaluations on a more nearly com· 

parable basis than would their counterparts in 

Terminals. The greater similarity of the behavior underly;::Ig ratings by Enroute supervisors in different Centers should operate to 

increase inter-center rater reliability; i.e., comparable behavior would be rated in the same 

way in ·different Centers. Wit."t an increase in 

the reliability of the ratings, there would be a 

corresponding increase in the probability of 

finding significant reiationships betwten the 

ratings and the tests and experience variables. 

Finally, in studies similar to this one, it is 

usually found that training performance 

measures can be predicted more accurately than 

job performance measures. This is partially accounted for by. the greater behavioral homogeneity of trainees who complete the trammg 

course. Since the course itself acts as a screening device, the poorer students, and it is assumed the poorer job prospects, are eliminated. 

With the elimination of the more inept trainees, 

the range of behavior evaluated by job supervisors becomes restricted. This, in tum, tends 

to lower correlations· between the predictors and 

job performance measures. 

Overall, the findings of th.e present studv 

support the use of aptitude tests in preference 

to undifferentiated, pre-employment, job-relat· 

ed experience for the prediction of training and 

job performance of E!'route and the training 

performance of Terminal Air Traffic Control 

Specialists. If only one selection method could 

he used, aptitude tests should he the method of 

choice. 

However, since there were positive relationships between some specific types of experience 

and training and job performance, additional 

weight in the selection process should he given 

for such experience. It is recommended that for 

selection purposes, Tower, Center, GCA, and 

RAPCON experience be considered as positive 

indicators for ATCS training in the Terminal 

and Enroute specialties, and that GCI, Pilot, 

Station, and Communications experience should 

be given no weight in the selection procedure. 

Selection by a combination of aptitude tests 

and experience requirements would have an 

additional advantage. Individuals having experience of the kinds found to be useful as predictors, are somewhat familiar with the work 

actually performed as an Air Traffic Control 

Specialist. Applications for ATCS trai~ing from 

such individuals can be assumed to indicate a 

continuing interest in the occupation. These 

individuals also would bring to the training situation an aviation-related vocabulary which 

would reduce the amount of training time spent 

on terminology familiarization. 

It is well to remember, however, that the definitions of various types of experience are far 

from standardized and that their relationships 

to specific jobs may change markedly with 

changes in the job elements underlying the experience. In contrast, aptitude measures are not 

affected by this limitation and should remain 

better predictors for longer periods of time. 

-10-
The preceding recocr:mendations with respect 

to experience variables apply only to the Enroute and Terminal s:;tcialties. Studies currently underway on flight Service Station (FSS) 

trainees, a thirc type of Air Traffic Control 

Specialist, rna y ?eveal a quite different picture 

of the predictive value of the various types of 

experience. In a Flight Service Station, com· 

munications activity forms a l& c'ge share of the 

work. Consequently, it would Hot be surprising 

to find that for FSS trainee~ communications 

experience was a positive predictor of training 

and job performance. 

With continued research a number of the 

preceding questions may be answered; and it 

may be pos~ible to achieve even better prediction of ATCS training and job performance by 

identifying additional, important aptitude areas, 

improving the measurement of the present areas, 

or by developing differential predictor weights 

for the different ATC spe<::ialties. 

SUMMARY 

A study of over 700 Enroute and Terminal 

Air Traffic Control Specialist trainees revealed 

that different kinds of pre-employment, jobrelated experience had differential value for the 

prediction of training performance. In general, 

experience most directly related to air traffic 

control work was a positive predictor; experience related to communications and piloting 

was negative. It was also shown that for Enroute 

trainees only, a composite variable (IAT) 

representing the sum of tower, GCA, RAPCON/ 

RATCC, and center experience had a statistic· 

ally significant, but small, relationship with 

ratings of job performance. In contrast, aptitude tests were superior to the experience variables for the prediction of all training course 

performance measures of both types of trainees, 

with the exception of Laboratory performance 

of Terminal trainees. In the latter instance, l: 

AT was superior to the tests. For job performance, it was found that only ratings of 

Enroute trainees could be predicted by the 

tests and that although the tests were superior 

to experience as predictors, the relationships 

were small. As determined in other studies, it 

was again shown that age at entry into training 

was negatively related to training and job performance and that the negative relationships 

were greatest with grades in simulated air 

traffic control work in the training school laboratories. The findings of the study led to the 

recommendation that selection of individuals 

for Air Traffic Control Specialist training be 

based upon an aptitude test battery and pre· 

employment, job-related experience in tower, 

GCA, RAPCON/RATCC, and center work. 

REFERENCES 

1. Brokaw, L. fi.: School and job validation of selection 

measures for air traffic control training. W ADC-TN-59-39, 

6570th Personnel Research Laboratory, AMD. AFSC, Lackland AFB, Texas, 1959. 

2. Cobb. B. B.: Problems in air traffic management: H. Prediction of success in air traffic controller school. Aerospace .lied., 33:i02, 1962. 

3. Trites, D. K.: Prob!ems in air traffic management: I. Longi· 

tudinal prediction of effectiveness of air traffic controllers. 

Aerospace Med., 32:1112, 1961. 

4. Trites, D. K. and Cobb, B. B.: Problems in ai~ traffic 

management: III. Implications o! age for training and job 

performance of air traffic controllers. Rept. No. 62-3, 

Civil Aeromed. Res. Institute, FAA, Oklahoma City, Okla., 

1962. 

5. Trites, D. K. and Cobb, B. B.: Problems in air traffic 

management: Ill. Implications of training-entry age for 

training and job performance of air traffic control specialists. Aerospace Med. (in press.) 

• Opinions or conclusions contained in this report are those 

of the authors. They are not to be construed as necessarily 

reflecting the view or the endorsernent of the Federal Aviation 

Agency. The authors gratefully acknowledge the assistance 

of ~Mr. R. S. Beebe, Mr. Joseph Federico, Mrs. La Von Graham. and Miss Carolyne Mantooth during the many tedious 

and technical aspects of this research. Th~nks are also due 

to Drs. George T. Hauty and William F. O'Connor and Mr. 

William J. Berkley for !heir helpful suggestions during the 

preparation of the manuscript. 

--11- U.S. GOVERNMZNT PRINTING OFFICE. 1!Hi4 0-732-431