positive conflict items did not discriminate and
generally, for true conflicts, the greater the lead
time available. the greater an item’s discriminat-
ing power. Journeyman controllers identified
potential conflicts sooner than developmental
trainees, and identified almost. all potentials that
became real conflicts. . Developmental controllers
generally missed a number of real conflicts. were
slower than journeymen in calling them out. and
frequently did not attend to aircraft altitude
separations. However. the range in test perform-
ance among developmentals was great. and some
performed as well as journeymen. Some con-
fliction items in these films were too easy and
some had a negative relation to a developmental-
journeyman criterion dichotomy.

The observer had much idle time during free
response film versions of the test. as the traffic
presented is light and aircraft widely separated.
In this sense. the setting resembles.a common air
traffic situation, but an extended amount of test-
ing time is required to present only a small
number of confliction items. Better use of ex-
aminecs’ time was desired. An initial change
during test development was to structure the
items. using slides to present questions on the
screen above the film. Mean response times for
developmental and journeyman controllers to
react to each conflict in the free response ver-
sions were determined. Journeyman mean times
were selected to be determiners of aircraft posi-
tions when presenting confliction questions in
the structured versions. for this maximized the
discriminating power of each item, Two-choice
conflict items. asking whether a pair of aircraft
would conflict. were presented for thirty seconds
when unexpected changes occurred among tar-
gets. such as when new aircraft entered the pic-
ture. Four-choice conflict items, asking which
pair would conflict if a confliction occurs. were
presented for forty-five seconds when traffic
changes were slow. A “None of these” respons
was introduced to permit inclusion of non-con-
flictions that might he predicted to be conflic-
tions. and this was used as the forth alternative
in ail four-choice confliction items. Ttems were
assembled into the described test format. but they
didn’t utilize half the available testing time. This
large amount of idle time between questions also
provided much opportunity for observers to
change their anwers to earlier questions regard-

ing potential conflicts as the aircraft approached

each other and the correct answers became more
obvious. In the free response film version. this
problem was controlled by a requirement for
entries from a coded clock onto the computer-
scored answer sheet whenever potentia! conflicts
were reported. It was found with the paper and
pencil version that presentation of a new item
every 45 seconds would keep examinees so. busy
that they would have no opportunity to make
such changes. More test items were needed to
make this control procedure work.

The test films presented simulated traffic mov-
ing across a radar scope plus a table which in-
cludes detail data on each aireraft. identifying
the target. its altitude, speed. and route on the
sc. pe. The scope uses lines to represent airways
or highways in the sky. with alphabetical iden-
tifiers of starting, ending, and intersecting points
on the airways. The top of the scope is North.
and a mileage scale is provided at the bottom.
Ample information is presented to prepare a
variety of questions related to a controller ac-
tivity. It appeared possible that most factors
used in the current Civil Service Commission
aptitude battery for controller selection could
be measured within this simulated air traffic
setting.

Ttems were written utilizing this available
information to measure such aptitudes as direc-
tion following. table reaGing. interpretation of
data. spatial visualization and orientation, esti-
mation of distances and relative target move-
ments. and arithmetic. Some items included
were of a very simple nature. and others were
written in a multi-factor format to increase their
level of difficulty. For example. the directions
for this type of test required instruction on how
to read the table. so initial table reading ques-
tions were included that were of the very easy.
instractional type. Awareness of distances across
the scope. reading the table to determine the

speeds of aircraft. and mathematical computa-
tion to determine their rates of closure (all of
these heing rejated to horizontal separation) were
required to solve a complex problem such as
estimating the travel time in minutes between
two aircraft at a given moment.

The reiation between item typex and total test
homogeneity was determined. and this ratio was
used to determine the number of items per type
to include in the test. A result. for example.


Table 4 eo

Correlations Between Selection Tests and Var‘ous Criteria

Selection

- Tests
Number Arithmetic MAT Directional Dial Selécted
Criteria of Cases Reasoning Conflicts Aptiwudes Headings Reading Battery
Training Scorés
Terminal Option 301-310 136% 321
Enroute.Gption 207) Perey
“Both . 202 3708"
A ing?
Supe=visory Rating
Flight Service Station 195-245 204%
Terminal VFR 169-190 22188
188-188
181-241 o2e
733-833 123%"
ATC Progression®
Flight Service Station 157-159
Terminal VFR 191-193 075 232%
IFR 148 120
Enroute 107 170 280%
All Options 058 07 1192
Aggregate Criterion®
Flight Service Station .
Terminal: VFR 17¢"
IFR idee
Enroute
All Options 1309-1603
Decimals Omitted *p<.05 *p¢.01

a Table XI. 1
& Table Xu.2

¢ Table Xilf.2
d Table XV. 2

ssuccess in Training. .\. battery of precicto
including Multiplex Controller Aptitude Test,
606A and TO6AS, scored separately for Conflicts
and Aptitudes. plus Directional Headings, Dial
Reading and Arithmetic Reasoning tests becau-e
of the promise they demonstrated in an earlier
study (Milne, Note 2). were administered te the
January 1976 Class of students entering the ATC
Academy centralized training course. Persistent
mechanical problems in operation of film, slide
and slide programmer equipment gave strong
support to the Civil Service Commission's stated
need for development of these measures in 2
paper and pencil format. Thus the March 1976
class of students were administered only paper
and pencil versions MCAT 7064 and 606B. plus

the other selection measures,

Satisfactory com-
pletion of the initial. formal training program
for those enrolled in the Enronte sption was
hased on scores received on four ATC laboratory
problems plus scores on a Controller Skills Test.
For those enrolled in the Terminal course. per-
formance was based only on the ATC laboratory

Data are from Mies, J.M., Colmen, J,G, and Domenech, O.,

“tion research effort (Miex. Note 3).

May 1677.

problems. These scores require students to dem-
onstrate operational application. of academi
knowledge. Correlations between “selection tests
and training scores are presented im Table 4.
The MCAT Conflict and Autitude: scores con-
stantly produced higher validities than any other
test used. In fact. addition of the other tests
increased the multiple correlation by only '.017
for Terminal. .012 for Enroute. and .008 for both
options combined.

Performance on The Job. Test validation in
the operational situation. test correlation with
snpervisory ratings and career progression ‘scores.
was accomplished as part of a larger ATC selec-
A-compre-
hensive sampling design was constructed to
define the ATC population. constraints were
established. and stratified random sampling
methods then applied to select the -—primary
samples of specialists from within the total avail-
able ATC population. On-the-job performance
was measured by confidential job-task a:
ments prepared by each employee's supervisor.



Mechanical Comprehension, .63, and Arithmetic
Reasoning, .53.. This factor might be defined as
Mechanical-Spatial, since the spatial and me-
chanical comprehension selection tests have
highest loadings on this factor. This factor

seems to account for very little criterion variance.

Rotated Factor HI. Variables with highest
loadings on this facter include Course Average.
.82, Basic Lab. Control Tower Training, .65. and
Stopover Composite Flight Plan Lab, This
factor might be defined as Laboratory Perform-
ance, since Jaboratory activities have their highest
loadings on this factor. Among potential screen-
ing measures, ATC Occupational Knowleilge
Test I and the Multiplex Controller Aptitude
Test, with loadings of .22 and .21 respectively.
have the highest loadings on this factor.

Rotated Factor IV. Variables with highest
loadings on this factor include Multiplex Con-
troller Aptitude Test, 58, Free Response Code.
49, ATC Occupational Knowledge Test I. 46.
Navy General Classification Test. 40, ACT Oceu-
pational Knowledge Test II, .86. Charts & Pub-
lications, Base Ops., .36. and Precision Approach
Radar Lab., 30. This factor might be defined
as Air Traffic Controller Performance. since
selection tests involving the controller activity

and classroom and laboratory performance vari-
ables have significant loadings on this factor.

Rotated Factor V. Variables with highest
loadings on this factor include Reading. .70,
Radioman, .58, and Precision Approach_Radar
Lab., 52. This factor might be defined as Read-
ing, since that variable has the highest loadings
on this factor.

Selected Intercorrelations

Selected data are presented in Table 1 to high-
light three of the experimental selection tests
being studied here, their interrelations with each
other and with laboratory Flight Plans, Control
Tower, and Radar Training, as those grades pro-
vide the greatest range in criterion scores for
differentiating performance among FAA student
controllers. ATC Occupational Knowledge Test
I, a measure of each person's past controller ex-
perience. showed highest relation to performance
in laboratory classes. The thirty minute Multi-
plex Controller Aptitude Test generally had cor-
relations with criterion variables that are as-high
or higher than those between the Free Response
CODE test. and those same criterion variables.
These results encouraged further development of
the Multiplex Test.

Table 2

Selection Test Variables and School Grades

Selected Intercorrelation Data

USAF Air Traffic Control! Training School

(461 Persons Tested October 20-24, 1975)

Test and Course Variables 1 2

4 5. 6 7 8 9 lo. 11 12
2. Final Score in Block- i 34 41 39 -32 08 35 15 20 19 42 -o1
2. rat 49° 31 --33«O7)s 21S 2130159
3. HI 45  -36 28 33 16 17 27 32 39
4 Iv -51 28 06 18 7 O1 48 Ol
5, No. Counseling Sessions “12-25 02 -05 -04 11 13
6. CODE 7- (Free Response) x 2 xo *
7. MCAT 606AS % 61 59 x *
8. 600FS 66 O28
9%. 706FS 57 41 20
10. 406FS = :
11. Occupational Knowledge Test- I
12. u

*No Cases


was inclusion of twice as many target time-dis-
tance separation items as compass heading items
in the test. The order of placement of conflict
items had already been established by using the
mean response time of journeyman controllers
when targets were at certain location. New apti-
tude test items were placed in the remaining
positions, alternating from one type to another
and spiraling to increasing levels of difficulty
as testing progressed. Three alternate forms of
the test, MCAT 4, 6, and 7, were prepared.

Test Criterion Characteristics

Versions of the fre: response CODE test, the
structured Multiplex Controller Aptitude Test.
and other selection measures were adiministered
to students at the US Navy Air Traffic Controller
Training School, Memphis, during the week of
August 18-22, 1975. Grades of these students
were obtained as they -progressed through class-
room and laboratory training and passed or

Table 1

failed the course. Distribution statistics, inter-
correlations and rotated factor loadings of the
various tests with each other and with school
grades were determined. Forty-eight variables
were involved; five factors were extracted and
rotated. Definitions of the rotated factors are
presented below.

Rotated Factor I. Variables with highest
loadings on this factor include CTO Block I
Average, .76, ITEX Final Criterion Exam, .54
and Control Tower Training, Airport. 53, This
factor might be defined as Control Tower Op-
erator. the title given by the school to those
classroom grades with highest loadings or portion
of variance on this factor. Among potential
screening measures, a new FAA air traffic con-
troller ATC Occupational Knowledge Test I has
the highest Joading, 40.

Rotated Factor IL Variables with highes
loadings on this factor include Spatial Test. .67

Intercorrelations of Selected Test Variables and Laboratory Grades

USN Air Traffic Control Training School

{109 Persons Tested

August 18-22, 1975)

Arithmetic

Laboratory Courses Code MCAT OKT-I SCT Reasoning
Laboratory Flight Plans

VFR 05 17 23 10 os

IFR 18 26 22 19 26

Stopover Composite 09 20 28 26 09

Performance Run 19 22 22 25 18
Laboratory Control Tower
Training

Basic 16 20 28 19 21

Intermediate 19 ql 07 17 09

Advanced 26 7 27 19 08
Laboratory Radar Training
Air Surveillance

Week-l ~02 10 i 17 10

Week-2 -08 04 2i id 12
Precision Approach

Week-3 08 21 13 14 08

Week-4 16 04 09 ll 25
Course Average 22 27 43 34 23
Free Response Code 43 32 12 14
MCAT 42 20 23
ATC Occupational Knowledge 29 24
USN GCT 32


A next step in test development was to print
the slide versions in paper and pencil format.
The slide version and combination film-slide ver-
sion permitted time control at the individual item
level, allowing thirty seconds for response to two-
choice items and foriy-five seconds for response
to fowr-choice items. One paper and pencil ver-
sion allows uninterrupted work throughout test-
ing time. .with announcements when fifteen
minutes have elapsed and when only five minutes
vemain to work on the test. segmented paper
and pencil version allows five niinutes for re-
sponse to each cluster of items presented. with
2x cluster containing as many as eleven items when
they are all of the two-choice type.
parallel forms of the written test were prepared.

MCAT 406 including 43 .items. 606 containing
43 items and 706 containing 53 items. Time

limits are 25 minutes plus directions for MCAT.
406 and 606 and 30 minutes plus directions for
MCAT 706. Each form now has been extended
to 55 items. MCAT 407 incindes 23 conflict and
- 32 aptitude items. MCAT 607 contains 22 conflict.
and 33 aptitude items. and MCAT 707 contains
23 conflict and 32 aptitude items. Time limits
are 35 minutes for each of these tests.

Reliability

The questions presented in Multiplex Con-
troller Aptitude Test. Form 4. 6 and 7. remained
unchanged for some length of time. though the
media used for presenting items changed from
film to slide to paper_and pencil. This change
in media for presenting items has some effect on
an individual's test performance. but all forms
are qualitatively siinilar. Correlations of these
parallel forms provide a minimum estimate of
the measure’s test-retest reliability when adimin-
istered to members of a similar population.

These forms have been a:lministered to stu-
dents of the USAF Air Traffic Controller Tech-
nical School in all phases of training and to a
sample of forty non-controller high school and
college level F*\ employees. groups whose
abilities approximate those of the general popu-
lation of applicants that take the Civil Service
Commission tests. They also have been admin-
istered to entering FAA Air Traffic Controller
Academy students in the new centralized Air
Traffic Controller training program.
selected off the top of the Civil Service Register

persons

Three

for controller applicants. Table 3 presents dis-
tribution statistics for these groups on the ¥ari-
cus forms of the test. The range in test scores
is highly restricted for the ATC Academy group
when compared to the varge for the more general
population.

Correlations between forms were lowest, .31 to
50. for the ATC Academy students who had a
greater restriction in range of their scores. They
range from .60 to .66 for the USAF ATC stu-
dents and .8t to .90 for non-controller FAA
employees whose abilities more nearly approxi-
mate that population for whom the test is de-
signed. These coefficients indicate that the test
will provide reliable measures of performance
under well controled conditions. but a combin-
ing of items from two forms would ‘provide a
more satisfactory measure to use in an opera-
tional testing situation.

Test Validity

For test validation purposes the criterion used
5 Fsuece: in air traffic control work.

cess as defined here for ATC applicants is
hierarchical. including (1) satisfactory comple-
tion of the initial. formal training program.
(2) satisfactory performance on the jok. and
(3) progression or upward mobility within the
ATC system. Another element. (4) attrition.
may be a measure of non-success, Those enrolled
in the initial. formal training program are a
highly select group. for they scored high enough
on the Civil Service Commission tests to be hired.
but group membership becomes even more selec-
tive as those who fail to Jearn or perform ade-
This selection process

Suc-

quately are separated.
continues as they progress up the ATCS career
ladder and are evaluated for xatisfactory per-
formance on the job and advancement into the
more demanding and higher paid positions with-
in the ATC system. The range in scores on
these =clection tests will be greatest for the group
when they first enter formal training. but become
ever more restrictive with career progression.
As this restriction in range may have a, direct
effect. on the size of the validity coefficients. the
validities should be highest when the group is
in initial training and become Jower for that
portion of the group which progresses up the
carrer ladder.


JUN 13 E78
FAA-AM-77-25

ORIGIN.

DEVELOPMENT OF NEW SELECTION
TESTS FOR AIR TRAFFIC CONTROLLERS

John T. Dailey, Ph.D. uy
Evan W. Pickrel, Ph.D.
Federal Aviztion Administratien sa
Office of Aviation Medicine SR

$00 Independence Avenue, S.W.
Washington, D.C. 20591

December 1977

Document is
National Teehn
Springtield. Virginia

Prepared for
U.S. DEPARTMENT OF TRANSPORTATION
FEDERA:. AVIATION ADMINISTRATION
Office of Aviation Medicine
Washington, D.C. 20591



Service screening battery. In addition. it in-
cludes a measure of the ability to identify
potential conflicts in air traffic. a skill that has
been demonstrated experimentally to have sig-
nificant relation to success in the FAA ATC
speciality. All test questions are preesnted in
an air traffic control setting. which gives them
a job-related appearance not found in today’s
selection battery. Alternate forms of the test
have been developed in a paper-and-pencil for-
mat to meet Civil Service needs in their decen-
tralized testing program. The test has heen
administered’ experimentally to groups whose
abilities approximate those of the applicant pop-
ulation, and correlations between alternate forms
indicate that it has sfactory reliability char-
acteristics. It has been administered experi-
mentally to students entering the new FAA ATC
Academy and personnel on the job at operational
facilities, and constantly produced higher corre-

Pa

lations with ATC success than any other test
used in the validation studies, The available
data indicate that this new and customized in-
strument. when used in combination with other
selected measures, promises to be a significant
improvement over the existing battery for screen-
ing FAA Air Traffie Controller applicants.

Reference Notes

1. Buckley. E. P. & Beebe, Te The development of a
motion picture measure vent for aptitude for air
trafic control. FAA-RD-71-106. Atlantic City, New
Jersey: National Aviation Facilities Exp
Center, January 1

Milne, AL M. & Cobnen. J. G. Selection of air traffic
controle ’s for Federal Aviation Administration, Final
Report, Contract DOT-FATOWA-2371, Japnary 19

3. Mie:

M., Colmen, J. G. & Domenech, O. Predicting
success of applicants for positions ax uir trafic cou-
trol specialists in the Air Trafic Service. t
port, Contract DOT-FATZWA-3646, May 1977.



Table 5

Integer Weights Assigned to Tests

By ATC Option and All Options Combined

. MCAT , MCAT Directional Dial .
ATC Options Conflicts Aptitudes Headings Reading
FSs ° 18 ~ 40 (0) 7
VFR 55 . 10 22 (0)
IFR 49 : 8 13 (0) >
ARTCC 33 34 7 5
All Options 37; 21- 15 (0)

Note, Data are from Mies, J:M., Colmen, J,G, and Domenech, O.,

May 1977, Table XV. 3°

All data were collected at ATC facilities.
relations between selection tests and ‘Supervisory
ratings are presented in Table 4. Correlation

- between MCAT Conflict. and Aptitiide scores and
supervisory ratings are significant (p. <.01) for
the FSS and Ferminal VFR options and for all
options combined. The correlation between
MCAT Conflict scores and Terminal T¥R super-
visory ratings also was significant (p. <.01).
The MCAT Conflict and Aptitude variables were
prominent predictors in most regression equa-
tiors which predicted job performance “success,”
although neither predicted at significant levels of
confidence for the Enroute option. Directional
Headings appeared valid only for the VFR
option and for all options combined. Dial Read-
ing entered for FSS only.

ATC Progression. ATC Progression was
measured by comparison of the ATC option to
which the specialist was initially assigned when
hired to the option assigned on January 1, 1976.
A progression from FSS to Terminal VFR and
IFR and Enroute options was used to represent
levels of increasing complexity. -\ progression

of “high” was assigned within this hierarchy
when a specialist was in an option of a com-
plexity level the same as or higher than the
initial option’ assigned. A progression value of
“low” was assigned when a specialist was in an
option of a lower complexity level than the
option to which imitially assigned. Correlations

between selection tests and progression scores are

Cor.

presented in Table 4. Again the MCAT Con-
flicts and Aptitudes were prominent predictors.

Aggregate Criterion. An aggregate criterion
of ATC “success” was constructed from combina-
tions of the four individual criteria (training.
on-the-job performance, progression, and attri-
tion) and provided a five point scale value for
ATC success. The Arithmetic Reasoning Test
was valid only against the progression criterion.
and was dropped from the battery. The Direc-
tional Headings Test was dropped for FSS, since
it failed to enter the regression for supervisory
assessment. Correlations between selection tests
and the aggregate criterion are presented in
Table 4. All correlations are significant . (p.
<.01) for each of the four options and: for all
options. combined. The Dia] Reading Test did
not enter the regression for the VFR option and
entered last with a negative “b” weight for IFR
and for all options combined. The integer
weights assigned to tests are summarized in Table
5. It is evident that MCAT (Conflict and Apti-
tude segments) is the major factor in the-validi-
ties (R) derived from the multiple regression
analysis.

Summary

Anew test has been developed for initial
screening of FAA ir Traffic Controller appli-
cants. Its content includes the traditional types
of aptitude test items found in today’s. Civil


Media For Presenting Items

A next step in test development was to con-
vert. the slides plus film presenting moving tar-
gets to a “slide only” presentation, in ess
capturing pictures of the scope with targets in
the same position as when each question appeared
on the screen. Pacing of slice presentation was
the same as in the film plus slide version. so the
amount. of target movement from question to
question was unchanged. This format was easier
for test- administrators since they no longer had
to cope with film projection problems. and the
stationary targets presented by slide seemed
easier to read than the moving targets presented
by film. Persons taking the experimental ver-
sion in this slide format did grumble that the
test. seemed to take control over their time. as
each item had to he completed quickly before it
left. the sereen and a new item appeared. This
task characteristic has some commonality with
controller situations in the real world. as traffic
movement. tends to control their involvement and
pace of work. -Another merit of this version
would be its case for transition into a paper and

Table 3

pencil format that would meet. Civil Service
Commission requirements for us in their very
decentralized field testing situations. Films plus
slide and “slide only” ‘versions of the Multiplex,
Controller Aptitude Test plus other selection
measures were administered to students at the
USAF Air Traffic Controller Technical School.
Keesler Air Force Base. Mississippi. during the
week of October 20-24. 1975. Grades of these
were obtained as they progressed
kfough classroom and laboratory training’ and
passed or failed the.course. Distribution statistics
and intercorrelations of the various tests with
school grades determined. Twenty-one
variables were included, and selected data are
presented in Table 2.. ATC Occupational Knowl-
edge Test I. a measure of the individual's past
controller experience. shows highest selection test
relation to fina] scores in training. The slide
version of Multiplex generally shows higher cor-
relations with the criteria. Block J-IV Final
Score. than CODE 7 (Free Response) and the
combination film-slide versions of MCAT.

stiidents

were

Multiplex Controller Aptitude Test

Distribution Statistics for Various Populations

FAA ATC Academy Classes
January 1976

Enroute Terminal Enroute

March 1976

USAF ATC School
Students

FAA Employees
High
School

Terminal College.

Tests or N=106 N87

N=100

Nel4l N=289  N=62_N=18

MCAT 606 (43 items)
FS Mean
sD
AS Mean
sD
A Mean 35,16 3
sD 3.90
B Mean
sD
MCAT 706 (53 Items)
FS Meas

vy
fon
hae
we
os
x
8

a
u
as
"
3

40.20
7.48

“MCAT 40» (41 Items)
FS Mean
sD

A Mean
sD

28.81
5.32 o. IS
27.35
+, 08

as
eed
is

ve
& Ue

39.15
7.41
36,23
8.49

17.23
4.32

v
2

*
eh

S43

30.50
4.08

13.89
4.90

FS: Film-slide version, with time controlled for working each individual item.

AS: Slide version, with time controlled for working each individual ivem.
‘A: Paper-and-pencil version, with time controlled ior working total

B: Paper-and-pencil version, with time controlled for working each five-minute group of items.

o3



DEVELOPMENT OF NEW SELECTION
TESTS FOR AIR TRAFFIC CONTROLLERS

This report describes creation of the Multiplex
Controller Aptitude Test. a new measure to
screen applicants for Federal Aviation Admin-
istration Air Traffie Controller positions. Its
content includes items to measure the iraditional
types of aptitudes such as arithmetic reasoning
and visualization that are included in today’s
Civil Service Commission tests. In addition it.
includes new job sample items from the con-
troller activity, uses figures to show air traffic
on a simulated radar screen. and requires iden-
tification of potential conflicts between aircraft
in that simulated traffic. All test questions are
presented in an air traffic control setting. provid-
ing a job-related appearance that gives the test
high face validity not found in today’s selection
battery. The fermat for item sequencing also is
a departure from another current aptitude bat-
tery design practice, that of clustering items
into homogeneous subgroups. Instead. in these
tests the items alternate from one type to another
and spiral to increasing levels of difficulty, a
mode of presentation found only in a few cur-
rent. tests such as the Standford-Binet. -\ result
shown statistically is that non-confliction items
ineluded for measurement of aptitudes repre-
sented in today’s selection battery show unex-
pectedly high commonality v 7th the new air
traffic items requiring detection of impending
conflictions. The resulting measure shows
promise of consistently producing higher validity
coefficients than possible with the current test
battery.

Background

The current. Civil Service Commission aptitude
test. hattery for screening air traftie controller
applicants has been operational since 1964. In
1970 the Federal Aviation Administration con-
tracted with specialists in the field of personnel
selection to search for ways to alleviate problems
being experienced in selection and retention of
air traffic controllers. They identified other avail-

able tests to increase the predictive validity of
this battery. plus areas for the construction_ of
new tests. One of these (Buckley, Note 1) made
use of a technique developed to evaluate total
man-machine system performance in contro] of
air traffic by measuring how well a fully trained
specialist could contro] simulated traftie presented
on a film using the proposed designs. In an
adaption of this technique for the testing of
personnel skills. films of air traffic as seen on-a
radar scope are presented. and observers asked
to identify the aircraft they predict will violate
separation standards. This instrument added
significantly to a composite for predicting on-
the-job success (Milne. Note 2). but the equip-
ment and space required for its administration
generally are not available in Civil Service Com-
mission testing situations. Therefore test de-
velopment. procedures were initiated to derive
measures of this same skill in a format that meets
Civil Service needs.

Test Development

The original Controller Decision Evaluation
technique (Buckley. Note,1) required observation
of simulated air traffic situations as they unfold
in a film of a radar scope. The observer pre-
diets and records potential conflicts as soon as
he detects them. in an unstructured. free response
mode. During experimental administration ‘and
study of three film versions for development of
the Multiplex Controller Aptitude Test. it was
found that identification of potential conflict=
was easier. quicker and more accurate after ob-
servers adjusted to the scope and its targets. when
only a few targets were in near-confliction, and
when their rate of closure slow. Items were
more difficult when the observers were first ex-
posed to the scove and its targets. when multiple
targets were in near confliction and when the
rate of closure amony targets was rapid. False



Technical Report Documentation Page

6. Performing Grganizonon Code

Controllers

T. Report No [2 Goverament Accession Now TS. Recipient's Cotalog No. 1

{ {
| FAA-AM-77-25 i i H
[f) Tete ond Subtrie | 3. Report Dove :
| Development of New Selection Tests for Air Traffic { December 1977

Periormng Orgonization Report No.

7. Author's:

_John T. Dailey, Ph.D., & Evan W. Pickrel, Ph.D. |

1G. Work Ginet No ‘TRAIS}

| Orfresree Avraeten real Cre

| Federal Aviation Administration
i 800 Independence Avenue, S. W.

TT Conzeoct or Gran? Neo.

{ Washington, D.C, 20591 12. Type of Report and Period Covered
112. Sponsonag Agency Nome ond Address

; Office of Aviation Medicine
i Federal Aviation Administration «

OAM Report

: 800 independence Avenue, S, W. 14. Sponsoring Agency Code
Loan i SS 20591

| 15. Supptemenrory Notes

16. Abssroct
This report describes the development of a new Multiplex Controller Aptitude

Test for initiai screening of FAA Air Traffic Controller applicants, Its

content includes the traditional cyses of aptitude test iterns used for today's
screening, In addition it includes measurement of the ability to identify
potential conflicts :0 air traffic, a skill that has been demonstrated experimentally
to have a significant relation to success in the ATC speciality, Alternate forms
of the test have been developed in a format that meets Civil Service test
administration requirements, The test has been administered experimentally

to groups whose abilities approximate those of the applicant population, and
results indicate that it has satisfactory reliability characteristics. It has been
administered experimentally to incoming students at the FAA ATC Academy

| and personnel on the job at operational facilities, and constantly produced higher
correlations with ATC success than any other test used in the validation studies.
i The available data indicate that this new and customized instrument promises

i to be 4 significant improvement over the existing battery for screening FAA Air
4

H

i

1

Traifiic Controller applicants,

: 17, Key Words j 18. Destaburvon Stotement

i Selection Document is available to the public

I Air Traffic Controllers through the National Technical Informa-

; } tion Service, Springfield, Virginia

: 22151

19. Security Class-f, lof ths report? DO. Security Classif, (ef thes page! T21- No. of Poges | 22 Tren.
Unclassified Unclassified

Form DOT F 1700.7 :2-77° Reproduction of completed page outhorized


Note

This document is disseminated under the sponsorship of the Department
of Transportation in the interest of information exchange. The United tates
Government assumes no liability for its contents or use thereof.