·~·~~"' 

1 

Doc FAA ' 

AM 

82 

1 1 

UNCLASSIFIED 

Technical 

Report 

distributed by - ----- •I 

~~\ Deftnse Technical Information Center 

· ~· r DEFENSE LOGISTICS AGENCY 

Cameron Station • Alexandria, Virginia 22314 

UNCLASSIFIED 
NOTICE 

We are pleased to supply this document in response to your request. 

The acquisition of technical reports, notes, memorandums, etc., is an 

active, ongoing program at the Defense Technical Information Center 

(DTIC) that depends, in part, on the efforts and interests of users and 

contributors. 

Therefore, if you know of the existem;e of any significant reports, etc., 

that are not in the DTIC collection, we would appreciate receiving copies 

or information related to their sources and availability. 

The appropriate regulations are Department of Defense Directive 5100.36, 

Defense Scientific and Technical Information Program; Department of 

Defense Directive 5200.20, Distribution Statements on Technical Documents; Military Standard (MIL-STD) 847-A, Format Requirements for 

Scientific and Technical Reports Prepared by or for the Department of 

Defense; Department of Defense Regulation 5200.1-R, 1nformation 

Security Program Regulation. 

Our Acquisition Section, DTIC-DDA-1, will assist in resolving any questions 

you may have. Telephone numbers of that office are: (202) 274-6847, 

274-6874 or Autovon 284-6847, 284-6874 

June 1982 

I 

~ 

I 

I 
·-----------------

JACK R. HUNT MEMORIAL LIBRARY 

DAYTONA BEACH, FLORIDA 904-226-6595 

MICROCOPY RlSOLUT!ON H.ST C~tA.RT 

I 

i 
>- Q.. 

8 

t'- . -·-' 

1:..:! 

SELECTION OF APPLICANTS FOR THE 

AIR TAAFFIC CQ'IROLLER OCO}Pf.Tlc:t' 

Donald B. Rock 

John T. Dailey 

Herbert Ozur* 

James 0. Boone 

Evan W. Pickrel 

Federal Aviation Administration 

Washington, D.C. 

*Office of Personnel Management 

July 1981 

Document is Available to tlu.: Public Through the 

National Technical Information Service 

Springfield, Virginia 22161 

Prepared For 

U.S. Dep~rtment of~Transrortation 

Federal Aviati'.)fi Admirtistration 

Office of Aviation Medicine 

Washington, D.C. 20591 

82 1 ') 

(.; 105 

{[) 

E 
OC!I'ICE 

This cloctm?nt is disseminated under the sronsorship of the Departrent of 

Transportation in the interest of inforiration exch?nge. The United States 

Govemrent assrn-es no liability for its contents or use thereof. 
Te-c.hniccl l<•poH ':)~cume-ntotion Pcgt 

--'--·:= k•;·-:-· ..... ;··:c·~ ;; ._, 

··---· ' 

' July 1981 I SELECTlO~ OF API'LICAKTS FOR THE AIR TRAFFIC 

CO~TROLLF.R OCCt:l' AT! OK 

I 

------. -------·---: .. ,-.,::-.__ ',,-,,, . . . j ____ ; 

FAA 

P•·IC'"''"''O 

Office 

.. _... 0•;;.~ -------- r0• 

of 

.;:• 

Aviation 

... _,-. ;;:~~: A:':''"''' 

Medicine 

i 11 

~ 

'. 

SOO Independence Ave •• SW 

~ashington, D.C. 20591 

I 

'" -------:'I) 

The revised test battery recommended to OP!-1 includes the Multiplex Controller Aptitud 

Test and the present OP!-1 test, AbStract Reasonin~ and Letter Sequence. The Occupati.on31 Knowledge Test is used for granting ado:i.i.tional earned credit in place of the 

p1·esent OPM Rating Guide. This new test battery meets all professional requirementS 

for an effective test including fairness.~~··----

'-

17. K.., Wo.ol• 18. cr ... , .. b.,_holi, S.to,t.,....,. 

Selection-Air Traffic Controllers 

Occupational Know1edge 

DoLuffient is available t~ the public 

through the National Technic:al Information 

Aptitude Testing Se·rvice~ Sprin~field,. Virbinia. 22161 

Personnel Research 

19. $,""'·"'-Y Cl•u•f. Col thr1 •••o<tl 

Untlassi f ied 

Form DOT F 1700.7 10-72' 

20. Suvrrty C::lo'nrf. (el th•I..,Ot•l 

tlnda&sified 
:· .. 

~FLFl'Tin:\ w· .\!'i'U\c\\TS ~·nl! rill: 

AIR TRAf'r'IC CONTHOJ. OCCUPA'i'IOt-0 

CONTENTS 

1. INTRODUCTION 

2.. HISTORICAL DEVELO?f.IENT OF PRf.SENT sr:tRC'l'ION PROCEl'~URF.S 

3. PROBLEM AND RES~ARCH OBJECTIVES 

' 4. AIR TRAFFIC CONTROL JOB ANALYSIS 

5. OVERVIEW OF RESEARCH STl'DIES 

6. 1972 STLIDY - SELECTION OF AIR TRAFFIC CONTROl. SPF.CIALISTS 

i' 1!177 STUDY - SELECTION OF AIR TRAFFIC CONTROL SPECIALISTS 

8. STUI;l¥ OF AIR TRAFF!C CON~'ROL JOB APPLICANTS 19.76-1917 

9. STUDY OF NEW APPOIIIII'EES ~0 THE AIR TRAFFIC CONTROL 

OCCUPATIO!< 1976-19'8 

10. DEVELOPMENT OF THE MUL1'IPLEl( CONTROLLER APTITUDE ~'EST 

11. DEVE!.DPMENT OF THE AIR TRAFFIC CON~'·ROL OCCUPATIONAL 

KNOWLEDGE TEST 

12. STUDY OF AIR TRAFFIC CONTROL JOB APPLICANTS 1978 

13. STUDY OF NEW APP0IIIII'EES TO THE AIR TRAFFIC CONTROL 

OCCUPM'ION 1978 

14. UNIFORM GUIDELINES ON EMPLOYEE SELECTION REQUIREMENTS 

15. CONCLUSIONS AND RECOMMENDATIONS 

kcC5;io-;-For 

iit:s c~A&i -~ 

D!IC TA~ f"=J'"' 

· Un::!.HOi"''l:JcC>d 0 Ju~.;t-ific:"!tiotL~~--1 

---------------1 

liY-----------1 

Di.stribr:tion/ 

Availability c"O·des 

-·-1Avo1l rtnd/or 

' Spec inl 

1 

2 

4 

6 

7 

11 

32 

41 

55 

61 

71 

1(10 

128 

156 

~~:~'\ \ ·-~r.,_;tru / 

·'-...... .. ~- ... 
. ~ '. 

SELECTION OF 1\PPL!ChNTS FOR THE .}\IR 'rRIIFFIC CONTROL OCC~'PATlOK 

INTI!ODU_C'J'ION 

This report has been prepared to consolidate various personnel r-esearch 

efforts during the past 9 years directed toward iwproving the select1on of 

applicaflts to work in tl'e air traffic control {A'I'C) occupation. The position 

classification standards: for the Air Traffic control Specialis-t (ATCS) Series 

(GS-2152) eEtablisi"ed by the Office of Per~onnel Manag~rrent tOPM) def'ine three 

aajoi optioOs wit~in t~e occupation. 

Flight Service Stations {FS~ which provide preflight briefing and 

assistance 8nd advisory services to pilots during .flig_ht; 

'fenrinals which provid-e control and separation of air traffic at 

airports, and, 

Centers whic~ provide control and separation qf enroute air traffic. 

The Fede-ral Av.iatiori MuJ.nistration (FAA,) is the mtjor enploying agency for 

A'n:S's with a workforce of about 26,000 enploy~s. in the GS-2152 t=-eries.. They 

represent al.nost. 50 percent ·of the total FM full-tirre staffing. Sele-ction 

and hiring: of .~pplicant·s for the occupation by FAA over the past 5 years has 

ran9ed· between 1, 70-0 a.nd 2,000 new enpl0)ees annually and can be expected to 

continue-at about the same levels. The Depart~nt of· Defense {DOD) employs a 

total of about 30.0 civilian air traffic personneL DOD rr.ay select applicants 

frow the saare- OPM register us=ed by Ure FAA, although their use of the register 

is mininal. Positions as ATCSs are essentially nonexistent in the private 

sector. 

This report su~r.marizes the vat;ious research efforts and, bae:ed on the results, 

wakes specific reCommendations for changes in: 

Present OPM. tests through which applicants qualify for employ~nt consider-ation in the ATC OCcuPations, and.t 

'The rating and ranki~q proced-ures used to e-stablish a re«lister of 

eligible applicants for appointnent to the occupation. 

~he researeh docu~nts and studies referenced in the tpxt of this report are 

identified in the biblioqraphy. 

1 

,··. 
.-,. ... 

HISTORICAL DEVELOPMENT OF PRE.SENT SELECTION PROCEDURES 

Selection of ATC. personnel has been an area of cvncern for a nurrber of 

·agencies over a cqns.iJ~rable tiwe .perio<1. Continuing resear;-ch since the early 

1950's has been directed towafd inprovin9 the processe·s by. which applicants or 

candidates for civil and ~ilitary ATCSs are evaluated, tested, and selected. 

Refei-ence {1) provides a _survey of selected researCh studies on ATC selec·tion 

during the period 1952-1972. 

ln August 1960, the FAA Civil Aeromedical Institute (CAM!) t.egan administering 

a heterogeneous battery of corr.nercially available tests on an experinental 

basis to newly selected ATtSs at the FAA Acaderey in Oklahowa City. After the 

9 week Acadeny ATC training course·, the student•s average acaderric te$t scores 

and average laboratory scores were su~med to form 8 Co~osite which was corre-

.lated {Pearson product - moment formula) with the COEPOSite of the experinental aptitude· test scores. "l'he coefficients ranged frotr .• 35 to .54. Based on 

these findings, it was concluded that aptitude tests could iw.prove selection 

of ATC applicants who would be successful in the occupation. 

'l'he colfinerc;'ial testt used experirrentally were· con·sidered to be rore suscep,-

ti~le to corrproR'ise· than tt'lose controlled by the Civ.il Service· Cozrmission 

(CSC). Consequently, those co~me~cial tests that showed the most prottise were 

U.Sed to identify CSC tests which ~·er·e similclr in factor content. Ta~le I 

identifies the five tests an"d the ~ero-.orde£" correlations derived fibrr. these 

studfe~ (2). 1'hese esc tests and an additional Air Traffic Problerr.s Tt'st 

developed for FA" were th~·n errploy.ed, beginning in 1961, in another ser ~es of 

testing sessir at the FAA Acadeny. Subseauent regresE=ion analysis re·Sulted. 

in identifying five best predictors of training success usin9 cour$e grade and 

pass or fa i 1 status of· ATC student!;. 

TABLE 1 CSC TEST VALID.I"!'!ES 

Testing Dat~S:. 

Septenber 1960-June 1962 

CSC Tests1 

e:>c s1 

esc 24 

esc 157 

esc 157 

esc ll5 

esc 540 

Spatial Patterrs 

Computations 

Abstract Reasoning 

Letter Sequence 

Oral Directions 

Air Traffic Problefll!.'l: 

*p ~ • OS: **p L:.. • 01 

lTest number assigned by esc 

2 

(li = 183) 

Course Grade 

.37*"'" 

.28** 

.28** 

.. ss•• 

• 23'** 

.41** 

(N = 143 - pass) 

(N • 40 - fail) 

Pass/Fail 

.27•·• 

.16* 

.18* 

.45** 

./.3** 

• 29** 

I 
Beginning in 1962, the te~t battery identified in Tabl.e. t serve·d as a rr.c..jor; 

deterrrina-nt .. in t.he sel.ectionof one-third of the- applicants ·wi-th little or no 

prior AT-e re-lateo expP.rienee. Applicants with experience continued to be 

selected without a requirenent for fornal testing. CAMI continued to collect 

and analyze data on the test bqttery after its adopticn·by the esc. In 

January 1964, the est battery was intrOduced a·s e R"eans to qualify all applicants regardless of their previous experience, although air traffic related 

experience continu-ed to. be used as a ranking factor for placing applicants on 

the esc register for selection. 

In October 196£'., further modifications: were made tO the ATC selection system 

ained at relieving a shortage of ATC personnel in a rapidly expanding ATC 

system. A major change provided for ~i~ing epPlicants witr specialized ATC 

experience (particularly in n:ilitary radar control) at a ~igher grade. (Gl;i-9/ 

GS-11) without requiring tt'at tpey take the CSC tes~ t-attery. 

These ATCS selection procedures renained esse-ntially unchanged from 19,68 until 

19-73 when a traxirTturr. entry age (31 years) for the Ce·l'ter and Terrrinal options 

was establiS;"hed by Public Law 92-297. At this tine, esc and FAA re-establiEhed the requireme-nt· that all ATC applicants take the CSC: te·st· battery and 

ach-ie,..~i! a score of 210 or higtler fo'r enploynent eligi_bi-lity. FroJP 1.973 to the 

present, th~ ATC selec·tion procedures bave- retrained essentially the sa.tre~· 

The rrethods and standards for estC\blisbing ATC errployrrent eligibility- havevaried fron- tirr.e-to-tirre since 1962. However, tbe total selection ProCess ... ha$ 

typicallY' involved two ele~ren-ts: the use of the esc test battery to deterrrine 

eligibility for en;Ploytrl€'nt; and tl1e weighting and use of' pr-i6r aviation- · 

re.rated experience ei~her dir.ectly or. indirectly in se).ectill.g from aJTOng those 

applicants eligible for enployment. References (2-), (3), ~4), and (5) provide 

soucces for wore deta.iled informtion on t-he developrre.nt of ATC selection piocedures and (Chapter I of reference 6) pres'ents a nore extensive disc-t\ss'ion of 

cuTrerit selection processes~ 

3 
·-··-___.......__..--'----------

PROBLI>I AND RESEI.ll<::lf ODJECT IVES 

Attritiol'). of ATCSs duririg the dev ... 1 6prrental trainin.9 period has been an area 

of continuing Concern fQr the FAA. Aa:.~lysis of t{aini,_o.9_lQ.!!-~.es over ~-~~-B.a.st 

9 yea·Es highlights_ ttu~- _fact _th_at at_tri~·ion ranged betwee_,n_ 25 ~-~~~·i?.e_ff¢nt dUrlng--fhe-·-2 --to· ·s y;ar, .. t-raihlr19_-pe_r_i<Xr-r·eqUfted-t~o--r~each -tt;e· r~11. Perforrr.ance 

ieVei (FPL). TJle. __ rrej.o.L.PCJ.t:t oL .thi.:La..t.tr-.(ti_Q(t iii...l:J:~ed t"r_om _the inability of 

t'rainees to acquire a·nd de~ronstr.ite the skills and ·x-n~"'7iedge f6 Pro9r·e·ss· 

sati'sfactori1y through the training prograrr. Thie: has been- the case indepen:-

dent of \tolhetj;·;r--or----not-thE! ___ t_r-aTfd~ng. -h-~s heen conducted on a cmrpletely decentralized basis or when selected phases of train~ng have been conducted at the 

FAA Aeedew¥ on a centralized basis. 

Attrition rates of this magnitude involved invmD:.nUos_g_es qf .lf4llioM..__2_f 

d'O~J~.~-~----~nn~.~) .l_y_ to· .. the~tU._In~:Sil.Ai.i~i;=I£~~JJ~9-L~ .. ~? __ ~.!'_a_~~}=i~_! tS: ~Of equa 1 .. ,.----

iuportance to maintaining lin ade_quate staff of full perforrr.:mce contrcllers is 

the irretrievable loss of •tine,~ since a ~ew applicant hirPd as a"replacenent 

.ust start at the beginning of the tr-aining cyC1~. · 

'l'bis slgnificant irwestaent loss to the FAA ha-s- two major con-pOnents. First, 

the SC'Ieening and ~relection critE:ria.us-ed to d~ter1fline which apPlicants to 

h·ire for the occup,ation;· second, the length of tirre takr-n to identify trainees 

who do no_t de_rroJistrate the ability to progress to t~e fell perfO:rrPance level 

required by ·th~ occupation. 

To establish rro're con.pre.hensive and uniform traininq, ar'ld to provide earler 

identific.ittion of trainees who do not denonstrate .-the necessa.r;y skil-ls and 

abilities, the FAA re-established a C<'nt.ralh·e~ ATC Initial _Qualification 

Training Progran· at the .FAA Acad-eny for the terrrinal and cf'nter options ar.d 

re-Structured t"he follow-on training conducted at the ATC facilities in January 

1976. ·-Selected Phases of training leading to FPL are conducted on a pass/fail 

basis usirtg defined job-related training outcomes and standard?· A cqrr.parable 

pass/fail tra-ining pr09raJr was established for the FSS option in Septelt'ber 

1978. 

Terminal, center, and FSS initial oualif-ication training at the FAA Acadeny 

consists of approx·i ~rately 15 weeks of instruction in both acade1ric and laboratory environ~nts where students m~st demonstrate. the ability to learn and 

aPply the knowledge and s.kills taught at the Academy. Failure to pass .t·he 

training i_s· the baS.is f:"r removal from the- ATC occupation or the FAA.. While 

the attrition rate fiuctJates trow Class to class, the· average loss rate 

during the initial qualification traini-ng over the past 3 years in terminal 

and center tr.aining has been 24 percent... Four percent of. the students voluntarily withdr-ew frorr. training and 20 percent failed the _course. This rate 

vades for-each year. For 1977, the failure rate was 21 percent and withdrawal 5 percent; for 1978, failure 30 perce-nt and withdr·awal 8 percent; and 

in 1979, failu~es were 28 P.e~cent and withdrawals 7 percent. 

4 
. The FAA A:caclell"'f training hts 

of -t~1ose trainees who do not 

to the jotneyrr.an ATc-s· level. 

conducted on a decentralized 

\ 

been effectiv~ in proviCing early identific~ti0n 

derronstrat~ t~e necess·ary abilities to progress 

Between 1970 anO 1976, wh~n all ATC training was 

basis at the individual ATC facilities, the 

averag~ length of serviCe for ATC trainee~ who separated ~row the occupation 

was bet\o/E!'en 18 and 24 nonths. Witt> initial qualification tr.aining conducted 

at the FAA Ac;::adeny, ll'Cist attrition W-hich can ne expected to occur due to per·-

forrran<::e takes place during the first 5 r.onths of errploynent. By itself 1. this 

Cha-ng-e in FAA A'l~ training prov'ides a cost avoidance on tt)f!l crder of $!0. () to 

'$12. 0 ni Ilion a year. 

There etill r·etr.ains, however, the concern "'·ith the overall attr~tion rate of 

NrC trainees. For exanple, salary and trainil;l<J costs for the initial ATC 

qualification training progr:arr represent abOut $10,-000 for each eRPloyeE 

-hired. With a h"lring rate of 1,801) Ji.ew enployees annually, a 24 ~rcent ~ 

·attrition rate equates to an inveStJPent loss of about $43 Trilli_on each yE&r. 

Co~sequently, the pri n6ry purpose· of this personnel research is to develop 

iuproved selectio·n pr~edures in order to hire those applicants who clearly 

deapn$tra-te aptitute for ATC and whose potential fo.r success is greatest .. 

Referen¢~5 (8), (9), and (10) provide sour.:ces for Il'Ore detailed inforwation on 

ATC traininee -.!ttrition, co&ta, and training progr·ams. 

• 

5 

- ----~- ·--·--·--- --:--·--··-- j 
ATC JOB ANALYSIS 

• l)icti6nary of OCcupational Titles proVide~ a general .description of the 

fuMCtions and activitieE_of ATC£E in each ~f the options. In addition, more 

4etailed job/task ~nalyses have been cornpleted for a variety _of purposes. 

Position ClassificatiO.f!· In 1978, the CSC corrpieted a revised 

Clas-Sification Standard fo~ the GS-2152 ATC -occupation which identifies 

knowledge, ~kills,. and aptitudes required in the FSS, terrdnal and 

center options. 

Perforrrance Evaluation. Between 1971 and 19-75, tl"le_ Systert Develop~nt 

Corporation urider contract with the FAA conpleted a detailed job/task 

analysis of the terminal and center options, to identify·- the basic 

skills needed for successful performnce of each task requ_ired of a 

controller, as a·basis for developing o~jective perforr~nce standard~ 

and measures. The results are currently used by the FAA in cOnduct~ng 

•..;ver the shoulder'• evaluations and extended' training rev.iews for ATCSs 

under the ATC Technical Appraisal l'rogram {TAP). 

A'OC' Work .Attributes. As part of a broader research s.tudy on· A'IC selection, Education and Public Affairs (EPA), Inc., undet contract with the 

FAA, conpleted an analysis of the work attributes assOciated with the· 

ATC occupation and p.roposed nethods for their neasureJne"nt. Their 

report (7) identifies nine major derr.ands o! .A.TC work· (e.g., cognitive 

n .. >(._ru:in~:cnts) i over 30 attf:ibutes (e.g., judgnent and reasoning 

ability) associat~d with job demands: a·nd, de5cribcs one or rrore ATC 

tasks related to these attributes. 

In addition to these rore recent A'I'C, .job a-nalyses, others going back· to the 

·1950's have been conducted ( 1) • 

6 

' 

'. 
OVERVIEI>. OF ~ESEARCH STUDIES 

·-:Personnel research related to different facets of the ATC occupation has been 

a continu-ing FAA .prograrr. since tf•..: eclr.ly 19-60'·~- This report, however, will 

.>foc:1,1s on the results of studies direc::c.d to initial selection of appliCants, 

'particular-ly thE> develoP.rrerl.t of sc·reeniny tests a·nd the use of prior aviation 

-.:experience in the selection and placenent pr()cess. O"er the !)ast 9 yea,r·s, 

·rtudies h~ve enco~assed sawple! of four rrajor population groups. 

FPL ATCSs; 

Develop~ntal ATCSs with several ye~rs of A~~ trainin9 who have ~ot 

reached FPL status; 

N~W._ATC appointees wl"lo were just- entering th~ occupa~ion; and, 

Applicants for ATC work. 

While a number of interrelated ctudie-F have been· corrp1eted, perhaps tt~e he~t 

way to provide· an overview is to briefly identify therr. in a chronological 

aP.Quence altho~gh they were obtain conduct.ed as parallel efforts. 

1972 - FAA ATCSs. ·A rrajor study t!~) of sone 14 experitrental t?sts· as 

candidatEs fo~ improving selection· of ATCS applicants wa5 cow.pleted by 

EP~.. The stUdy was based on concurrent analys.is of about 800· ATCSs 

co~ering all three options and included both journeyrran and new 

appointees in the ATC occup.ation.. 

1977 - FAA A'XS. Base-d in part Q..n longitudinal analy~is of ttte infor·-

~ion o~tained in the 1972 study~ EPA conpl~ted a ~onprehensive 

analysis of 11 experimental predictors lnvolving over 3,000 employed 

and separated ATCSs.- Thi-s study (13) als~ enconpassed all three. ATC 

options. 

1976-1977. - Applican.ts fc"r ATC Work. Since no inforJTation was available on the chatacteristics of the population which applies for ATC 

work, esc,. i!(t;t FAA's reque.st, adrr,inistered three of the exper i nental 

tests or predictor forttS along with the present GSC test battery to 

a-bout 7,500 applicants for ATC work when the exam was opened in the FAA 

Eastern ·and Southern Regions. The infor~tion obtained pr~vided a 

variety of necessary data ~~ the characteristics of individuals who 

expre.ss an interest in the ATC occupation by taking the qualifying esc 

test·. It alSo provided a basis for cor.r.:. ·.:ing: for restriction in. range 

in the statistical analysis of cs~ tests and experinental predictors 

adft'inie-tered to those- individuals hired for A~ work., 

1976-19.78 - New Appointee• to the ATC OccupatiC'n. With the estoblish-

~nt of the ATC initial qualification training at the FAA Acadeny in 

1976, the Office of AVia.tion Medicine and CAMl. initia-ted a Continuing 

p·rogratr of experilrental tt.st developwent, administ-ration, and analyses 

of dat-a on new. hir"es in the center and- tC"rmi.nal optiOns. Experinental 

tests,- includi.nq tho.s=e used ih the·:l977 studY: by ~PA an~ adreinist"ered 

7 

' 
\ 

to th(' applicant populat,ion by C5.C, w-ere given to new ATC hires at 

the FAA Academy prior to the beginning of training. ln addition, the 

CSC tf>St battery SCOt'eS were obtained for each individual whenever 

possiblt~. About 3,000 new ATC hires wE:re included in this study with 

complet~ dat:_a on all va-riables for over 1,800 appointees. 

Analysis of the data for this sa::.ple togethe-r vith the information obtained on the 7,000 opplicaots by esc in 1976-77 provided the 

basis for identifying th~se existing esc or. experimental tests which, 

in corr.bination, resulted in the best rrediction of success in ATC 

training. It should at~~ be ?Ointed out that this s~~ple group (and 

subsequent ATC training classes) provide the basis for longitudinal 

analysis of tests and training scores "'·ith on_-the-job training and· 

succ"ss as. employees por.ogress tot.oard FPL status. 

1.978-1979 - New APpointees to the ATC Occupation. Based on analyses 

completed through 1977, the FAA Office of Aviation Medicine developed 

parallel foms for those ex~erimental tests which provided the bes.t 

predictive capabi-lity. B"eginning in June 1978. two paralle:. versions 

of the tests were administered to each employee enterirtg ATC training 

between June 1978 and January 1979. Ab~ut 1.100 trainees comprise 

the sample ~or this study. The data ;nl)v"ided the basJs for item 

analys.is of the parallel test forb~ and together with othel" variables .• 

their correlation: v:ith tr~ining success· or failure. 

1978 - Apo~}_,_~ts for ATC:_ .,.o.rk. During Sept~mber - Octoher 1978. the 

eSC reopened the ATC eXamination for applicants. At FAA'S request,. 

esc adrr.inistered the parallel fonns of the experirner.tal tests used at 

the F:-u\ Academy along. with the standard esc ATC test battery. esc 

also obtained information on sex and ethnic groups, for th~se 

applicants, "'·!li-ch previously ·had been unobtainable·. This applicant 

popul.:1.t ion was comprised of about 7, 000, including aboti.t 1. 000 who 

took the examination on a "walk-inn basis as part of the affirmative 

action recruitment program. This information provided a basis not 

only for item analy·s~s of parallel test forms relative t1... the applicant 

population. but an opportunity to eva.luatc the effec:t of changes 

in the qualifying test battery on selection alternative-s and Criteria. 

~·· I 
The general relationship betwe~~ these studies is shown in Figure l 

They are summarized in the followilig sections With respect to: 

Objectives 

Sample Descriptior.s 

Predictors 

Criterion Measures 

An,.lytical Methodology 

Results 

9 
FIGURE 1. 600'AAI. llEl..I.TIQIISHlPS B£1ittJI ATC SEUCTJ()f( RESEARCH SnJUI£! 

l9116 

~-....a-J 

a...l,...u o! 19 

~~u~ 

t"Wata 

Jtl• 

('wtt:r&.l A~ 

t.~l Ot-1-&ft• 

u.- T'l"auu.aot 

....... ...-

~~ .... lit ~U..Uatl-of 

f, _.Uleat~l'l l.q.d.••n,al 

....... ,~ft~.-.1 )--....;~ ... u ~ .... a.TC: ,..... au-.. (JII:;kT • on1 lt16 .. 19'~• 

,... ... Uld 

tt.KJ"1pt.1 ... 

1Ut.1•Ue<a on 

l.TC Appl&eant 

,......laU.O,. 

10 

Wect~ •r IIIDat h'M1n.i._ 

Ate trwn ... ~ 

J 
. 1\172 STUDY - SELECTION Of' AIR TRAFFIC COI'ITROL SPECIALISTS ~ 

Ob ectives. The focus of this research was on selection as 

per ormance. It was based on concurrent validity analysis. 

questions addressed were: 

r~lated to job 

The pr:mary 

To what· extent is it possible to predict the quality of job 

performance of .a journeyman ATCS fran a battery of tests adr:linistered at the time of job application? 

To what extent can improvement be achieved by assigning applicants 

cons!stently to one of four sel.ected ATC opt!ons (FSS, Center, I?R 

or VFR tet'.Ainals) and. within these, into high or low density (activity) 

facilities? 

• To· what extent do the measures selected. to accomplish the above 

objectives affect black and white applicants with an equal degree 

of fairness?· · · 

Samlle Description.. The total sample was comprised of approxil!!ately 800 

emp oyees 1o"ho were either journeymen ATC specialists (FPL) or new ATC 

appointees including an overs;unple of black ATC specialists. Sample· 

selection provided for distribution between four types of ATC facO.li·ties 

(Centers, IFR and VFR tei'mincils, and Flight Service Stations). Samples 

for Centers and IfR terminals were further stratified between high and 

low activity facilities. Racial di·stribution, incl•J~!.ng the oversample, 

was 93 percent white and 7 percent black. 

Using 15 major cities in the u •. S., the sample was randanly selected 

(except for the over-sample) fran ATC facilities within a 100-mile radius 

of the 15 hub cities. sampling was controlled to exclude ATC specialist~ 

over 36 Yeat:s. of age and to· insure that FPL Specialists had no less than 

3 years nor more than 10 years of ATC experience with FAA. Because of the 

smal+ IIJillber of ~n in the Ate workforce, it was. not possible to stratify 

the sample selection based on sex. Participation on the part of ATC 

specialists wa.s voluntary. All subjects were administ'ered a battery of 

paper-pencil tests or forms including the present esc test battery. 

In addition, a subsample or about 260 Ate Spee!al!sts· who took the 

paper-pencil battery were administered a series of "psychomotor" tests in 

a separa~e testing session at FAA faciUties at Oklahoma City. Of the 

journeymen, 34 percent took the psyc~ • .:cotor test:~ together with. 32 percent 

of tile new appointees. Seledtion of th1s subsample was controlled .for reg!. on, 

type of Ate facilities and variance on the confidential supervisory performance evaluation whi~ was used as the .criterion measure. Racial 

distributiOn of the" "psychall:ltor"' subsalilple was 83 percent w!lite and 17 

percent black. 

11 
Preqictors. In selecting l>··cdictors for this study, four opjectives were 

considered: 

They should cover as broadly as possible, the range of job and worker attribute$ identified with the ATC occupation. 

Experimental. tests should not substantially overlap areas already 

covered by the· e~isting esc test battery which was to qe administered to the ATC Specialists. 

Experimental tests should ~ selected which, based on prior research:, appearee to have potentia! validity for ATC selections. 

The· two selected test batterie.s (paper-pencil and psychomotor) should not require rore. than 8. hours each to administer; 

Within these objectives, the following were selected: 

Paper-Pencil !)atterv 

- Aptitude .Te-~ts 

Cut"rent esc test battery ( 5 pal'ts) Minimum Coins test 

Dial and Table Reading Test 

Knowledge and· Interest tests 

Dailey Technical and Scholastic Test 

ATC General Information Test 

- "Personality• Tests 

Concept-Adjective Test Closur~ Speed Test 

- Background Information 

Biographical Inventory 

Psycholilote:r Battery 

Directional Headings Test 

Hidden Pattern& Test 

Press Test 

COntroller Decisions· Evaluation (rom;:) Test 

Multiple Task Performance Test 

Compresl!ed Speeah Test 

12 
\ 

TPe Paper-Pencil Battery was administered to the total sample group. 

adminostrators were FAA etr.?loyees, tra~ned 'oy Sducation and ?ubl:c 

The Psychomotor Battery ~as given to the subsample at FAA 

in Oklahctna C:.ty where. equ:pment requ:red tor adr'..inistration was 

Detailed d:.scussion on the source, ~ature of the test, method 

··,ofscoring, reliability and vaEd'.ty for the selected tests is provided 

·::,elsewhere • i: '· The contractor obtained and kept all test results 

' · · ,;.and infornation on ind:vidual ATC S;oecialists on a conf:d·ential basis. 

"No ind:.v:.dual data were provided to FAA. 

1;·~~~~~~iJ~~·t:.:A~~fter examining a. number of alternat:.ves. a Confident:.al ~ form was used as the cr!terion agains~ which experir,i 

tests were validated. These evaluations .were obtained by the contntctor directly from the ATC Specialists' supervisor. They were not reviewed hv 

nor were copies prov:.ded to the agency. The contractor retained all 

indiVidual evaluation data on a conf:.dential basis. The evalua~:.on covered 

the broad areas of performance with:.n a number of task behaviors for each 

Pe~forrnance Area 

Knowledge 

Percept:.on 

Coc:prehens!on 

Memory 

CC<lllll.lnication 

Judgement 

·Traffic Management Techniques 

Performance Work Stress 

Interpersonnel Skills 

Other personnel skills 

Task I t'-'ms 

-----

3 

6 

6 

2 

8 

4 

4 

4 

3 

5 

In addition an "Overall Performance" category items and a summary evaluation were obtained. 

was a seven-point rating scale with "1" being 

"7" the lowest. 

consisting of four task 

The -sumnary -evaluation 

the highest performance and 

Descriptive Infonnation. Based on biographic responses from 304 FPL ATC 

Specialists, the follow:.ng bad<ground and educational information was 

obtained: 

.. 

• 

981 were men ; 2~ women 96~ had prio!" military service 

721 claimed prior (military) experience as a contrPll!ilt' 2~. claimed.experience as a pilot 

481 took the esc test battery for appointment; 10~ more than 

once,before pass!ng 

97i completed hi~ school 2.5~ attended co liege; less than 1~ completed college 

13 

·-------- -~---"------. 

-.:..._. 
'•.• 

Analyti~l Methodology. 

statistical treatments·. 

Data collected were subjected t.o a variety of 

Among the princ~pal methods were: 

• 

~ tiple regression analysis in which test measures were 

correll\ted with perfonnan<::e evaluation measures to detenni~e 

how well they predicted job performance. 

Analysis of variance for testing differences between groups; 

minority and non-minority; journeymen and new appointees; 

and, those hired with previous aviation-related experience 

compared tp those hired withoo~t such experience. 

1-t.tltiple ctiscdminan~. function analysis, in which tests were 

evaluated in. terms of thei.r ability to aaximize placement 

within option and activity levels. 

In sllllimary, thi:J study concluded that: 

Capac! ty. of the te$ts to predict job performance of journeymen 

ATC specialists produced mixed results. The esc test cattery was 

aarginal in predicting job performance, as mea~ured by 

supervisory <'Valuations. This result, however, was due pr.iaarily to the· restriction in range of the esc test predictors. 

This restriction resulted not only fr001 the selection processes 

but also froo the screening which took place during the developmental training period as the controllers included in the sample 

progressed to the journeyman (FPL) level. This study did not 

attempt to correct for the range restriction and· consequently 

the predictor relationship of the esc r.,sts to the job performance criteria was not detennined. The psychomotor tests 

developed coroistently significant correJations with supervisory 

job perforaance evaluations • 

.. 2. By combining. paper-pencil and psychomotor tests, the analysis 

indicated that it was possible to improve assignments to the 

different ATC options based on tests results and, with additional 

tests, to high or low density cente!' or IFR tenninals. 

3. If relevance to the- job is the primary factor in detennining 

test acceptability, the tests proposed from the research met 

that criterion. They also predicted job performance equally 

well for each racial group (black and white). However, blacks 

as a group consistently scored lower on the tests than did whites. 

14 
This study provided FAA "it.h valuable infonna,t!on alld insights on the 

problems associated. with sele,;•.ion and placement of applicants for ATC. 

work. HO"ever, action on the res~'lrch results and recomrnenda tions was 

defer!'ed due to a number of consid!'~ations including the complexity of 

the spec.ialized equipment required for the psychomotor tests, logistical 

difficulties in te.st administration ar.d complexity in test scoring and 

ranking of applicants. 

15 
\ 

1977 S11JDY - SELECTION OF AIR TRAFFIC CON:RCL. SPECIALISTS 

Continuing concern with the rate of ATC trainee attrition and ccnsiderat!on Of reestablishing centralized ATC traininv at the PAA Acad~y· 

resulted :n a review of FAA selection and screen!nt; i)Olic!e:s in 

December 1974 ~~~and core~letion of~ cost analys:s st~Jy o: alt~rnatives l"l in' M>rch 1975. · A.'110ng the actions resulting ~rom this review, 

FAA contracted with Education ar.d Pub~!c Affai~s. Inc., (EPA) for a 

. follow-on analysis of ATC selection tests in Jw.: 1975. A su:n."Br/ of 

the final EPA report is provided els'"'<here (I < \ • 

·Objectives. The objectives of this research were directed to essentially 

. the same concerns as the previous (1972) study completed by EPA: sel~ction, 

placement, and fairness. However, the study design significantly expanded 

sample s!ze, structure, and representation;· encompaSsed more criterion 

11111asures of ATC job success; and incorporated an evaluation of prior 

aviation-related experience and educational level as predictors of ATC 

Job success. 

Long!tud!.!Jal Analvs:.s of 1972 Experimental Tests. The first effort !n 

this study W<l,S to analyze on a long! tudinal bas:.s the relat!onships 

• between the exper!mental test results obtained during 1971 and cr:terion 

measures related to the ATC specialists who participated as 'Of 1975. 

Valid!ty coefficients of each test score with each criterion measure •~re 

calculated. A factor analysis of the tests was also undertaken to determine overlapping of underlying var:ables. The tests selected from these 

analyses were then subjected to multiple regression analysis to determine 

· the minimum number of tests or test scores w'h:.ch WOIJld predict the . 

. aax!mum proportion of variance in each of the criterion ceasures. 

Table 2 identifies those 1971 experimental tests and predictors .(X) 

which, based on their s~le correlation with the various criterion measures 

establ:shed for the analysis, wa1"r!!nted further study by means of field 

validation. 

16 

~····-, 
Table 2 Preqictor and Criterion Measu.res of. ATC ''Success" 

U975). for Controllers Pat:ticioatir:g in 1971 ATC 

Res.earch · 

Sepa- ProgressiW.+ Present 1971 Supv. Sup/Sta!"f 

ration Attrition ~ion Assess. Positi:m. 

m DEV FPL EV F7L ·FPL••• FPL*'" 

em: X X X X X X 

D1.al. ~ing X X 

!)Jr. lleadi.ng X X 

··Air 'l'raffic Prob. * .,. X X x· ,.,. - Ai:ith. Reasoning** X X 

X!.C General Info. X . Conc:ept Adjective X X X X 

.lliDgrapbical Info, X X X X X X 

*Air Tra'Uic ProP!ems Csc Test No. 54o 

-Arithmetic Reasoning was Part 5 of the Dai,ley Technical and 

Scholastic Test (TST) 

-*Supervisory assessment criterion data available for 1971 FPL. 

ATCS only. Nci' new 1971 appointees progressed to ATC superVi$ory or staff positions by 1975. 

With some modificatiops the§e experimental tests used in 1971 beCame the 

•core" test battery for the 1977 research. A more detailed discussion 

of the methodology and results of this longitudinal study is prov~ded elsewhere (14). 

Sample Descr-£Etion - 1917 Study. A comprehensive sample design was 

oonstructea denne the Ate popul;.tion to ensure a representative 

sample for three specific "year of· hire" groups. These gr-oups repreaented three ATC caree!' "stages": ( 1) New Hires ( 1976); (2) Developmental ATC Specialists with 2 to 3 year-s ATC experience with FAA ( 1913 

and 1974); and, (3) ATC Specialists with 2 to 6 years experience at the 

journeYDBn (FPL) level ( 19.69 and 1970). 

In lllldition to these pr:lmary ATCS sample.s, three additional ATCs 

samples were inclu<:led: 

• An cversample of curr-ently employed women and minority 

ATCS's in tbe same. three "year of hiz:e" groups. 

17 
ATC Special;sts who participated in the 1972 research. 

A l!alllPle of ATC spec~"lists Who were. hired during the three 

tilDe periods sampled, bu;; •'ho had separated from ATC work 

before reaching FPL status. 

Sample selection was constrained to exclude employees who were in ATC 

starr or supervisory posotions or over age 31 at the t:me they were 

hired except in the FSS option. 

Sample selection for the three primary groups was based on stratified 

, ranacm l!alllPling methods to provide a proportionally representative group 

total constrained ATC universe for each of the four ATC "options" 

VFR, IFR, and ARTCC) with respect to both the initial and current 

Option of assignment. 

3 identifies tl;le various ATC specialists, samples by year grOup, the 

number desired and the samples actually obtained. Since participation was : vtilur•V~Mr. the number invited was expanded where possible to provide for 

""'oun;<•o•<•n ani! noltresponses •. 

TABL&; 3. ATC SPECIALIST BY YJ;:AR HIRED, 

SAMPLE GROUPS AND ATC CAREER STATUS 

A'l'C Saznple ATC Percent of 

Year CAre.er Nulnber Desired Obtained Desired 

Groups Status Irwi ted --- Sample S4!ftple Sample 

(1) Elnployed A'l'CS FPL 1344 BOO 754 (9nl 

(2) A'l'CS overs&~~~ple !"PP. 151 200 3l ( 6\) 

(31 Separated AT'CS DEV 362 

(1) Elllpl oyed A'l'CS DEV 1127 BOO 740 (93\) 

(21 ATCS Oversample DEV 258 200 72 (36\) 

(31 Separatt~d A'l'CS DEV 166 

Ill ElnploY'ed ATCS New Hires 

(DEVl 610 610 590 (97\) 

(1) Employed A'l'CS FPL. 4BO .480. 270 (56\). 

(2) Separllted A'l'CS FPL/DEV -·- 74 

18 
\ 

number of women anc;l m!nor~ties who. volunteered was· not adequate for 

···~~~~~::;: by year group or ATC option. Consequently, the ana1ys:s on test 

·:; combined year and option groups to prov.:cte 235 women and 321 

The oversa:nple for women a.nd m!norities was used only for 

of the fairnes~ of the ·ATC "success" predictors. The final 

report {Ch.!ptt.'r \'II PI r~·ll'l"l'lh'l' 1',) pl-<>\Lks a ,-,,::.plctv ,\is,· 11 .~:-~i"n ~·n 

r:h.' t h~)dll l ~·1-:y, 

Two of the tests used in the 1972.study were roqdified for 

1977 research •. 

CODE (Controller Dec!sion Evaluation). This test consisted of . three film versions of a computer simulation of ooV:.ng air 

•• traffi·c patterns appearing on a radar scope. Initially these 

were converted to slide projector presentation to eliminate the 

. need for oovie projector equipment and to simplify bOth the 

response· recording and scoring. Group adm!nistration of the 

various adapt ions of CODE during th:.s test development phase 

clearly pointed up the practical problems of using film or slide 

project:on equipment in test adm!nistration. Consequently, a 

paper-pencil version was developed which incorporated measures 

of abilities to identify potential confEcts of aircraft as well 

as the traditional kinds of aptitudes within an air traff:.c control 

context. The resulting test, the Multiplex Controll~r Aptitude 

Test (MCAT) was nade available to EPA as ·a subst:tute for the 

CODE tests. 

Arithmetic. Reasoning, In the prior research. the ar:!.·thmetic reasoning 

test was one part of the Dailey Technical and Scholastic Test (TS!). 

Since this part could not. be. giv<Jn as an iso.lated te>Jt, an Arithmetic 

Reasonirn; test developed by. the Army Air For~, which was s!ctilar, 

was used !n the 1977 research. 

In ad<!i tion to the predictors derived fran the longitudinal analysis of 

the experimental tests administered in 1971, three other predictors were 

used. 

Pre,.employment Experience Questionnaire (i'tQ) 

To obta,in specific data from participating ATC specialists on 

various kinds of pre-FAA exoertence and education, a questionnaire 

was developed from the esc Rating. Guide elements u:;ed as a basis 

for granting additional credit in the ·ATe employment selection 

·Jll"oc.es-.. 

19 

I 

,. '!'~ ,-,, 
. • 

'.,--' 

ATC Occupational Knowled-ge 'l'e~t (OKT} 

'l'bis· test was developed to be "job-know1edge spec:.nc, • Conse-: 

quently, 1 t Wlj.S not included in the 197'1 r-esearch for the purpose 

of evaluating its ·use in screening ATC job applicants for employ'-

ment eligibility.. It was used to measure the "quaHty" of pr:.or 

exPerience as a potentially improved basis for granting additional 

credit for experience in the selection process in place of the 

existing esc Rating Guide. 

• Sixteen Personality Factor• Questionnaire ( 16 PF) 

.· the· 16 PF Questionnaire '-s a4ministered as part of tl!e mec;lical q~ification process to all enterlng ATC Specialists. It is designed to measure important perso!lll-lity characteristics not otherwise measured and the 1977 research offered the opportun:.ty to assess 

its. utility for possible selection or placement purposes. 

experimental test battery used in the 1971 research my be divided 

""'""'JLv into two groups of inatruments; (1) Cognitive Predictors, tests, 

predetermined right or wrong anawer's; and (2) Other Predictors 

·w::~Q:I)~;~~~r. responses or-dinarily have no inherent value of "right" or 

" except as measurabie against criteria or values external to the 

the follooiling identifies the pred:.ctor's used_ within these two groups. 

Cogpitive Predic.tors 

!til tip lex Controller APt! tude Test 

Directional. Headings Test 

Diu Reading Test 

Arithmetic Reasoning Test 

ATC General Infornation Test 

Present esc Test Battery (5 parts) 

ATC oCcupational Knowledge Test 

• Other Pred!.ctors 

(HCAT) 

Pre-Employment. Experience Questionnaire 

Concept Adjective 

,· :-::;:;;::. 

;,~·:\:;,- \ -~--:.t·_. 

Biographical Inventory ·· ·. · · ' . -c-,,,, , · ·Sixteen Personality_Factor Questionnaire (16 PFJ 

, . :·;x:~~ • 

.. .. '"· : 

• 
ATC Specialists who particO.pated in the 1972 research. 

A sample of ATC specielists who were hired dur!ng the three · 

time periods sampled, buc who had separated frcxn ATC work 

before reaching FPL status. 

Sample selection was constrained to exclude employees who were :n ATC 

staff or supervisory positions or over age 31 at the time they were 

hired except in the FSS option. 

Sample selection for the three primary groups was based on ~tratified 

random sampling methods to provide a propm·tionally representative group 

or the total constrained ATC universe for each of the four ATC "options" 

VF"R, IFR, and ARTCC) with respect to both the initial and current 

optic>n of assignment. 

TABbE 3. ATC SPECIALIST BY YEAR HIRE[), 

SAMPLE GROUPS AND ATC CAREER STATUS 

A'l't: Sample ATC Percent 

Year career Number Desired Ob~ined Desired 

Groups Status Invited Sample $!t.!llple sample --- (1) Employed A'l't:S Fl'L 1344 BOO 754 ('}4\) 

12) ATCS OVersample FPL 151 200 31 ( 6\) 

Ill Separa·ted. ATCS DEV 362 

(11 Employed ATCS DEV 1127 800 740 (93\) 

(2) A'l't:S OVer sample DEV 258 200 72 (36\) 

(31 Separated ATCS DEV 166 

Ill Empl.,Yed A'l't:S New Hires 

(OEV) 610 610 590 (97\) 

(l) Employed A'l't:S FPt 480 480 270 (56\) 

(21 Sepaat·ed A'l't:S FP.L/OEV 74 

18 

o! 
nurn~er of women and m!norities who volunteered was ~ot adequate for 

by year group or AT: option. Consequently, the analysis on test 

combined year and option groups to provi.de 235 wo:nen and 321 

'·•!"''~'•'·"·· The oversample for' women and m.:r}or:ties was used only for 

of the faimess of the ATC "success" predictors. The r:nal· 

report \L!J .• ;·tt·i· \'! 1 ,q r~-~~·n.·n.·L· I'd !'~"·''- i.il'L' .1 .. -:-.pll'l•: .ti····ll~~-d"~'i ,,n 

·····l h· .,[,. 1-l'):y. 

~·o of the tests used in the 1972 study were ~dified for 

1977 research. 

CODE (Controller Dec!sion Evaluation). This test consisted of 

three film versions of a computer simulation of moving air 

traffic patterns appearing on a radar scope~ Initially these 

were converted to slide projector presentation to eliminate the 

need for movie projector equipment and to simplify ~th the 

response recording and scoring. Group ad~~nistration of the 

various adaptions of CODE during this test development pr~e clearly pointed up the practical problems of using film or slide 

projection equipment in test adm!nistration. Consequently. a 

paper-pencil version was developed which incorporated measu~es 

of abilities to identify potential confEcts of aircraft as well 

as the traditional ki.nds of aptitudes with!n an a:r traffoc control 

context. The resulting test, the t1ultiplex Controller Aptitude 

Test (MCAT) was made available to EPA as a substitute for the 

CODE tests. 

Arithmet:c Reasoning. In. the prior research, the arithmet:c. reasoning te~t was one part or the Dailey Technical and Scholastic Test (TST). 

Since this part could not be given as an isolated test, an Arithmetic 

Reasoning test· developed by the Army Air Force, which was sim!lar, 

was used in the 1977 research. 

In addition to the predictors derived from the longitudinal analysis of 

the exper:!.mental tests adm!nistered in 1971, three other predictors were 

used. 

Pre-employment Exoerience Questionnaire (PEQ) 

To obtain specific data from participating ATC specialists on 

various kinds of pre-FAA exoerience and education, a questionnaire 

was developed from the esc Rating Guide elements used as a basis. 

for granting additional credit in the ATC employment selection 

j)rocess. 

19 

I 
ATC Occupatio~!. Knowledge Test {OK'!') 

This test was developed to be "job-kn.owledge specific." Consequently, it was not .. included in the 1977 research for the purpose 

of evaluating its use in $Creening ATC job applicants for employment eligibility. It was used to measure the "quaEty" of prior expe~ience as a potentially improved basis for granting addi~ional 

credit for experience in the selection process in place of the 

existing CSC Rating Guide. 

Sixteen Persona:l!ty Factor Questionnaire ( 16 PF) 

The 16 PF Questiqnnaire is administerM as part of the medical 

qualification process to all entering ATC Specialists. It is designed to measure important personality characteristics not otherwise measured artd the 1977 research offered the opportunity to assess 

its utility for possible selection or placement purposes. 

experimental test battety used in the 1977 research my be divided ~ghl.y into two groups of instruments; (1) Cognitive Predictors, tests, 

predetermined right or wrong answers; ana (2) Other Pred:.ctors 

~nr'" cam: responses ordinarily have no inherent value of "r!ght" or 

~oru•" except as measurable against criteria or values external tc the 

The following id.entifies the predictors used w1 th:.n these two groups. 

Cognitive Predictors 

lttltip1ex Controller· Aptitude Test (MCAT) 

&irectional Headings Test 

Dial Reading Test 

Arithmetic Reasoning Test 

ATC ~neral Information Test 

Present CSC Test Battery (5 parts) 

ATC Occupational Knowledge Test 

Other Predictors 

Pre-Employmet'lt Experience Questionnaire 

Concept Adjective 

Biographical Inventory 

Sixteen Personality Factor Questionnaire ( 16 PF) 

20 
than r'e-adtn:.nister the pres·ent esc test battery' the intent was to 

the esc test scaNs for part!c!pat!ng A:c spec!al:su fran existing 

However, th:s d!d Pot prove to be 'feas:.ble for a large number of 

employees in the sample. Lv~~equently, aP4lys!s of the experimental 

in relation to the existing c.::s test batter-J as predictors of ATC 

.icess was not possible in this study. 

"cc)gn!t:lve predictors" iind "other pred!.ctors" were each analyzed 

aga!nst the :!r'! ter:on r::easures used. to def!.ne "sucCess" :..n the 

occupation. They were also analyzed together in a supp~emental repcrt 

to determine the extent to which "other predictors" aaded to the 

.nc1ot.v of the "cognitive predictors." 

the objective was to obtain pre-employment and 

j!;~~~:g~;~·:~rc~at:lon on all ATC specialists. !n the sample groups. The battery was adrn:.n!stered to approxinately 50 percent . and 1973•1974 sa:nple and all new hi.res in the 1976 gro~p. 

:c,.Jile,ct;:on of e:ilployees to take the experimental test battery was controlled D~)v;.oe proportional distribution by ATC option within each year group. e~erimental tests were also admin~stered tc all employees included in 

ATCS oversamp'le groops. 

· ,·. Prec!!ctor data were obta!ned in se..,eral ways. The Prior Experience Questicn-

(PEQ) was completed by the ATC specialists at the tim" tney agreed to 

.. · participate. The experiment')l tests were adm!n!ster'ed to the new hires 

·It the FAA Academy on the!r f!rst cay of atter.dance by the EPA staff ..,. •. ,,, .. ~u by FAA personnel. For those ATC spec!aEsts assigned to facilities, 

te:~ts were given by FAA test administrators who were trained by EPA. 

full d!scussion of the data collection methOdology is provided in the FAA 

.·report (Cl~aptcr V~ll of refere-:1cc 14). 

Criteria and Measures of ATC Success. To dete~e how valid experimental 

:. tests, prior experience and education are as predictors of A!C success, 

·operational def!n!t.ions (criteria) of success and ways to quantitatively 

measure them had to be established. The follOW::ng four criterion measures 

wer<e used in the study and were then ccxnbined into a s!.ngle "aggregate" 

~Ueasure of ATC success. · 

21 
\ 

Training Perfol"''ll'!nCfi!. This was me.asured by scores rec.,::.ved on 

the ATC Laboratory Problems anct the Controller Skills Test during 

initial. ATC trainong· at the FAA Acadef11Y. These scores were selected 

since they require students to demonstrate· operatoon<1l appl!cat::.on 

of academic knoWledge. 

On~the-job Perfonnance. This was measured by· conf::.dent:!.al job/task 

assessments prepared by the employee's supervisor. It included 

54 questions on ATC job tasks and four general questoons on qual:ty 

of job performance. Responses to a seven-point "overall" rator.g 

scale were selected as the measure for -:>n-the-job perforn:ance. 

Progression. Thos was measured by the ATC "opt:.on" to wh:.ch the 

A!CS was initially assigned when hired, compared to the "option" the ATCS was ass.igned to on Jar.uary 1, 1976. 

Four "options" were defined to represent general coo:plex: ty le.vels 

of ATCS work (FSS, VFR, IFR, ARTCC) wit.h ARTCC defined as the 

"highest" complexity level. Wothon this hierarchy, progress::.on 

values were establoshed·as follows: 

•r gh•......,ssigned when an ATCS was in an option of a ccmP ty level the.same·as or higher than the '-nitoal opt!on 

to wh:Cch assogned (i.e., VFR initial assognment; IFR curre!1t 

assignment on Jam:'l.ry 1 , 1976). 

"Low"--asogned when an ATCS was in an option of a lower 

complexity leve.l than the option to which :n:t!all_y __ 

ass!gned (i.e., ARTCC initial assignment; FSS current 

assigr.rnent on Jan,.ary 1 , 1976} • 

Attrition. This was measured by whether or not ATC Specialists hired 

during the year groups samp'led for this study were still ·-~Inployed in 

A1C work. Those still employed as ATC SpeCialists were assigned a 

"hiqh" score;.- th_ose separated we:re assicmed a 01 low'" score~ · 

Aggregate Crite~ion of ATC su=cess. This was constructeQ from the 

four. individual criterioA (training, on~the-job performance progreslion ancl attrition) and their measures and provides a five-point 

scale value. for ATC "succe$s. " 

The predictors of ATC •success" used in this study - experimental 

tests, prior experience, and education -- were analyzed for validity 

against each of thE! four criterion measures of ATC"sucC"..ss• -- trainin~ 

22 

1 
scores, superv:sory asse~~~ts, progress:on, and attr:t:on. F!nal 

conclusions anq recocoendations are based on the \alidity of the 

predictors with the aggregate criterion of ATC success. A full 

d!scuss!on of the develo~ent of each of the cr!te~!on ~asures !s 

prov!ded in the EPA Report lL:i:.q'tt.:r \. ,,: r~~~.·ro..'lh'•,: l-.J. 

Descriptive !n("orm;tion. Data on the education and ex;:>erie~ce oa·~grouc.d 

of ATC special!sts •'ho participated in the 1977 study are orov:ced :n t::e 

f.PA Rep'-"rl (Chaptt.~r \"I I tlf t·ct ... ~rcncc 14). i ho..• ::_,.Jsl :=l~Ht.. ... ·~l ..:iJ,)Il_~:t.:s bct·.,:ct:n 

the thr~e 11y~ar-l.li-hirt;?" grl1llps (,~nd the 1972 :->~U7;plc) \:ere in educativnal 

level, rdlit3ry_ toervice and- pilot e:-:pericnt::c. T<1ble !. co1::pan:s the \'.lrious 

saml'le gn'ups on ~t:-l<!cted variables. 

Tl\BL£ 4. EDtr...A'!'lON AJ:D £XFIERIE!:CE LE"'.lE:...S 

Year Hired as A'!'CS 

1969-70 

Prior to 1969 1973 1973-H 

1971 1970 1974 Over sa."'!'.ol e 1976 

(N•3~) (~9) (N=661) (N=1C3) tN='59~1 

· ; High School n• 34\ 24\ 25\ 1H 

25\· 53\ 53\ 59\ 56\ · Some College 13\ 23\ 16\ 29, · CoUeqe Deqree· ( s) l\ 

Military Service 96\ 75\ 74\ 56\ 71\ 

A'l'C ~per ierice 72\ 

ITR 35\ 39\ 27\ 32\ 

VFR ·-- J9\ 38\ 26\ 37\ 

Pilot 2\ 30\ 34~ ll\ "33\ 

The sample of 30~ ATCS hired prior to 1971 was derived fran the 1972 study 01). 

ATC $pecialists ·could have been. hired between 1<)60 and 1970. 

Analytical Methodology. Before undertaking the valid! ty analysis, a number 

of prel!.minary &nalYses were made to: 

• 

• 

~bstantiate that tile sample of ATC specialists who did. volunteer 

was not; exoept for the proportion of minorities, essentially 

different from those who did not volunteer. 

Caletilate the means, standard deviations, reliability, and intercorrelations of experimental predictors and ATC suc<;ess criterion 

measures. 

23 
Detenn.:.ne that the order of ~~:;.T test. act:-..:n:s:rat:on ct:.d nc: affect 

test results. 

Artalyze t:"'ain:ing scores a:1:.1 convert them to sta'1-:!ar:.:zed(Z) scores. 

Analyze pre.:.e~plo:~ent ex;:;erience and establ!sh er::~p:r!cal scale 

values for validat:on. 

Analyze prec!ctors for !nch::>!on !n the val!dat!cn study. 

fUll d!scuss!on of these prel~~naFJ a~lyses !s prov!ded !n the EPA 

t (Ch>lptcr X ~~f n.:f~o·t·t.·nn• 1!.). Tht.• st<,Jti.stic:tl results l1.n distribution 9 

, st~nHJard •Jc .. ·vLtt L'ns. and lntt·t·nq-r..:-LHi•'BS ~n·t:· l'r .. ,vid~·J in Tables iX.::! 

tlrr<'ll<h 1:\ . .::!1 ut that r~l'~'rt (14). l~·t·~ult~ of prt.•li::·in:.n· st.Jtistical ~nalYses 

the n .. JU-<.:l.!;;uitivl· pr .. •dil'l(lt'!; arp .. ·.lt' cl~i•\,'il~t"t-' (15). - . 

a result of these prel!Jn:.nary analyses, two of the. cognitive tests 

excluded from the val!da:t:.cn analys!s. 

C!v:.l Serv:.ce Tests. The !na~!l:ty to obta'.n the scores on the 

CSC test. battery. r'or a s:gn.:f:cant number of ATC spec:.ali.sts in 

the sample and the effect of restr:.ct!on in range on those scores which 

were ava.:lable. ser:ously :nter~ered w:th proceed:ng wi:h the 

vaEdity analya:.s of the esc tests :.n the EPA study. 

ATC General Infontat:.or. Tests. Th:.s test was excluded fran vaEd!ty 

analys:.s since :.t was only acr.::.nistered to the n.ew h!res attend!ng 

the FAA Academy training; :.twas not g:.ven to the 1973-7~ or 1969-70 

ATC samples because of time constraints on f!eld testing. 

following predictors were :.ncluded in the pr'-mary val!-

ltlltiplex Cont.roller Aptitude Test (MCAT) 

. Directional Heading Test 

· · Dial Read!.ng Test 

Arithmetic Reason!ng·Test 

ATC Occupational Knowledge Test (OICT.) 

Pre-employment Experience· (PEQ} 

following non-cognitive predictors were included in the supplemental 

IUclat:lon analys!s: 

24 

I 

~''·':.~.-
Concept Adject!. ve 

Biographical Inventor":{ 

S!.xteen Personal!.ty .facto.r Cuest:on."la!re· ( 16FF) 

In sunmary, the val:ctat.:cn oothodology for the cogn:t:.ve prec:c';.ors (and 

prior ex~erience) cons'sted· of the follO"'i:lli ste;:>s: 

1. Statistical analyses of each of the Slilallest h.:nogeneous sa:::~les 

(!.e. each ATC option and all ATC options comb!.ned W:.thin each 

year grouR aga:..nst each of the four ind:.v!dual A:C success 

c:r!.terion se~telz·). Th!.s resulted in 118 separate analyses • 

. 2. Pred!.ctors selected from Step 1 (based on val!.d!ty coeffic!.ents 

and s!gnif!.cant levels) for each ATC option and all. ATC optior.s 

c001bined across all year groups were .then analyzed to :dentify 

the best overall set of pred:.ctors for each cf the four :.n~ivicual 

criter:on measures. Th!s resulted :n 17 separate analyses. 

3. Results of each of the four :.nd!vidual criter:.on l!l<;'asures >;e:·e 

then examined for each ATC opt:.on and all options camboned to dete~ne the best set of predoctors across cr:. terion measures. 

This resulted :.n 17 sepa.Pate ar.alyses. 

4. The final set of predictors from Step 3 were then validated against 

.the aggregate ATC success criterion leading to derivation of 

.reign ted test and experience scores·. This involved five separate 

analyses. 

5. The weoghted test battery and exper~ence scores were then validated 

against the· four single criterion measures. This involved four 

separate analyses. 

The specif!c analytical techniques a;>pl!.ed and the rationale for eac.'l of 

the s.u·p~ in tilt.:' mL·thnJt ii J.~Y m·a.•d at"t.' Jully di~;l·u:-;s'-·~! in Ch;lptl·r :\ t'f 

referctu.:l:! 14. Tlw rt.•sults til llu.~ analvso..·~ .:md the \'aiiditi('!-> oht~lincd are 

p.rovided in Cl!~tplt.•rs XI, XII, XIII, Xl\' and X\', \,f t·ctcn.•n.:c lt.. tlu ... • 

_,\ppcn:JiccH" to the final .f-J'A rctwrt (]b). 

The methodology previously discussed was modHied for analyses of the 

liOn-cognitive predictors. to canpensate for the differences imposed by the 

Wle of •personality" as opposed to 3ptitude or knowledge test~. 

The l!¥lst significant change in methodology resulted from the need to 

«DPirically establish the appr'opr:-iate directionality of the various noncognitive predictor scales before undertaking validity analys:.s. The 

specific analytical techniques applied in evaluating the non-cognitive 

predictors are !dentif)ced in reference J· s. 

25 
\ 

The methcd0logy to evai\.l3te the f'airness of the el<per~ntal test battery, 

prior ex;leri<:nce and the Oc,.:,%t:onal Knowledge Test ;;as directed to 

detenn:.nir.g differential validi•:: in accordanO'e with the c:\ifonn Gu:ctel:r.es 

on Elllployee Selec.t.ion Procedures ( t i' which state: 

"When member:; of one racial, ethn:c or sex group character:.stically 

obta:.n lower scores on a select:.on procedure than me~bers of 

another group, and t~e d!.fferences are not reflected :n ±.fferences 

!n measures of job perfornance, use of the select:.on procedure 

may unfairly deny opportunit:.es t:> rne:obers of the group tha~ 

obtains the lower scores." 

The methodology for evaluating fairness aspects of the pred:.ctors used 

in this research is discussed in Ch.:.lptct· X\'1 l1 f rt<fert!Ih.'l.: JL.. 

Re;rults. The follow:.ng briefly Slt!lll:Br:.zes th<' results of th:.s research 

11tudy. More detailed discuss:.on is provided in references I '. I~, '"'d 

15. 

Selection 

Two of the. el<per:mental tests--'Multiplex Controller Apt:.tude (Me;;:) 

and Direct:.onal Head:.ngs--"red:.cted the ATC success criter:.a 

established for the study at stat:.stically s:g'1:.f:.cant levels of 

confidence for all ATC opt:.ons como:.ned and for tne three prjrarf 

year groups sampled (1969-70; 1973-74; and 1976). TWo other ex- per~ntal tests--Arithmetic Reasoning and Dial Read:.ng--either d:d 

not predict the ATC success criteria or d:d not add apprec:.ably to 

t!le prediction values obta:ned from MCAT and D:rectional Headings. 

Preemployment aviation-related eXperience and the ATC Occupational Knowledge Test, while not intended for use in determin:.ng :.n:cial 

appointment eligibility, predicted ATC success at.stat:st:cally signifiCant levels of confidence and increased the val:.d:ty co-· 

efficients obtained with the experimental tests. 

Education-beyond high school level prior to FAA employoent did net 

predict ATC success in either .a positive or negative direction. 

Essentially all of the controllers in the samples included in this 

study had at least a high school education. 

• Other eJq>erimental instruments--16 Persc!Jhl:.ty Factor Questionna:.re, 

Concept Adj,ective· and Biographical !nfol'!llat:.on Questionna:re--did not 

add «ppreciably to the predictive capability of the el<per:mental 

test battery, preemployment experience and the Ate Occupational Knowledge T~tst. 

26 
!'~:~:~~~=" derived from the analysi·s of qombined predictors against ;J ATC Success Criterion by ATC option for all year grry~ps 

provided in Table 5. Weights for the exper:mental tests 

der•ivE•d from the llllltiple regression ~na.lysis. Separate weignts 

"'"'e>·<.!P'•u for each .test for each ATC option and for all ATC options 

derived weight<; for.jlll ATC options combined were MCAT 

~lct--;n; MC,AT Aptitude--21; Directional Headings--15; and, Dial 

T.ABLE 5. VALIDITI.ES OF COMBINED PREDICTO!lS 

AGAINST THE.AGGREGATE ATC "SUCCESS" CRITERlON 

(BY ATC OPTION-- ALL YEARS COMBINED) 

Weighted Test W~ighted Test Battery 

Weighted Test Battery.~ and PEQ~ 

Batter:! Scores PEQ OKT 

N r il! R 

-· 

df 

--·-·-. R 

r.ss 196 • 2..3** 193 • 2.6•" 192. • 2.6"* 

VFR 479 • 2.6** 474 .44CCO 42.3 .45** 

lFR 499 • 2.6•• 494 • 39** 443 • 43** 

.RTCC 445 • 30** 42..5 • 32.*~ 388 • 37** 

.. All Options 1309 .2.6** 12.87 • 3Z*• IZOS • 34**· 

•• a ~ •. 01 

Academy trainees ( 1976) in the . Terminal option we1·e not identified by IFR 

or VFR options. Consequently, they were included in both the IFR and 

VFR option Ns, but only counted once !.n the total N • s for all ATC 

options. 

Reference (13) Appendix A shows the validity coeff!.cients derived for 

the experimental test battery (unweighted), prior experience, ATC 

Occupational Knowledge Test (OKT) and education level for each of the 

four ATC success criteria (excluding the aggregate criterion) by ATC option. 

Appendix B (13) shows the validity of selected experimental tests with the 

aggregate ATC success cr! terion. · 

27 

... 1 

~;.--
Placecent 

The we:ghted exper:mental test battery did differentiate between 

the FSS, T.enn:nal (IFR, VFR) and ARTCC options. Average scores for 

ATC specialists in FSS tenn:nals and ARTCCs were different at statis-· 

tically significant levels of confidence. The average FSS score 

was lowest and ARTCC highest. Table 6 provides the ccc:parative 

mean scores for the var:.ous ATC .options and year grQL:pS. 

TAB.LE 6. MEANWElGHTED TEST BATTERY SCORES 

BY ATC OPTION 

ATC 

Option. 

1969-70 & 1973-74 

ATC HIRES 

.!L Meo.n SD 

1970 NEW ATC HIRES 

(ACADEMY ·TRAINEES) 

.2L_ Mean SO 

FSS 

VFR 

IFR 

AB.TCC 

19o 

170 

189 

lBZ 

227.4 

244.6 

250.0 

264.J 

40.2 --- (TERM) 

35. ---- 310 

37. 

36.6 Z63 

242.0 

247.9 

35.0 

38.4 

Sox 

lllri 

Analysis of varianG:e shows that the means· of scores differ significantly 

at the 1 percent level of confidehc~ between FSS, Terminal (VFR and IFR) 

and ARTCC for the 1969-70 and 1973~74 groups. The 1976 ATC ~roup 

difference bet\..'een terminal and ARTCC is si~l1ificant 3t the 5 peT'ccnt ]cvt: 

Test Fa!mess 

Women as a group scored lower on each of the predictors and on the 

aggregate criterion of ATC success. In each case, these differences 

were s±gn!ficant at the l percent level of confidence (Table 7). 

TAilLE 7 - ~!Et1o'\_~ ST~"iO.fl!\0 D~\'l,~Tl0:\_5_ASD t-TES1 

RESULTS FOR PRED!CTL>RS A:"D THE AGGREGATE 

ATC ''SUCCESS" CRI1CR!O:i FORXE~D~ 

- - - ~ - - - - - - Predictors- - - - - -

Weighte~ 

Telt Battery 

N Mean SO 

1397 ~reo 17l ~JA.S 480 

(tool.l4f~· Ol.l 

P£Q 

N Hea:n SO 

2:f36·J7i 2.9 

23S .B 1.6 

(t-1S.02;~.0ll 

28 

or. 

N Mean SO 

Tila 76.6 IT:3 

15S 64.6 1.6.3 

(t:•ll. 21;p<. Oll -. 

-criterion---

Aqg·regau ATC 

•success" 

N He&n ·SO 

1254 3.1 1.4 

158 2 •. 6 1.4 

1t•4. ~'' l'i.on 
Validities. of the pred!'tors against the aggregate ATC success 

criterion are provided in ;'~ble 8. Except for prior exper:.ence, 

these validfties are significate!. at the 1 percent or 5 percent levels. 

TABLE, B - · VALIDl.TIES OJ' Til!: WEIGI!Tt!l TEST 

BATTERY., PEQ AND OKT 1\GAINS'I' TH.E 

AGGREGATE 1\TC "SUCCESS" CRI.TERION !W SEX 

(*•p~.05; •-••p_f.Ol) 

Weighted Prior J:xperi- .ATC OCcupatiQnal 

Test B&tter:t ence !PE2l KnowleQse (OKT) 

~ N. r N r N r 

Ilion l3e6 ~ 'ffiT ~ 1308 ~ 

women 165 .19•• 229 -.04 154 .14··-

It should be noted that few of the 229 women in the prior experience 

sample had aviation-related experience. Consequently, the validity 

results. are due to lack of variance. 

Ccmparable statistical analyses were liB de between minority and nonminority groups and between non-minority and blacks as defined by 

OMB Circular A-46. The results with respect to the test battery, 

prior experience and the ATC Oc.cupational Knowledge Tes.t are 

provided in Tables 9, 10 and 11 respectively. 

29 
Nnn-Hinorities 

All Ninoritics 

Non-Ninorities 

Blacks 

Non-Hinorities 

All Minor-ities 

Non ... t-finorit ies 

Blacks 

Non-Minori~ies 

All Minorities 

Non-Minorities 

Blacks 

I'REDICTOR 

-·· ... -. -··· \,'pfght('d T~o.•st 

B.:lttcry ___ . 

N 

1323 247.7 

SD 

371 

---( t :._10_._30 _:1'.____._()1) 

1323 247.7 371 

145 207.4 407 --Tt-;;·i"z:-3o:P~0 

PREDICTOR 

Ptiar·Av-iat ion 

Exper ien_s_<:__ (_P_s_<ll_ 

N Hean SD 

2115 3.9 2.9 

...B_l __ ~-- 3.0 

__ -~< t=8_,__4_R_:_r-=---o_I) __ 

2115 3.9 2.9 

__!9_4 __ }c:J... _ 2 • 8 

(t=l0.27:p_.Ol) 

TABLE ll - ATC OCCUPATIONAL 

PREDICTOR 

ATC Occupational 

t:nowledge Test 

N Nean SD 

1247 76.5 12.8 

229 69.1 14.7 

(t=7.85:E- .01) 

1247 76.5 12.8 

134 66.9 15.2 

(t=8.12:p_ .01) 

CR !Tff: I o:: 

1

'SuC("('SS" . - . -· .. - :-.: ~le<~n sn 

1 308 'L ~ I . ~ 

~/!) 2.7 1.!; 

(t:o:5.0_9_:r. __ .0·1) 

1308 ].2 ].4 

144 2.4 ].) ------ -(,~,-6-.-3-1 , ;,~~:o1 >-- --

CR:TERJO~ --·-----. 

11Succcss" -~-~-- ---- . ·-- ----~-----

r\ . Hc~m SD 

2"097 3.2 ].4 

_3__1_!!_ __ ....2,_6 ____ __1~_4 

_____ (t_c]___,_i_S_=L-_o_l)_~-

2097 3.2 l.t~ 

193 2.4 1.3 ----,,-;::s--:39-, ~~~:myK_Nil~·'LE~cE s_MIPL__E 

CRTTERlON 

-----~ Aggregate ATC 

--~~~.!!..~~'!.·~· --- N Nean SD 

12J5 3.2 1.4 

...1J..?.. 2.7 1.4 

_j_t=4. 36..'L_:.Q!) __ 

1235 3.2 1.4 

135 2.5 ];) 

(t=8.12-:p-.01) 

I 

_1~ 

j 

I 

' I 

' 

i 

j 

j 

r 

' 

f 

I 
M:.nar!.ties· as a g~oup, and blacks as an element of the minority 

group, scored lower on each of the predictors and on the aggregate ATC success cr:ter:on. In each case. these ct:fferences were 

significant at the 1 percent level of confidence. 

Validity of each of the prect:.ctors W:.th the aggregate success 

cr!terion for mindr:t:es and non-m!nor:t:es are prov:ded :n 

Table ·12. 

Table 12 

~ty· 

l!lmrity 

Correlations f~r the weighted te1t batterv P!Q 

•cale and OKf ag&ln$t ~he aggregate crlterlon 

by mlnon.tv/no~~c;.':-,orlty status 

Wei8bted 

Te5t Battg: PEg OKT ..!L ..!- ..!!.... ..!- ..!-

1308 .203** 2097 .148"* 1235 .201"* 

243 .219"0 318 .247** 227 .272-* 

When blacks were analy2:ed separately fra!n all other minorities, 

the validity of prior experience and the Occupational Knowledge 

Test were sustained (r = .259 and .256, both at the 1 percent 

level of confidence). Val:!.d:.ty of the ~ighted test. battery was 

.120 wh:.ch did not reach the 5 percent level of confidence 

(p = .075). However, the difference in the .val!.dity for non-minorities and blacks was not statistically significant. 

Conclusions 

The results of this study supported the conclus!.on that several of the 

exper-'..mental pred!ctors were sufficiently vaEd and fair in pred:!.cting 

ATC success that they should be further developed for operational use 

in tbe selection of applicants for the air traffic control occupations. 

However, the lack of sufficient esc test scores fo.r the sample popu- lation and the restriction in range on those scores wh!ch.were ava!lable made it impractical to compare the validity of the current esc test 

battery with the validity of the experimental tests. In order to address · these questions, the Civil Serv:ce Commission, na.' the Office of 

Personnel Management (OPM), and FAA jointly directed efforts to obtain 

the necessary information fran a group of about 11,500 applicants for 

ATC werk· during the Fall of 1976 and Spring of 1977. 

31 
I 

STUDY OF ATe· JOB APPLICANTS 1975-1977 

Objectives. Historically, there i;2s ·been. little or no infol'!llation avaiiable on the character:!.stics of ind:!.vidua1s who apply for pos:t:ons :n the 

Air Traffic Contrbl occupation. Consequently, th:s study was developed 

to collect and analyze data on ATC applicants with 'respect to sex, education, prior aviation-related experience and the relationsh:p between 

these variables and scores on the esc test ,battery and expar:mental tests. 

One diret!t use of th:s information wa.s. to establish a bas:s for canparing 

the valid:!.ties of the esc test and the exper:mental tests by correcting 

for the restriction in range on esc test scores for approx:!.mately 2,000 

newly hired ATC trainees who attended the FAA· Academy during 1976-1977 and 

for whom esc test scores were available. 

~ffirn~~ffi~ea:~··~;'~rr'~nthe period November 1976 through Jarruary 1977, 1e opened the canpetitive ATC exam:.nat;,on 

in the FAA Fastern and Southern regions. These included the areas encompassed bY Regions II, III and rv· of the Standard Federal Regions. 

Approx!.mately' 11,500 applicants took the ATC Civil Service test during 

this period. Of this group,. about 7, 500 also completed the exper:mental 

tests and the Prior Expe.rience Questionnaire. While the SilJI!ple from the 

11,500 population could not be controlled, differences between the means, 

standard deviations and other statistics on the esc tests for the 4,000 

who did not complete the exper:mental tests or provide prior experience 

information were not statistically significant from those ~no did. 

Predictors. !n addition to the five-part esc test battery, tw.o experimental ·tests, MCAT 706A and the Directional Headings Test, we~e administered 

to the appl~cant group. These test forms were the same versicns as used 

in the 1977 study conducted by Education and Public Affairs «~;,.,pt••r 1 .,,-

reference 14). The same l'rinr Experience Qucstionnai!'c (1.5) t..·as alsn given. 

All tests were admi.nistered by esc exam:.ners aS' part of the normal canpetitive testing procedures. Applicants were infonned that the experimental tests would have no bearing on their eligibility status and the 

experimental predictors were administered after completion of the competitive CSC test. The Directional Headings Test and the Prior Experience 

Questionnaire had to be mnually scored and converted to ADP records. 

This wa-s dane by FAA under contract. Detenn!natians on whether applicants were men or women were coded on the Prior Experience Questionnaire 

based en the applicants name. At this point, Federal regulations prohibitsct e>l)taining any ethnic or minority group infonnation from the 

applit'ar:t:.~. 

32 
Criteria. Since this study enc<Xl!Passed ATC applicants, it was not feasible 

to establish operational criterion measures of validity. However, a 

variety of statistical analyses wer·e mde based on men and women a,Jpli-· 

cants and these who passed. the esc test in contrast to those who 

failed. 

Analytical.Metho<Joiogy. The data obtained on this ATC applicant group 

were analyzed lf:.th respect to frequency distributions, means, standard 

deviations, analysis of variance and intercorrelation of predic:.Ors. 

Results. Table 13 snows the means and standard deviations for the total 

applicant group, the pass group (those who scored above 209 on the esc 

test), and the fail group by men and women applicants.. · 

T.ULE ll 

SD.ECflON TEST PERFORMANCES-. Al'PL!CANT CROUPS 

CSC TESTS 

TEST 'l'OI'AL GRClJP PASS GIOJP FAIL O!OJP 

m.. (W) ~ @ (W) MFA.~ @ (lory ~ @ 

csc~~t. 

Tota-l (7412) 39.6& 9.6 (3960) 44.80 6.8 (3722) 34.61 9. 3 

Men (5120) 39.63 9.5 (2980) 44. )}U 6.9 (2740) 34.32-*- 9.2 

WOlllf'n (1607) J9.ft-3 9.7 (66)) 4~. 91 6.6 (94'4) )5.56 9.3 

CSC-51 

Total (7lol2) 26.65 6.9 (3960) 30.74 3.8 (3722) 22.60 6.4 

M•n (5720} 27 .t.O 6.4 (2980) 31.05 3.6 (2740) 23;4 3 6.4 

WolDen (1607) 23.98** 7.0 (663) 29.37*"' 4.4 (944) 20.20U 6.0 

CSC-135* 

Total (7412) 21.49 8.6 (3960) 27.00 5.1 (3722) 16.03 7.8 

Men (57,20) 21.88 8.4 (2980) 26.97 5.1 (271.0) 16.35 7.8 

Women (1607)" 20.1)** 9.0 (663) 27.11 5.0 (944) 15.23** 7.9 

C:SC-157* 

Total (7412) 29.29 10.3 (3960) 36.65 6.0 (3722) 21.99 8.3 

Men (5720) 29.1o7 10.2 (2980) 36.40** 6.0 (2740) 21.93 8.2 

...... (1607) 28.62** 10.9 (663) 37.15 5.8 (94<) 22.21 8.0 

CSC-540 

total (7'12) 18.98 13.1 (3960) )7.70 10.1 (3722) 20.34 9.6 

Men (5720) 29.78 U.8 (2980) 37.84 9.9 (2740) 21.03 9.5 

Wome-n (1607) 26.11** lJ,7 (663) 37.00 10.8 (944) 18.46** 9.8. 

CSC TOT¥. 

Total (74!2) 202.03 52.3 (3960) 21.4 •• 26 23.2 (3722) 160.16 37.3 

Men (5720) 205.0l. 50.9 (2980) 244.23 22.0 (2740) 162.42 36.8 

Women (1607) 191.20** 55.2' (663) 244.26 24.0 (9<4) 154.08•'* 37.8 

EXP[kiKE!!IAi TE~TS · 

l!CAT (CSC,510) 

Tot .I (6812) 2S.76 9.1 (3340) :n.&s 7.2 ()481) 20.12 0.9 

Men (5241) 26.57 9.0 (2688) 32.14 7.0 (2552) 10. i2 7.6 

WOmen (1498) 22_.95** 8.7 (607) 29.53 7.2•• (8!11) 18.46•• 6.5 

DIR HEADlN.G 

Total (7073) ,,.99 22.6 (3583) 59.,3 17.3 (3489) 33.68 19-' 

Men (51,62). 48.87 21.9 (2899) (i0.63 16.6 (2562) 35.56 19.5 

U0111en (1526) •o.n** 2).3- (6)7} 56.58 19.4** (889) 28.41*" 18 . .1 

•esc 135- (followtn, oral directions) and esc 157 (Abstract Reaso~ing) tests are doubled 

weighted in computla& the esc. teat total acore 

**Si«niftcant r.t the- 1 perc-ent hvel of cortJhlencf!' •. 

33 

---- -· -----.---··------- ·----·-···· J 
Frixa Table 13 !t is ev!dent that :!.ri. totaL wome.n score socewhat belO)I 

men on mst of the esc tests and en the two experimental tes~. These 

differences, wh!le statistically significant, are not statistically 

important due· to the large sample s!ze. For exa.':!ple, furtr.er analysis shw:~ that sex accounts for only ., percent of the var!ance !n the esc 

total test score; j·percent :!.n the MCAT; and 2 percent in the Directional 

H~dings tctal test scores. In other ·words, most of the ·.-ar!ance in these 

three pred!ctors !s due to factors other than sex. 

In ter:ns of the "pass" group, the mean esc test score for men and women 

is essentially the same. Consequently, d!scount!ng veterans preference 

and additional credit allowed for aviation-related experience, women who 

pass the esc test should have about the sa.~·opportun!ty to be selected 

for the ATC occupation as men. 

Frequency distribution for each of the five esc tests and the two experi-' 

mental tes.ts were also examined. Two of the esc tests, esc 135 and esc 51, 

show a cark"!d negat!ve skew (-1.30 and -1.80) ·respect!vely) and consequently, provide 1!-ttle d!fferent!ation between the appEcants. 

FIGURE 2 

CSC·- FOLLOWING ORAL DIRECTIONS 

... 

135 (DOUBLE t<EIGHT£0) 

N~RN= 2/.S 

.S.D.= s.h 

·~~~~~TrrMTT~~rM~rM~ ' 10 .. 10 

SC:OIUS 

.. )0 •• 

,.., 

... 

34 

CSC - SPATIAL RELATIONS 

51 

MlfRA/-: 26.7 

6.Ll= 6. 7 
Frequenc:y distribu_tion graphs of all of the esc and exper~en~l tests are 

provided in reference ~~LlTable 14 provides descr:!.pt.ive information on education and experience levels 

for lll!n and women :!.n the applicant gro<~p. Av:!.ation-related exper:!.ence data 

was obtained r~oo the. Prio.- Experience Questionnaire (PEQJ. Th:!.s identified 

the·spe<;ific ""rk or skill el~rtts which, based on the present esc Rat:!.ng, 

allows granting extra cred'.t point" to those appl:O.cants who pas'! the esc tes·t. 

T-A!-LE 14 

De&cript!v~ Statiatice-Applicant Group 

EDUCATIOl' LE\'EL TOTAL APPLICk~TS 

.!1~ 

Btsh school or lu·• 1225 (2l%) 

Leaa than J years college 2352 (U.%) 

3 or ...,.,.. years :olleF.e 2126 .!.llll 

Total 5703 (100%) 

.Women· w.ith: 

B.lgh achOol orless 355 (22'%) 

Lesa than 3 years colle~e 618 (39%) 

3 or ~re ye&rs coll~ge _lli (39%) 

Total 1601 (lOO:) 

txPEP:!I"SC! t£:\'EL 

~ith 

No aviation-related experience 4393 (77%) 

Aviation-related eiperience ill1 Jml 

Total 5120 (100%) 

WC'IIIrn with: 

No aviation-related experience 1483 (92%) 

Av!atioa-rela~ed experience _,ill .u!L 

Total 1607 (100%) 

EDUCATION AND EXPERIENCE 

With aviation-r~lated ·~~rtence ~ ~ 

Bish·•cbool OT 1••• 21% 23% 

Lea• thaD 3 yea~• college 44% 39% 

)·or .ore yeara college 35% 38% 

Wo aviation-related !!Eertence 

atah acbool or leaa 22% 22% 

Leaa thaa 3 yean eollege 40% 39% 

3 or .ore year• · collese· 38% 39% 

About 22 percent· of the total applicants were """"'n. However, as Table 14 

ahows t,11e distribUtion of education level for men and women is approximately 

the siu.e. It also shows that approximately 78 percent of both the aien and 

women tia,ve s01118 education beyond high school. 

35 
With respect to av:aEon~related exper!ence, 2J percent of the tOOn as canpared to only 8 percent of the women, identified experience which could 

result in granting extra credit points for se1ect!on eEg!b!Hty, When 

education. and experience are canbined, the distribution of \':ducation 

level remains relatively consistent for both men and women and fo~ those 

with or w!thout av!at!on~related experience. 

From this informa t!on it is evident the.·; the level of edu(!ation for men 

and women is essentially·the sat!!e and d~s not serve to differentiate 

between the app~:.cants based on their sex. The distr:.bution of aviation~ 

related experience between men and women does differentiate between the 

applicants w! th 23 percent of the men potentially ~eing eligible for 

extra credit in contrast to only 8 percent of the women. 

Next, the ATC applicant sample was ~lyied in terms of passing or failing 

the esc test in relation to sex, education and experience. Table 15 

summarized the results of th!s analysis. 

TABLE 1$ 

ATC Applicant, Sample 

High School (lr leu 

Leu than 3 years co·ll~ge 

3 or more yea~a cOllege 

Total 

EXPERIENCE LML 

No aviation-related experience 

Aviation-related experience 

Total 

EDUCATION ' EXPERIENCE 

Hish achool or le11a 

Leaa than 3 yeata colles• 

3 or •ore yeara colle8~ 

Total 

No Av{atl~ e!pertrcEe vtth: 

H1Jh •ehool 0~ leaa 

Le .. than. 3 yeare collese 

3 oF . .oT• :ru.r• coUeae 

Total 

36 

TOTo\!. 

APPLJCA..>JTS 

5720 (78%) 

]j2"f 

1607 

(100%) 

~ 

1581 (22%) 

2973 C'O% 

2757 Jml 

1311 (lOOt) 

5876 (80%) 

14Sl ill!2 

7327 (100%) 

308• (21%) 

632 (44%) 

...w ill!)_ 

10.48 (100%) 

1273 (22%) 

2341 (40%) 

~.illll 5 (100%) 

CSC TEST STATUS 

~ FAlL 

52% ,., 

41% 59% 

<50%>• (SV%)* 

32% ~6% 

49% 51% 

6)% >9% 

(50%)11 (Th%.}• 

45% 55% 

68% 32% 

(50%)* (SO%)"' 

52% 48% 

67% 33% 

(~i>· 21% 

(32%)• 

27% 13% 

44% 56% 

57% 43% 

(4)1)* .<cssi}• 

.... \:1 

" ,< 

i.,' 

:l 
In total. at-out 50 percent ot' •.he appEcants passed the esc test. Howev<lr, 

proportionally oore ren (52 perce;:•.) passed than wooen (41 percent). 

When specific education levels are ·exam:ned, it is evident tha.t 

those applicants with the most education have a significantly better 

c.'lance of passing the test. Applicants with 3 or oore years of college 

passed at about twice the rate of those who did not go beyond high school. 

The same rel~tionship holds also for experience. Applicants with aviationrelated experience also pass the esc test at about twice the rate of those 

without experience. 

When pass or fail rates are examined with education and experience combined, 

it is evident that: (1) level of education is essentially proportionally 

distributed between the experienced and non-exper!a>ced groups (Table 14); 

and (2) experience increased the pass rate in each of the education groups. 

Applicants With no college or experience pass at a 27 percent rate. Those 

with"3 or more years of college and aviation-related experience pass at a 

79 percent rate. 

TAllLE 16 ---~---

CSC '11:$T PASS RA'I'l: 

110 lf.ITH 

El>UCJI.TION LEVEL EXPERIENCE EXPERIENCE. ' INCR<:ASE 

Hiqh School or less 2'1\ 52\ +25\ 

Leas than 3 years college 44\ 67\ +2:n 

3 or 1110re years college sn '~' +22\ 

Two-way analyses of variance were c.arried out for each esc and experimental 

test• For this purpose the '!experience group"was further classified into 

subgroups since the esc Rating Guide provides different levels of extra 

credit based on specific types of experience, The·"key" to the experience 

and education subgroups are as follows: 

Av!at~on-Related Experience 

0-1 No aviation-related experience 

2-2 Comu~ication experience only 

3-3 Non pilot with IFR or· Air Defense Coamand (ADC) experience 

4-4 Non pilot with Vl'R Ate experience 

5-5 Pilot with IFR or ADC experience 

6-6 Pilot experience .only· 

7-7 Pilot with VF'R ATC experience 

37 

i. 

I, 

i 

' 
Education 

0-1 High school or les3 

2-2 Less than 3 years of college 

3-3 3 or :nore years of college 

/ 

Table 1'7 provides the results of the analysis variance for the esc total 

test score.' The lowest mean score ( 173.10) is tor the "No f.x;:>erience and 

No College " Group; the highest mean score (241.00) is for the "Pilot with 

VFR Experience and 3 or oore years or. College" Group. The analysis of 

variance shows that ex;:>erience (row) and education (column)each is statistically significant (p 5:.01 J though there is no interaction between experience and education. 

further analysi.s, however, shows they are not statistically important for 

the total group since only l pe:'Cent of the variance in esc total score was 

accounted for by either variable. 

TABLE 17 

ATC TESTS VS •. EDUCA:TION JJ.'D EXPEIUENCE CROUPS 

TOTAL CSC ·Ttst $<X>lll (CSS TOTAL \IT.) 

N H.ATR,lX OF CASES t.'Irlt NO!<O-HISSJON DATA ON" All 3 -VARIABLES 

ED GROt"P_:_G~:'lf:S oo;.~ £XI'. CltOt'P C(1[S ACROSS 

0-1 1:1. l-3 ~ B ~ H 

o-1 - 1239 )4 140 28 19 61 ~-:. 

2-2 - 2266 75 216 47 56 "" IJJ 

3-3 - 2179 70 62 25 )I 230 160 

SU!' 5684 179 418 100 106 477 347 

fiA;"~s or \'ARIABLIJ esc TOTAL l.'T 

ED CROl'P t.OES fl('\..10 !.XP. CROl"P CO~S ACR~ 

0-1 l:l 1:1 ~ ~5 6-6 

D-1 - 173.10 179. Sl 210.89 20!.96 1;8.68 191. 07 

2-2 - 197.lo0 20l.o.79 218.55 2)).2) 2:i0.61 221.52 

3-3- 209.11 221.59 221,..98 222.72 246.10 216.17 

MEAN 196.59 206.56 216.94 221.85 224.1) 224.31 

liUHMAR'!' or A.~ 'iSis or VARtA.'-C£ I 

Soul'ce 

TO! AI. _,_ 

15/H 

2973 

275-7 

7311 

7-7 

211.56 

225.19 

241.00 

230.36 

.... 2488e~.2? 2 124444.63 SI.2S o.ooo 

c.1 1S2242.89 6 25373.82 10.45 o-.. ooo 

a.n .. c 44120.11" 12 3693.35 1.52 0.109 

VlTBIN ctu. 177019f6.SO 7290 2428.25 

~ 

179.1.!8 

202.83 

214.4) 

202.16 

However, when the esc total score was cate~orized by "pass" or "fail • 

groups, education and experience together accounted for nearly 10 percent 

ot the variance in pas~/fail status; so taken together, the effects or 

edllcation and experience are important. This same relationship ap~lies to the two experimental tests as well. 

38 

I 

I 

'~ 

:I 

l 
' 

'lh.:sc d.:lt.l. sfo...·: (1) tt.,;t .:tpplic"1nts \~·itt: a\·iation-rclatc-d i:x;x·ric-na:- have u. hi~hcr 

Frobability of pac;sin-~ L~· c;c test ((;8 pc•rcent as C0'1'a.~ tc 45 pe:rCX!Tlt fer t.""lcsc 

""'it:h:>ut c:q:crience}; and, (2) that ft?W Y.Clf"'Cn ha\'t' a\'iJ.tion,..n .. latcd e;.q"<'Ti(;!ntxo. Th~"sc 

0.0 findings taken to.~cther, ~--\'('r, are not suf!:icicnt to exrlair. the 1~-.·:r pass 

rat£' !or- ~n (41 f-""rn ... nt) ·as a:Jn'l'arC'd to rrcn (52 f.'<'n::ent). i..,~lyina ~J'ese rrol::>abilities 

to tl'le cxpericna:> U.'ld nc c>:per.icnC(' c~rours. it \lroC!uld b:> expected that 7·;: w::rr::t would 

ha\'e passc>d; cnl~' f·G3 did r,uss. For rr.:>n, 2,879 n:-r'l ...ould be e:-:p.."'r-tc ..: t·· jass; .:_•,980 

actually did pass. Cbn..e;oc:umtly, sir,~ly controlliw !or the fact ~.t~ 1t m:ol, ha\\. ;f)re 

a\'iatiorrrelated C'Xf~ricnet.• docs not explain "-'hY ¥.O~n soon:' :ONer ... ::- ro:;t of tllo::> 

tests. 

Interoorrelation of e-ach of t.hf· tests, education "'""' e~tt"rienc€ .:J(> pn:'l\:idc.C lr:. ':'able lo. 

1.\JL.[ 18 

ATC TtST COkiU:l..AllOIOS -- TOTAL SAMPLE 

fOSlTION um l 2 l ' • 7 • ' 10 l1 " n " " " l7 l8 

T T TS Tl Tl c esc TS 24.S Sl.S 40.S S7.S J5.S SC. T TOTA TS TS lQ.T DI!.T tO.L P£Q EO.C XI' .c lH 

""' COo< cou CORE COKE OHL L.\."T lO~C JO.A OTAI. Dll.A DH.I OTAL [\'[L Ace ltCL1' lOL'P C:PX 

tZ4 srou 10 " •• " " TSl SCORE 

T540 SC'OR.!. ., " " " •• 80 " " '1157 SCA1R.! 

Tll~ SCOltE 6J " " esc t~''A.L .. esc tor AJ. \."r. " " " THO C " " " " " TSIO A " oO " " 77 T510 TOTIL " " ., ., 77 DII.A " " DH.JI " " " " " " " t»>TO!AL •• " " " " 11 £0. L£\i:t 27 ll .. " ,, ,, EO. tHLi. 6 " " " 16 l7 PEQ AC(; 26 l2 " " " 24 " tD.CiU>t:P 10 2l l4 l7 " 20 l6 " txP. CROt.'P ll " ' " l4 " , " w. cPx 

ms: 

(fl) ~C total 1., the corula,lt-n of the um.· ... t~ht.,.d esc t"'&t ·.c:orr• 

(1) CSC Total VI" ta th1t ton-.lation of th.- ~;et~hted esc t-eat ac<'rn 

(8} T510 C ta the corr•lat1on of HeAT "atrrTaft c,..,nfltct" scortoa 

(9} TSIO A ta thr cor.rehtton of "CAT "aptttuile" scores 

" .. " ,, 

" ,, ,, II 

• " " •• " " " " " " " 20 77 " " 24 

" .. .. " " " " " " " " " 

(11,) Ed Level h thl! c:ornh.tlon of indtvfd11al IA<ela e>f t'dur•tl<m (OJ.l)) See -nf (Jio) c•l 

(IS} PIQ. ACC h the correl•tiC'n of aviat!on-ulated eiCpt'rtenu ltalu derived dptrlcally. See ref (12) Chapter 4 f()r the ~Wthodol"~Y for developln« the fl.taln 

(16) td Croup 1a thto ctorrehtion "'the three t-4ucatton gr(lupa; (l) hltth 1d\00l or le .. ; (2) Ina 

than ) yean of collPge; (J) lor .au yun of c-college. 

(U} Ixp Croup is the correhtton of the aevton aviation-related eJCptortence croup• 

(18) E.xp CP. Jl ts the cortehtlon of the aroup. vtth avlatlo.n•relaud toxpertenc:e {eJC-c:l>.Mltaathto no 

Hl'er1enc:e applt.:aat•l 

• ,. 10 

.. " 2l 

lJ " " " " " " " 16 

" " 20 " 17 ,_ ,, l1 " " 18 , " z.:. 18 3l " " " 

'lbese oorrelation. values ·were uSed to correct for restrictions in ranoe on test 

so;>res of individuals hired as ATC trairees who attenred the FAA Ac~ .. my during 1976, 

1977- and 1978. 

Se\-cral l!llltiplo regression analyses ~re carried out. using esc test .. pa.ss .. G: ·..:ail" 

as a criterion for t.'lc appliCant oroup. Table 19 shc:Jt,.IS the results for the present esc 

test battery, t.hc -ordt?r in which the individUal tests entered the annlysis and the-ir. 

oontribution to the R2 whidl rreasures. t.he anount of variance aca:)\mted for. by: each test.. 

39 

II 

" ' " 7 

" " , 

" 10 

2.:. 

·-····'" .· ... ·~·~. 

\ 
TABL~ 19 

&TC lrtG.ItUSIOM. AIIU.L'I'SII TOTAL l..,.ll 

filiAL '"*"'"IIY 01 ll'-AUSIOII ON PVlNOlfiiT UIII-Aal esc. tot., 

... LTUU I o.,CtH llll 1 I!' S.OUAUO. .... 93 $TO. ruo• 01 an. o.nu C0..S.T&IIIT O.OOJJ 

loiiULYSIS Or ''* or .... , w.t.•Uotwtl ..... •• sou.uu ..., .. , ..,,. lt:~US$10. uwu •• UO),IDG 2 .. 0.QD nu.tu •• IUJOl,ioil,. uo.. ....... . .... I .... .,., .,., CMAHGl W&llUIU I • ., ... IITio. n••~o 

, -· ... ~ ... 'UTUl ' ... • • 1111- •s.o 011'(11[0 ••• ..... STAtiO. , INTUIO co•. COl~ • - • • ... t • IJ40WS cor•- •• cor•· to • •• OIU1101 IJCU.,.t ••• . .... t'IIIIAL I f'JCU"t Dl&.tll DILUU Dll'. ..... • 

I 

• 

I O.?tt o.sot 0.101 T1Sf.SCO.t: o.o••• o .. aoo, 0.)041 .... _,., nn.~ .. , .• 0.1,.:1 ··-"'' • • 0 197 0.042 T~O:S.tOIIt: O.OOM D.ooo. O.JI:JO tH.4SO tiii,U.t O,tlt!l o.aue ._, ... • • 0.102 ........ 

•-•» O.Ol!J 

O.Ot4 tqs.scocr 

TS.t. SCOtti o.OtU 

o.ootJ O.ODqe 

o.aoot o. 0.2oJ4 •••. 01t IU.ttO D. IOU O.:tHI , .. ,. 20T,JU JaJ.c:n o.•no 0.1110 I I ··- ..... ··- TU.JU«t ··- •. 0001 ..... y U1.Ut tii.Ut ... ,., o.tno 

Table 20 provides the results of the two experimental test~ in predicting 

the CSC pass or fail status. The difference between the R for the'Je tests 

(45 percent) compared to the esc tests (~s percent) indicate, that they 

differ in some degree from the esc tests in what they measure. 

In the case of the experimental tests, test 510 A (MCAT Aptitude) whi.ch· 

measures aptitud~ ~ similar to the present. esc test entered the regression 

first and accounts for 40 percent of the total variance. 

TABLE 20 

aft IIGtllU10ft UI&LYIII tOTAL &MIJtl.l 

NUL ......... Gf UGI:llll• Gill DOti&Oitlt VUI&kl ac.tetM 

IIA1tJtLI • .,L t I •ou~•lO 

IJD. t-.. 06 o• Ut. COiiJ.Yalll 

AULTltl O"' .... , .... ,, •• •• ltGlr"UI6lil 

IIU.Du&L ·-· 

• ... -· .,., 

' ••• • I 

I 

- ... • '" • I o.u• ..... I • O.MJ .... ,. a • ..... ··-

O.M:?S 

o.•·cs& 

0.172« 

. .... " 

..... 

IOUUUen.?-14 

ton.au 

--

•• • 10-

•••• ..... 

.. .,. 

.... aou••' lft.:r:JI D.-tJ• 

YPUILI 

(lt1U(D 

t• ''SHOWS 

HLUUl· 

1110.& 

D11.10U&. t1to:c 

• -·· eat to l.lvU. ....... 11 o: 

I. STilteD 

• •• tuo• ..... .. fiC:Il .. , • 

o.oJa O.D;UJ o~oon O.OOOJ •...• , ...... 

40 

HTA, •••n ·-· auitta.t .... , •• l 

lta.o . , . ltffUt:O· COl. c~. J• 

COlP· io •• .... ......... "UCll'!IT a-~.ua Ml.l.tlD n•. '"" o.ntt , ........ .. ,,.~. •. ,11•• l.nft 

o.n11r """·..., a.n·.JOt ...... .. .,... 

o.sno Q.tM-. a ...... ....... ...... 
Men -n 'fot.al 

110 Exp. 

STUI)Y OF NEW APPOINTEES TO THE ATC OCCUPATION, 1976-1978 

Objective. This study was initiated by the FAA Civil Aeromedical Institute (c~vJ) in July 1976 to evaluate the present esc Test Battery and exper~ental predictors in relation to the success of newly hired A7S 

trainees in passing In:t:al Qual:f!cat:on tra:n:ng for the En Route 

and Tenninal ATC opt:ons at the FAA Academy. Upon entry on duty, r.e·.; ATC 

trainees are g::;eri a 2-week general orientation at their ATC fgcolity (or 

regional headquarters). They then enter Inotial Qualification tra:n:ng at 

the FAA Academy in Oklahoma City for the ATC option to which ass:gned. On 

their first day at the Academy, they are given a series of exper~ental tests 

and questionnaires. Their participation in the test program is voluntary. 

The experience has been that most all of the students volunteered. 

Extensive efforts were nade to also obtain the individuals' test 

scores on the five-part esc battery and their total earned ratongs wh:ch 

include additional credit for veterans preference and aviation-related 

experience as provided by the esc Rating. Guide. Th:3 information was 

then correlated. with their subsequent training status to evaluate the 

esc and experimental tests which were most pred:ct:ve of training 

success. 

~J.e Descript:'.on. Dur!ng the period July 1976 through Apr:.l 1978, · ~8 students entered Tenninal and En route ATC tra!ning at the FAA ' 

Academy. Selected characteristi'cs of th:s total group are prov:cted :n 

Table 21. 

TABLE 21 ----·~·-

SELEC1}0N CIIA~C:IJ'_~l ST_J CS-1'(11"-~~ROU P 

~ FAIL WI';'!!DRMf 

M (\) '111 (\) !!c (\) 

2034 <m> 4J6 cffi> 110 (4\) 

428 272 (64\) 128 (30\) 28' (6%) 

3oOi 23o6 (77\) 564 "'ffiT I3e (4\l 

834 579 (69\) 206 (24\) 58· (7\) 

AYi&tion-Related Exp. 19J6 157.9 (82\) 296 (15\) 61 (3\) 

.!ill!. 62 ~ 19 (8\) 

UDkDown 22<) 148 

'fot.al 3008 23o6 564 138 

41 
Ilea 

WQmen 

Ti>Ul 

·No Exp. 

In order to evaluate the esc and exper:!.mental predictors, it was necessary 

to have canplete !nfonnat:!.on on the. students. The 138 stCJdents h'hp ...=.tt·,_ 

drew did not have lat>oratory cr!teripn· scores and were el!.minated from the 

sample. S:!.m!larly, the stu·:ients for h'hom pr:.or ei<per:.ence. information 

was missing were also vxcludl'J. An addit.:.onal 821.; students h'ere excluded 

beeause they did not have one or more test scores, e!ther from the esc 

test battery taken prior to their employment with FAA or were students h'ho 

did not volunteer to take the exper:!.mental tests. 

The final sainple used for the an?-lys:.s after these exclusions consisted of 

1,827 students distributed as shown in Table 22. 

T,\!ILE 22 -----.----

'l'OTAL N N ~ (\) 

1587 l3l4 (83\) 

240 .162 (68\) 

1827 1476 (8l\) 

558 409 c1n1 

FAIL 

N (\) 

273 {i7\) 

78 . .!.3~'1. 351 (19\) 

149 (27\) 

\ OF 

ORIGINAL 

POPULATIOI< 

61.5\ 

56.1% 

6o77\ 

Aviation-Related Exp. 1269 1067 

iB27 r:rn 

(84\) 202' 

ill 

(16\) 

66.2\ .. 

66.5\ 

The failure rate for this sample (19.2 percent) was essentially the same 

as for the total population (19•7 percent) after withdrawals were excluded. 

Predictors. The tests used in this study were: 

- Civil Service Test Battery 

(esc 24; esc 51; esc 135; esc 157; and, esc 540) 

- Directional Headings Test (DHT) Total Score 

DHT Part A 

DHT Part B 

- Multiplex Controller Aptitude Test (MCAT)· Total Score 

MCAT Aptitude (A) 

MCAT ConfHcts(C) 

- Dial Reading (DL RD) 

-. ATC OcOllpat'ional Knowledge· '!'est (OKT) 

42 

' I 

I 

II 

i' 

I 

I 

I 

I .d 

'I 

l 

l 

~I 

I 

.I 
The Directional Heading Test was the same version used in the 1977 study 

conducted by. EPA and '-lhich was adm;.nistered to the 1, 500 A7C applicants in 1976-1977. 

The MCAT consisted of several different versions (606 A; 606 B;· 706 A; 

706 B; 607; 707) which varied in the n~ber· and mix of questions 

(aptitude/conflict) and in the length of tjme allowed for the test. 

The OKT was form i01B which was essentially the same form utilized by EPA 

in their 1977 study. The Dial Reading Test was also the same test used by 

El'A. 

Table 23 provides the means and standard deviations for the sample of 

1,827 ATC tra!nees at the FAA Academy on the various predictors and compares 

them to results obtained for the 1976-1977 ATC applicants group as well as 

with the ATC sample group included !n the 1977 ATC Selection Study conducted 

by EPA where comparable statistics were obtained. 

:lf\.I~I~UJ. 

~~t!:'!!l_t:RA'!J~'E _Il_~S~K Tl~J 1 vr_.-~~L\'I~.L~l~l_<_:.s_ ·-

1975~77 ATC APPLICANTS 1977 1976-78 ATC 

esc TOTAL esc PASS EPA STUDY TAAINt:ES 

(N•7412-682l) (N•3690-3340) !N"'B23-1229) (N-1827) 

ru:!: ~ so ~· ~ MEAN ~ HEM .2£ 

CSC-24 39.66 9.6 44.1!10 6.8 m 47.!)7. 6.7 

csc-s1 26.65- 6.7 30.74 3.8 Nll 31.90 3.3 

CSC-540 21!.98 13.1 37.70 10.1 Nll 42.88 9.7 

CSC-157 29.19 10.3 36.65 6.0 Nll 38.17 6.4 

CSC-135 21.50 8.6 27.00 5.1 Nll 29.31 4.2 

ltCAT (A) 16.59 5.7 20.27 4.1 HC 23.14 4.$ 

JICAT (C) 9.17 4.2 11.38 3.9 NC ~5.74 4.1 

JICAT TOT 22.76 9.1 31.65 7.2 NC 38:e7 7.~ 

DHT A 24.20 12.0 30.69 9.8 Nll 32.68 9.0· 

DHT B 22.80 11.7 29.25 8.9 31.9 6.9 31.63 9.0 

DHT TOT 47.00 22.6 59.93 17.3 IIA 64.34 17.0 

DL•RD IIA IIA 39.7 9.4 40.97 9.2 

010' Nll NA 74.9 13.6 66.96 16.3 

43 
(Table 23 Continued) 

r\A - Statistics not available since thl' tests \,.'etc eittu•r not 

administered to these groups or could not be obtailwd frnm 

availabl£' records.. In the case of DHT, for the· l 977 El'A 

Study only ParL; B ,,,.as scored and usud in the analysis. 

~C- Statistics not comparable. In th£\ 1977 EPA Study, t\·.'0 ft.lrns 

of !-!CAT were a.thninistercd to the> lest sample {706 ,\, 606 A). 

In the analysis- all ,.apt.itudeH questions fr.om both fotms "'crt~ 

combined for a total aptitude score (me.:Hl -= 42.8; SD-6.2) .:ts 

were all "conflict" qu,estions (mean-= 29.4; SD-5.6'). 

Criterion Measure. in th~s analysis, the ATC laboratory average score was 

used as the criterion- measure of ATC training success. Selection of this 

criterion was based on several conSiderat-ions. First, prior stuJics 

demons C-rated tha-t these scores w~re: the most re1 iable predictors of suhst~·­

quent suc.ces·s as an Air Traffic Controllvr. SecPnd, the labc.Hatnry trainin~ 

phase consists of a seri-es of opera-tional air t·raffic control pndJl ('ms in 

which students must deMonstratl"' their ability to appJ y the acaJc>m.i.c knowledge and skills acquired in training. Finally, this phase is ~._·ondu('tt.~d on 

a pass/fail basis~ The laboratory score constitutes 65 p('r<:('ll[ of the 

students total s.c:::orc.in the laboratory pha-sC' of tr.:dni11g. Alrl()st all of 

o-f the students who fail Initial Qualiftcatlons trnining do .so in the 

laboratory training phase. lt should be point<.•d out that during the pcri-t,d 

from July 1976 through April 1978 sevcrotl ct1.1ngcs t,•en.~ madP in ~he labDr.:Hc•t·:-· 

training phase l.•hich aff'ecteC the ·pa.ss/fai I r~tio of .~·,tmJent.s. In .July 197& 

the weight giV"en to· the laborat,iry average was lncrt~.l.sed from 35 percC'nt t'' 6) 

percent. In September 1976 the number of graded Jab prnb]c-m~ "'as incr<.'a.sc-d 

from four to six; and in May 1977 "procedural errors" were incorporated in 

the grading for failure to handle aircraft in a timely manner during th"' 

problem exercise. In order to accomodatc these chan~es and the differC'nn~s 

between the Enroute and Teminal Iaborator·y procedures, the 1 ahoratory 

.1verages were converted to st:andard scores and data ftom the two ATC ,)pt i0ns 

combined. The standardized laboratory averag,e criterion is lahel£\-d "ZLAB." 

The reliability of the laborat-ory average for Enrout~ tniinim~ is .80 

(2,223 students) and for the·Terminal option .81 (1 ,982 stud~..~nts). 

A detailed discussion of tl\c ATC Initial QUalifications training progr:tm 

and follow-On training at the ATC facility level for the Terminal and 

En route options are provided in references 23, 24 • .and 25. 

44 

.. 
Analytical Methodology. The first step in the analys:.s was the calculation 

ol' descriptive statistics. The remaining analyses presented several unique 

problems. 

Several di·fferent experimental forms of the MCAT were ecip~oyed :in the 

testing at the FAA Academy and· the order of administration varied for 

each form. ~nsequently, it was necessary to standardize the scores for 

these tests. Since MCAT 706 A was also administered to 6,821 ATC applicants by OPM in 1976-1977, the .scores on. the various forms of MCAT 

used by C~~ were standaraized by· linear conversionz using the same metric 

as MCAT 706 A. The order effect was handled by using the scores fran MCAT 

706 A given first. References 19 and 20 provide a more detailed qiscussion on the methodology used for these conversions. 

The second problem involved in evalu~ting the CSC and experimental tests 

was the restriction in range effect (21) since criterion (ZLAB) infonnation 

was available only on those individuals who were h:.red. This results in a 

spurious low correlation of esc test scores with the criterion. In order 

to adjust the restricted correlations so they would reflect the relat:.cinship 

between the tests and the criterion for the appl:O.cant group, the correlations· were corrected for their restriction in range. Usual methods for 

this correction in· the. three variables case, assume .that unrestr:O.C'ted information is available only on the variable used for selection or the third 

incidental variab~e but not both. With the infonnation deriv~ from the 

1976-1977 ATC applicant population, unrestricted data were ava:.lable on 

both variables. A modified procedure was developed to mke use of all of 

the available information in correcting for range restriction. A full 

discussion of the procedure and the derivation of equations is prov:O.ded in 

reference 22. 

The unrestricted correlation matrix from the total 1976-1977 ATC applicant 

group used for correcting for range restriction is provided in Table 24. 

The means, standard deviations and sample sizes (N's) were provided in 

Table 23 except for the Ea·rned .Rating. 

hl>1ello. unrestricted Correlation ~trix fro. the CSC 

.1 z ) • 5 g 10 11 12 lJ I• 19 

T T T T I ,. ,. T EAR 

VNUAIILL 24. 51. S.C. 157. ll5. 510 SlD 510. OH. H[D. 

SCORE SCORE SCORE S:ORE SCORE ,A .c TOTAL OH.A lli.B TOTAL RATE 

1 rz~t .. Score ., ., 66 

l rSl.Soore Slo. 57 75 

) TSr.o. Seore 06 S6 59 76 

• US? .. Seore 59 56 57 65 

5 TUS.Seore 5o' 56 " 17 

' TSIO.A 60 61 69 77 

10 TSIO.C "" •z 55 57 60 

11 TsiO. TOTAL 53 51 67 70 76 

12 DH.A •r. Slo 82 ~) 66 

u DH.B ., Slo •s •• I• OH • .TOIAL •• 57 7Z 

" tAAI<ED RAT£ " n. 

45 
The Earned Rating for the applicant group was derived by taking the- raw 

weighted esc total score, transllll!ting it to a metric scale of 1)....100 and, 

where appropriate, adding veterans preference (5 points) and additional 

credit for experience ro, 5, 10, or 15 points based on the esc Rating 

Guide criteria) to· the sct-:-"s of applicants who passed the esc Test 

Battery. This Earned Rating 1~ the basis on which OPM ranks and 

places eligible. candidates on a re&ister from which selections are 

made in accordance wfth OJ'M regulations. 

Table 25 prov±des· the restricted. correlation mtrix for the 1,827 ATC 

trainees at the ATC Academy. The means and standard deviations for this· 

sample were previously provided in Table 23. It should be note that the standardized (Z) scores were used for the MCAT tests in ·deriving the· intercorrelation (mean = D; SD = :!: 1). 

TABLE 25 

RESTRICTED CORRELATIO~ ~~TRIX USED IN REGRESSIO~ fu~ALYSES 

---ZSCORES--- esc Z¥ esc 540 esc 15' DHTA DKTB Dill' TOT HCATA HCATC HCAT TOT ZLAB tl.·Ril 

esc 24 1.000 0.))) 0.11<0 o.z~s 0.2SS 0.271 o.z1• 0.14<1 O.Z04 0.09~ O.JU 

esc 540 0.)3) 1.000 0;14S 0.1, o.zzz O.ZZ7 O.Z04 0.190 0.221 0.09~ 0.289 

esc 157 0.140 0.145 1.000 0.090 0.140 0.126 0.2•1 0.164 o.2z• 0.071 O.Z81 

DB'!' A o.2•s 0.1, 0.090 1.000 0.80) o.•so· o.zsa O.Z68 0.29& 0.207 0.1)8 

DB'l' D o.2SS O.ZZ2 0.11<0 0.80) 1.000 o.,. •. 0.2S. O.Z68 0.)08 0.22) 0.)41 

DHT TOT 0.271 0.227 0.126 0.9SO 0.949 1.000 0.2112 O.ZIIZ 0.)16 O.ZZ7 o .• )S6. 

I HCAT A 0.21• 0.204 0.241 o.2se o.2so· o.zsz 1.000 O•S96 0.90) o. 2"4 o.,&l· 

Z HCAT B 0.14<1 0.100 0.164 0.268 0.268 0.282 0.596 1.000 0.8112 0.250 o ... s 

Z HCAT TOT 0. 204 0.221 0.229 0.296 O.l08 0.)16 0.90) o. 8112 1.000 0.2.77 0 • .518 

Z LAB 0,097 0.096 0.071 0.207 0.223 0.227 0.246 0.2SO 0.211 1.000 li.27-2 

DL·RD 0.)4) 0.289 0.281· 0.))8 0.341 0.356 0.081 o ..... s o.su 0.272 1.000 

The correlations of particular interest are those· between the tests and the 

"ZLAB" criterion. These are zero orde.r validity coefficients. the effects 

of restriction are immediately apparent in the low correlations between the 

esc tests uSed in the actual selection and the ZLAB criterion. The two 

highest zero order coefficients are MCAT TOTAL (.277) and DL·RD .272, 

neither of which was restricte-d directly by selection of the train~s. 

Two. CSC tests, 51.....Spatia1 Patterns and 135-Following Oral Directions 

were anitted from the mtrix. They were eliminated based on the :1egative 

skew previously discussed. Extreme selection, such as in the case of ATC 

trainees, results in a sharp reduction in the variance in the selecte-d 

groups as. n·as shown in Tables 13 and 23 which compares the means i:t.nd 

standar4 deviations for the total ATC applicant <]roup and the ATC app_licant Pass Group, and the ATC trainee qroups. 

46 
When correcting for restriction in range, the difference be~ the applicant 

variance and the selected group variance is E!Tplqyed as a treast.J.t'(> of the. · 

>::~~e:o~;f~~curtai:; 'J.m;,nt that OCGUrs due to selection. It was not determined if the ·.·· in a violation of the ~inearity assurption; l'vwever, the ext= 

:,'.,•QH;piirll:Y bP~ the t:"-0 variances for esC 51 and esC 135 resulted in a corrected 

~~~~~~·~~~. that was much higher than the other corrected correlabons. In this 

a if the correlations for esc 51 and esc i35 ""re used in the m.!l tiple 

tegn!ss:ion ~ysis, none of the other tests either independently or in canbination 

significant to ther<,ltipl•· R beyon::l esc 51 and esc 135. These results 

cntsl·deJ:ed spurious and esc 51 and esc 135 ""re excltrled fran the following 

t~LU'tlF>le regression ahalyses. 

Table 26 pmvides the est:iJnated unrestricted correlations (as ,..,u as the actual 

.um:-estr:'lct:ed rorre1<~tion frcrn the esc sarrple). The rorrelations of primary interest 

the correlations of the te!;its with the ZIAB criterion. After correcting for 

restriction, the t-eAT at .531, Dial Reading at .466, and Directional Heading 

.461 have the highest zero order validity coefficients. 

esc 24 

esc S4'l 

esc 1s1 

DHT 

Dirt 2 

DH'i''roT 

"K':AT A 

IICATC 

JIICAT TOT 

ZIJ\! 

DL•RD 

l~l~ 26 Un~trfcted and Corrected Cor~lations Used lo Regression ~alyses 

CSC 24 CSC S40 CSC' JS7·· DHT 1 DfT 2 DHl TOT HCAT A HC"T C HC"T TOT 

.690 -- .570· 

• 700 

.061 

.szz 

.szo 

.6)0 

.630 

,650 

.670 

.690 

.SlO 

.650 

llAD· 

.)42 

.386 

.H9 

'~--

.... > ,, ~. -

ll.·llD 

• ~15 

.501 

502 

,.)98 

-~10 

.SZl 

.611 

.5Sl 

.63S 

.... 6 

··~~~tic'~The~ next step in the analyses was to enplay the unrestricted and corrected 

ji in a series of stef>-wise multiple ~ession procedures. F.ach rroclel 

refinenent of the previous ~1, with test srores reqress:ed on ZLI\B. The first 

excluded total SCOres for !liT and JI'CIIT, since these are the Sl.Jl> of their past 

that ...ere included. Results are present:ed in Table 27. 

r-nsr: 

ISq I'UI.I. • aq llrlltJCD • 

Dtrn:aua· • .... 7·. 

'fol;le 27 

DnTU.L IIIGttSSIOII III>EL 

• - 0.568'9 

., InA 

esc 24 0.0071 

esc·»> •Chi0066 

esc U7 0.0555 

DIIT .l 0.0,13 

Dllr I 0.0912 

II:AT.l 0,1452 

K:ATC 0.1668 

DL'ID 0.185.6 

ac. coxs:r. • 

TOtJd. !UEL \Jl'!B PAlO' SCORES 

0.3235, Hod.d 1 

o.ooeo lladel o 

0.3ZJ6 

JtiiJ' - 1100. 47 

ISQ • 0.3136 

• 

o.oon 

-0.0007 

0.0043 

0.0057 

0.0101 

0,03.22 

0,0/o&J 

0,0201 

pc. 0.0001 
In suc;:ce~ing models, esc 540 was eliminated bec.auSe of its lack of 

contribution, and total scores were used for DHT a.nd r1CAT instead of part 

scores~ The resulting multip1~ "~" was ~5672. Then n·ial Reading was .. 

eliminated becaus_e of its 'margJn~l value in the 1972 and- 1977 .fAA ATC 

Specialist studies, and DHT was eliminated because., in its current: for-mat 

(very brief and highly speeded) it is not suitable for operational Use. 

Results are presented in Table· 28. The initial model yielded a m'q.ltiple 

"R" of .5669', the final ... model yielded a multiple "R" of .5407. 

F•'l'EST 

ISQ PVU 

ISQ I.ED~ • 

DIFFEU!ICE 

DFH.• 2... 

Table Z8 

I • 0.541)"7 I.SQ • 0.2924 

v 

esc 24 

esc 1S7 

ltCATT 

IE(;. CX!HST. 

24 • 1S 7, !CAt TOtAL 

II£TA 

0.06011 

0.0964 

0.4391 

• 

0.%924 Hodel 5 

.o.oooo Model 0 

0.2924 

DFD • ta.oo. r-u.no • 371..890 

• 

0.0090 

0.00-74 

o.o,o 

03.2045 

P• O.OGOl 

A fac:tor·.analysis (principal axis analysis with varimax r:otation) · .. ·::as 

perfomed to explore the characteristics of the the test scores. The 

results are provided in Table 29. 

Table 29 facto~ Anaiy•i• oa Te•t ~c_ore• (P.ri.nci-pd Mil Analyai• - Yaric.x RQtation) 

Yarhbl .. Factor 1 Fact.or 2 Factor 3 taccor 4 Factor 5 Factor 6 

esc 24 0.97U -o.osso -0.02'61 -0.0~81 0.2583 .. 0.0211 

esc wo O,U96 O.l56t -0.112'9 -0.5921 o. 5849 -0.0564 

esc Ul 0. lS 78 0.0715 -o. 1Jo2 0.0079 0.5883 -O.OJSO 

IIIIT T 0.2638 •0.1068 0.0238 -0.0347 0.1l5~ 0,0543 

IICATT 0.2115 -0.1042 -0.0066 -o·.oJOs. o. 7)57 0.0521 

zua 0.2134 -0.1036 0.0141 -0.0415 0. 7'666 0.056J 

DL·tD 0.3131 .a.0885 -o.0788· -0.0704 o. ~-=:.v 0.0315 

I Wl"iace aecouate4 for 22. 7l88 2.5l35 7.9DSJ S.l392 42.U11 0 . .2097 

Two rather clear structures underlying the data with the orthogonal 

:r,o~Ltion•ppear. Fa~tor land Factor 5 account for 22.72 and 42;99 .percent 

var·ian~e· respective~y. Factor ·s contains the largest loadinq for 

tests and Zi.'.;G with the exception of esc 24. It is also notable. tl'lat 

d~vision seelllS to occur on both fact:ors between the esc; tests (24, 540, 

afld the experimental. tests (MCA1'; DI!T, DR·RD)· and ZLI\B. 

48 

'
The bet"- weiqhts· derived from the Final Regression r10del (Tabl.e 28} 

were converted to raw score weights and then assigned unit weights. The 

following equat·ion cons~itutes the. composite score: 

Yl • 1 (C~;;: 24) + 2 (CSC 157) + 4 (MCAT) 

where Yc. -= the compcsite test sct)re for the ba.ttery. Using the unit_ weights, 

the multiple R derived was .535~ compared to .5407 with beta weightS. The 

R2 value derived was .2867 with unit· weights compar'ed to .. 2924 using beta 

weights. 

A cross-validation stuPy was performed· t~ investigate the stability of 

the results. The sample was rai.uiomly separated into two groups, and the 

weiqhts derived from the f·irst sample were. applied to the s_econd sample to 

Qbserve shrinkage in the multiple R. Distribution statistics and· intercorrelations for tbe two groups are presented. in Table 30. 

Slllo!PLE 1 

.... sua: 4.7 

llSPANlC 14 

AM. INDIAN 0 

Oll!NTAL 6 

!SUMO 1 

om:• 730 

m.u. 7U 

%I.AI 

esc z• 

CIC 157 

MCAT 

TAilLE 30 DISTRIBU:riON BY RACE AND SEX AND 

DESCRIPTIVE STATISTicS 

w._ • Total 

l1 64 

ILACI< 

IISP.WiC 

3 17 

1 1 

I 7 .AH. INDIAN 

0 1 .ORIEH"TAL 

94 824 [SKIHO 

OTR>:R 

116 914 

:mTAL 

IlEAl! S~D. 

0.018 1.001 ZI.U 

46.998 6.1h esc 24 

38".490 6.538 esc 157 

35.608 7.451 MCo\T 

SAMPLE 2 

• •• Wo~~~tn 

45 16 

15 4 

0 0 

7 I 

I I 

723 101 

791 123 

IlEAl! S.;>. 

,.0.020 0.99Q 

47.026 6.853 

38.252 6.244 

35.616 7.307 

COUEU.TtONS CDIII!UTIONS 

ILAI 1.000 0.328 0.402 .0.537 %I.AI 1.000 0.326 0.396 

esc 24 1.000 o.soo 0.530 esc 24 1.000 0.50"0 

esc n> 1.000 0.610 csc·151 1.000 

11:.\T 1.000 IICAT 

49 

Totd 

61 

19 

0 

8 

2 

824 

?14 

0.527 

0.530 

0.620 

1.000 
The regresdon equation for the first sample is presented in Table 31. 

r-n:st 

lSQ FUI.l. 

ISQ I!EilfJCI:O 

DUFEREICI:t 

""' - 2. 

Table 31 

• - 0.5450 

esc 24 

esc 157 

MCAT 

t£.c. coxsr. 

CROSSV.U.lDATIO!-i 

BETA 

0.035t. 

C .. l023 

D.4S46 

0.2879 Model l 

0.0000 Model D 

o.n1o 

RSQ • 0.2970 

(1.0052 

0.0155 

o.ous 

3.0ll3 

DFD • 9"13. F-RATIO .. 19'2.,47 P.C.O.:>ool 

Unit wei.qht~f derived from t:t..e beta val1:1es for sample l were computed. 

These unit weights (CSC. 24 • l• esc 157 • 2, ancl, !!CAT. • 4) were then used 

ccmpute the multiple ~s and R2'!r·'in Sample l and sample 2 "i th the 

followinq.resul~s: 

Sample 

l 

2 

Mul,tiple R 

.5381 

.5292 

.2895 

.2801 

discussion of the .analysis and results on the C~ and experimental 

a~pointees to the ATC occupation in 1976-1978 is provide in 

'lJt,tfiU'ernce 20 • The r.e-stilts from these analyses supported the conclusion 

CSC 24·, CSC 157 and .!CAT tests be used in combination as a selection 

.ttm'v for screening applicant air traffic controllers. Some information 

provided that indicated a modified form of the Directional Headings 

suitable for operational use and a Dial Readinq Test should be exp.lored 

future studies. 

th11 analyses <>f tests in relation to ATC trainees hired during 

lt76·l,978 discussed thus far, the ATC Occupational Kno~·ledge Test (OKT) was 

fraa t.he vario~s regression analyses.. The OKT is a •job-knOt: ledg-e 

test and is no"t intended for use a.s a •pa.ss" c-r "fail• exam in 

"'''llri1UJ1L~n,g applicant eligibility for emploY!ilenl: consideration. Rather its 

limited to measuring ATC r.elated experience and knowlooqe as 

granting extr:o point credit to those applicants who pass the 

llp<ttlcti.ve test ba.ttery. Consequently, the An; Occupational Knowledge 

was. analyzed as a parallel effort. · 

50 
. 

' / 

I -, 

Ot;,jective. The analysis w(.o..~ corrlucteq to det·eri':'Iine t]:le re-lationship of Ot."1' to: 

the present method of granting extra_ credit based solely on A!~ related experience. 

Pass/fail st.a,tU::i during Initial ATC Qual.ificat.ion training. 

An ATC selection test Qattery CQmprised of CSC tests 24, 157 

and MtAT"and the combined estimated vali~ity of the test battery 

and OKT. 

Sample Description. The sa.mp"le for the analysis of OKT was the ·~ame group of 

1827 ATC trainees (Table 22) tha·t was used for regression analyses of the esc 

and experimental tests previously discusse9 .. 

Predictor. The OKT (form 1018) is a 100-item multiple choice test co•J<!ring 

air tra{fic con·trol reg4la_.1.0ns, com:r.'IUnication, ·fligtlt service station work, 

navigation aids, wea~her, and radar. The score is the total number of correct 

respons_eso. The mea.n ard standard d_eviation for the 1827 ATC trainees was 66.96 

and 16.3 respectively (Table 23). The 1\.uder-Richardson - 20 internal cor.sistancy estir.\ate o£. reliability yielded a reliability cofficient of .95 for a 

sample of ATC new' hires.. · 

Critex-ion. The same criterion rneasure;ATC laboratory a\.'erage score (ZLABl, 

that was used in the m1.;.1tiple regression: study was used in. analyses of OK:'-. 

Analytical MethodOlogy. The 1827 ATC trainees were ~rouped into_ vctrious 

A're relafed experience categories based on their response to a biographic-al 

questio~aire administered on the first day of training. Four types of 

A'IC rela-ted experience were identified. 

1. Ot.her experience (included air defense command, communica-tions 

o·perator and· prior ATe training without ATC oper-atio.na_l experience). 

2. Pilot experience 

3. VFR or non-:z::a-dar A'll: e~rience 

4. IFR or radar c~ntrol ATC experience 

These four types o·f experience (together with "No A'l'C related experience") 

were then used to classify each· of the 1827 trainees i_nto 16 experienee subgroups which provided· for combinations of experie"nce as listed beloW. 

51 
Subgroup Type of Experience (~) Group.(~) 

l No ATC related experience sse} 2 Ot)ler experience 34 

592 

3 Pilot 297} 4 Pilot + Other 92 

389 

5 VFR only 99} 6 VFR + Other 27 

7 Vi'R +Pilot 28 

8 VFR + Pilot + Other 37 

191 

9 IFR only 

10 IFR + Ot.'ler 86] 29 

11 IFR + Pilot 18 . 

12 IFR + Pilot + Other 11 

144 

13 IFR + VFR 

14 275} IFR + VFR + Oth'e:r 95 

15 IF.J! + VFR+ Pilot 57 

16 IFR + VFR + Pilot + Other 84 

511 

Total 1827 

Because of the small nuntber of trainees in some- of the subgrOuPs, they 

were combined into the following five major groups: 

Group 1: No experien,ce + Other experience only (592) 

Group 2: Pilot or Pilo~ • o~;er experience (389) 

Group 3: VFR + any additional experience except IFR (191) 

Group 4: IFR + any adchtional experience ex_cept VFR (144) 

Group 5: IFR + VFR + any additional exp.er ienc;e (511) 

The test· scores for OKT were corr·elated with experience and late:;- correlated 

with the criterion ZLAB.. Statistical analyses of ea,eh of the five experience 

9r0ups in relation to OKT score ranqes and- pass/fail status was also conducted. 

Results. .The correlation of OKT scores with experience was .61.- plCT correlated 

with the ZLAB criterion at .22 versus .ll for experienCe credit. This indicates 

that while OXT is closely relAted to experience, it is more predictive of~Uccess 

_than the pr.esent method of qiving credit based on experience in accordanc·e with 

the OPI! Ra tinq Guide 

The results of the statistical analvsis of' eact . .,r t.he five experience 

OKT score.s and .~TC traininq pass/!ail status is provided in 

52 
.... ~ .. ---

. .\ .. -·-- •' 

'r•ble J ~ 

F'aJiurc- Rate3 by OiT ::.t·orr uan~cs and Ot.. T "\eans by t'lper1rncr Groups 

Score m~er Palot Vftt JrR Vt"M· • lfR TOTAL 

To. tal I fall . Total I fa&l T.otal 'hil Total ' fall Tot&l • fall Total l F'ail 

O·S9 •J• 29. l" 67 ro ·'*"' " Jl .OS II 27 . Jl <J llj.';l 5SO ~9 .11 

60·6- ]7 l7.01 " 23 ""' 

,., 111. 1J • lJ. )I <l Jtt.e; 1 JS l9.6J 

65-69 11:0 ZS.01: ss ij.J-; .;: 1~.01. 1) 23. 11. •6 :3 -~"' 179 l0.1S 

• 

10-711 l' zo.6t 77 n.o: )t ~9. ill ,. :o.H ot 1-;.u ,,. loi .Ill• 

75-79 >2 ~." tJ IJ .II; jl l j . .,, ]4 1:. a' 111 1..1 .')i 2"11 11.61 

ao. A 1Z .0~ ..lL 10.S"S ..!..!.. 1.]. ~-~ .!lL S.H ~ J . .. .!1!/.L !LIIJ 

TOT&L 59> zo.c;t 1a• 1S.ZS HI Zt.Si ••• 1).21 511 _,. 3i0 1621 19.21 

Me ana 

on 

... 51 .• 9 6!9.911 12.ZS 76.28 1a.o 66.96 

atd. dev. ,,.._e 10.9 11.11 •. s 9.7 16.) 

While th,. mean OKT score for !:he total lB·27 ATC trainees is 66.96 

(SD ._ 16. 3}, examination of Ta,.ble 33 shows marked differences for the f"ive 

expe.r.i:~c:e g..:;:,ups and, with~ each g-roup by OKT score· range as shown below. 

EXperience GrouE: N Mean SD Percent· of Failure 

No + Other ExperienCe 592 51.49 14.8 26.9' 

VFR + Any other (except IFR) l9.l 72.25 ll.4 21.5\ 

Pilot + Other Experien~e 389 69;94 10.9 15.2\ 

VFR 5 IFR + Any other Sll 78.00 9.7 14.3\ 

ll'R + Any other (except VFR) 144 76.28 9. 5 13.2\ 

Of the total 1827 A.TC trainees, 359 (l9.6\) were in ATC related experience 

qroups which presently ee,rn extra credit under the CSC Raeing Guide for this 

.,.;>erienc:e and whO scored less than 70 on· the OKT. The failure. rate 'for the 

359 was 26.2\ as compared to 28.2\ for the Sll trainees with no (or other) 

A'l'C related experience who also scored less than 70 on OKT •. 

A8 can be seen from·'l'llble 33, as OKT·score- ranges get higher, the proportion of failures generally c:ontinuu to drop in each of the five experience· 

qroups. Rather than uainq the present method for granting extra credit. for 

experience, the .use (If OltT .Cores would be significantly 1110re predictive of 

A'l'C training· success •. 

Tlius tar, OK'!' data reported has compared the OKT tes·t s<;eres to the pre.sent 

methOd of crediting A'l'C related experience and each of these in tents <lf their 

predic:tion of A'l'C training pass/fail status. For OKT to be truely Useful, it 

,....t Contribute to predicting A'l'C training suc.,ess over and above what can. be 

achieved by a n- selection test battery. 

53 
As: previou.sly .:ihown.,. a test battery comprised of esc 24, esc 157 and 

MCAT had a.multiple R o~ .54 which accounted for 29~ of the variance on 

ZIAB. If the correlations t..~ Ol\T w.ith Zl.AB and the other test battery 

scores are· corrected fer restri~~ion in. range using Thorndike's formula 7, 

then the mult-iple regression Coefficient including_ OKT can be esti.mated. 

The new Multiple R, calc:uia-ted including OKT as well as t.,_e three tests 

previously incl~ded in the battery would be .60 and 36\ of the ZLAB va~iance 

would b~ accounted for. ThiS increase i~ significant (F = 204.3 pL.OOl). 

indicating that the addition of extra points using OKr scores would significantly improve prediction and that OKT should be used as a bas~s for 

determining extra credit. The analysis of OKT summarized here is discussed 

in nore depth in.· reference 2 7 

54• 

.. 
OEVEI-QPM):NT OF THE MtJLTIPLEX CONTFOLLER iwriTUDE TEST 

EXPERL'1EN'l'l\L TEST FORMS. Devel<>pment of the Multiplex Controller Aptitude 

Test was ~ased, ·in· part, on previOus reseai:-ch efforts directed to d'eveloping 

perfo.rmance indices for air traffic; controllerS durinq 1970 ... 1971 (28). From· 

this research, the initial versions of the Contrpller Decisi~n Evaluation 

(COO£) film tests were developed and used in the 1972 study o.f Air Traffic 

c·antrol Specialists condUcted by Education and Public Affairs (1~). 

The CODE test used a mo·f:ion picture film to present simulated air traffic 

in re~l time as they crOss a controller's diSplay scope, together with a 

mileage scale, lines to represent airways a.nd intersections, and a data. table 

tO provide information on each aircraft's identity, route, speed and altitude. 

The test was administered in a free response mode, with the task bein9 to 

predict violations of aircraft separation standards (conflicts) as early and 

ac.curate1y as possible. 

Experimental admin~stration of three motion picture film~ were_carried out. 

Analysis revealed that most of what was measured. in each 45 minute film could 

be derived from only 7 or 8 of the pairs of confli~ting aircraft. In addition, there was considerable idle time durin9 testing wPi~h offered the opportunit-y to measure mo:t;"e than si.Jnply conflict p~rforrnance. Consequently, test 

develQpment. procedures Were initiated to measure not only this skill but others 

al we-lL 

~ initial modifi.cation was to develop a structured. test, using a slide projec;tor to place conflict-type questions on the scre:en. However, whe.n i terns 

were assembled- into a film/slide. vers;io_n, h_alf of the test time· remained idle 

and this provided the opportunity fOr test subjects- to change their answers as 

the test progressed and the correCt answers became obvious. It was fouhd that 

by presenting a new ite-m every 45 seconds the examinees were_ kept su(ficiently 

busy so that this- sit"l,lat_ion could be con.troll_ed. 

Ample· information is available on the sc~pe and table to ask a variety of 

questions. Items were written· that utilized the available detailed information 

to.mecisure such .. aptitudes as ciirection followinq, table reading. spatial visualization., and arithmetic reasoning regarding separatfon distances and times4 

Some items were very simple. Others were written in a multi-factor format to 

·.increase their difficulty. For example, initial instructional questions on how 

to read the: table we:re very easy, but a complex problem such as estimating 

tra~~rel time in minutes between· two aircraft (horizontal separation) required 

awareness of dis·t:ances across the SCOP4. reading the ·table to determine their 

speed, and mathematical computation to determine their rate of closure. 

Items were administered experimentally, th-e relation between- item types and 

total teSt homo<Jeniet)'· det•J:mined, and this ratio used to Qetermine the number 

of items per type· to iriclude in the test. A result, for example, was- l.1lclusion 

55 

.. , . 
of twice as many time-distance separation_ items as compass heading items in 

the test. Confl~ct item· locations had beeO established bY mean jour~eyman 

response ti!nes when targets were a·t certain._locations. APtitude items w.ere 

placed in the remaining positions, 45 seconds apart, alternating from one 

type t.o. anbther and. spiraling to increasing·· levels of difficult:-Y as testing 

progressed. About 40 percent of the items were conflicts, and the other 60 

percent were a~1tudes. As films 4, 6 and 7 each provided· a differe~t pattern 

of air traffic, three different forms of the test·, MCAT 4, 6 and 7 were 

prepared. ' 

Versions of the CODE test, the film/s-lide version of MCAT, and other selec"" 

tion measures were administered to students of the Navy Air Traffic School, 

MemphiS, and g~ades of thes6 students obtained as they progressed th~ough 

traininq. Analyse_s included interc.Orrelations and rotateQ. factbr loaQings 

Of the various tfi·sts wi t-.h each other and with such school g-rades as laboratory 

flight pl&ns, con~rol tower, and radar control problems, as. these grades ProVide the qreates_t rang&- -in criterion scores for predicting FAA student scores. 

The film/slide structured ver.sion had correlations with criteria that were 

&J high- or higher than those of the free respons.e CODE. These result~ encouraged further develo~ent of MCAT. 

A next step in test development was to create a "slides only" version, in 

esse.nce taking pictures of the· screen each tim.e a new question appearE:d·. 

Paein9 of th~ slide present·ation was the same as in the film plus slide 

ve~sion, so the amount of target movement, how far an aircraft moved from 

questio~ to question, was unchanged. Free response, film/slide and all-slide 

versions were administered to students at the Air Force Controller School, 

Kei!sler, and grades of the students oO.taincd as. they prog'_ressed through training. 

The all-slide version generally showed highe.r- correlations with the crite-ria 

th&n either the free respon_se or fi1.m;lslide_ versions. 

:During these· test adiriinistrations, it becam~ evident that the use of motion 

pictUre or slide ptojecto·r equipment would not be feasible for operational 

·.teat administration. The effect of equi~en,.t· fAilure, room siz·e, lighting, 

·:•eating arrangement, and other factors coGid not-be controlled at the many 

.: locations where the Office of Personn-el -Management administers -the test· to 

was to print th-e slide versions in ·paper-pencil fo.naat .. , Three 

and administered to several populations including: 

students at the FAA Academy during all of 1976o A~ Specialists 

p&rt:lcl.p••ting· in the 1977 study conducted by Education and Public Affa.irs, · 

, Wider FAA contract; and, 7000 A'l'C applicants tested by the CSC in tha 

of 1976 and spring of 1977. 

form vas then lengthened to. 55 items and administered to-all students 

FAA AcademY from 1977 until June 1978; students at variou-s colleges 

universities; and, students cit the "Army Air Traffic .Control!~+ s.chool I 

Ruck~r. All. tJ:lree forms were administered to each of the Army students 

differences· in mean performances among fonn-s were found _that -could effect 

comparability. A"!.$0, student performances improved. considerabl~· on 'the 

form they were- administered. indicating. that learning w-as still taking 

56 
place. but ti1ere was very little increase in perform4nce on the third form. 

Differences. in learning rate among students was noted, and this ability was 

recognized .to be another pot...:..,tially useful m~asure. 

Development of various fon.:s of M~A:J., has resulted in reliability correlations 

that v~ with the restriction in ian·ge o£ the scores fo+. the populations 

tested. Co.trelations between early forms were .31 to . 50 fot· ATC Academy 

students who had greater restrictions in rarige of their scores; .60 to .66 

for USAF ATC students; and, .87 to .90 fqr non-controller FAA employees who 

more nearly approxim~te the population for whom the test was designed. Validity of the ~CAT test with the different cri~eria used for predi~tion of 

'"ATC success" .have been identified in the various studies previously discussed. 

A more detailed description of the development of experimental forms of the 

MCAT test is found elsewhere (lq). 

The tes-t is desi;gned to measure the skills of the air· traffic controller applicant within an ATC simulated setting. What is accomplished, in effect, is to. 

teach. person$ a set of s.implified ATC. rul~s, and then test how well they can 

applY those rules in specific situations. Test subjects spe"nd eight minutes 

study~g the directions, and· then start to answer the questio~s. Because the 

air traffic contrO.ller activity i·s ·complicated and rather difficult to learn, 

the test is deSig-ned so that initial items are very eaSy,. and then progress in 

an order of increasing difficqlty. The time limits are deliberately. established 

to make the test· a speeded test. While usual aptitude ba-ttery design practice 

is to clUster items irito homogeneous subgroup.s, in this test the content of the 

items is alternated from one type to another and they Spiral to increasing 

levels of difficulty. This mode- of presentation. is found in only a few teSts 

such as the Stanford-Binet •. A result shewn statistically is L~at -the nonconfliction items show unexpecte4ly high commonality with those items requiring 

detection of impending conflicts. Apparentl,y the detection of confl,ic·ts 

requires the same skills and operations mea-sured by the· different kinds of 

aptitude items presented. 

Figure 3 provide·s the basic format of the MCAT. About. 40 percent of the total 

55 questions ask for identification of potential conflicts, or "'!'iolations of 

separation standards. The remaining qUestions ask for such information as 

differences in the routes of pairs of aircraft, how far apart two a.ircraft may 

be at a giv.en moment, their compass headings, and if they will come· into conflict. 

DEVELOPMENT OF P~LE~ MCAT TEST FORMS. For operational purposes, multiple 

forms· of a t-eSt are necessary for rete5t purposes and control over compromise. 

Based on previous item analysis and experimental test fonn develoPment, each of 

the three experimental test forms (407A, 607A and 707A) were split into two 

parts by placin.g the odd items in one part and· the even items into a second 

part. These individ.u~l parts were then placed in various ccmbinations to provide 

six availablE: forms. Each form was a 55 item test and separated into two parts. 

57 
FIGt'R£ 3 

TH£ MUL nPLEX CONTROU,ER APTITUDE TEST 

Flight Information 

Aircra'ft Altitude N Speed Route w+e ~ 

10 ~no G 300 AHC A 

s 

20 5500 300 EIHB 

F 30 so,;o 450 DIJF 

40 6000 450 IIHJF 

50 6000 300 C.ll!B 

E 

A typical example of the kind of 1nfonqat1on provtd·ed in tbe te~.t prob,lf:m&· 

is given· above. The dr,awtng shows the particular flight ·paths that af.rcraft must 

follow. Changes in routes can occur only at the intersection bet~en two routes. 

Each x depicted on the routes represents -an aircraft traveling in a part~cular· direction i-ndicated by the 9'ailing dots. · 

To the side, a table containing critical flight informatio.n about each aircraft is provided. The number next to each x on the drAwing allowa the ID&tching 

of the flight information to the correct aircraft. The flight information lists 

the altitude, speed in miles per hour, and route that each aircraft ie flying. 

58 

·' 
Each part reqvi~ed a person to· learn its unique mix and pattern of traf!ic and 

make predictions as to traffic behavior and potential conflictions. ?art 1 

was administered allowing 20 minutes for completion.; 15 minutes was allowed 

for Part 2. The six forrris were· then administered to two sample· popul3.tions; 

new ATC hires entering the FAA Academy for training. after June 20, 1973·, 

and to approximately 7800 applic~~ts for ATC work ~ho took the present ATC 

test admin·i3tered by OPM in October-November 1978. Table v~· provides t.he 

de·scriptive statistics for each group for -each form of the MCAT test. 

L\BL[ 34 

P l·]~.!~~~R-~L\:\S;_l.~-0_~ . .:'J-~I")·~K_:\~\J_I·~-__1-~~·'_1{.~~~ {) F ~!CAl 

FAA AC..\IJEMY OPM ATe APPLICA!;TS 

FORM NO. IO STUO~'TS 1978 TESTED 19?8 

~ OPM ~ §R ---N Meat) so N 

4o6e 120 33.9 7~45 459 29.3 9.15 1233 

4e6o lJO 37.4 7.35 458 29.3 9.70 2328 

6o7e 140 40.4 7.45 434 33.1 10.40 1222 

6e7o .150 37.6 6.80 319 n.s 8.20 1086 

7o4e 160 33,8 7.65 115 29.6 8.70 1135 

7e4o no 39.8 7.25 356 32.2 10.34 862 

The test is for use with OPM ATCS applican'ts, and those are the data of primary 

interest. The FAA Aca:demy students data are included fo·r future t·est develop-· 

ment work, to estimate applicant scores o~ new forms of the test when exp~rimental data are gathered on FAA Academy st.uclents. While all forms are quite Comparable fo~ the ATC applicant sample, i~em revisions will be made in forms 

6o7e and 7e4o to increaSe their comparability.. 

~any different scoring methods have·been explored duririg development of the tCst. 

The test could be scored for Rights and Wrongs, with separate scores for co·nflictS 

and each of the Aptitudes, and with differential combining weights for all these 

subscores. The test could be scoreri fo·r the rate of improvement from· its early 

to later parts. Various-multiple regression ~tudies indicate that little is 

gained ~rom maintaining separate· $Cores for Conflicts and Aptitudes as was done 

in the 1977 study conducted by Ed-ucation and ·PUblic Affairs, Inc. Consequently, 

perfor.mance on all items have been combined into a $ingle scOre in Fubsequent 

analyses. The data also showed that little was gained through use of ~ combination of "Rights and Wrongs" instead of "Rights~' onlY .. 

For a group of 617 ATC students at the FAA Academy tested between June and 

December, 1978, a total "Rights" score test-retest correlation of .60 

between Comparable two part 35 minute forms of MCAT was obtained. A planned 

doubling of the· tes.t. length providi~g a tota·l test tirrie of 90 minutes should 

increase the reliability coefficien·t to·. 75 for this restricted group arid 

provide an adequate level Of reliability for the ATC aPplicant population . 

. 59 
The follO\o>i:ng su:rimp..r:.:;:es ':-;,e development of the MCAT test. 

CODE- fill':' .. ·.t~rs~ons 4, 6 and 7,. Scoring: 197l,c 40 items; 

1914, Stan!.:o-.e: :-:r;~. corrected item scores- best items. Unstruc:.';lred !"ree respo1.-::e. 

FiL~ Slide ~CA~ - Julv 1975. Tim~ cont~olled for answering each 

item. !-1CAT' 4C.O :s.; 41. items, 606 FS -·43 items, 706.FS- 53 items. 

Struc:~red ~ul~~~le choice. 

A!l s:ide ~~T - October 1975. Time ccrtrolled for answering each 

item. MCA~ 406 AS - 41 items, 606 AS - ~J items, 706 AS - 53 items. 

Struct.ured 1m1lt:.ple choice. 

Paper ?encil MC.1J ·- January 1976.. MCAT 406A - 41 items, 606A -

43 items· l25 Ct.i-~tes test time) 706A - 53 iterus (30 rn.ir..utes t.est 

time). Structu:ed multiple choice. 

Paper Penci 1 MOX - April 1976. Time controlled for answering each 

five =inute qr~ of items. FOrm 606B - 43 items, 706B - 53 items. 

St.ruc:.Ured :nulCple cho,i.ce .. 

Pape~ Penci-l MOT - January 1977. Time controlled for working tot-al 

test (35 r.t.!.:tU~e.s~. 407A, 607A, 707A eac}1 55 items. Struct\;Z"ed 

cult:.ple cboice~ 

Paper Penc :.1 MG.T - June 197Q. Parallel test forms·. Two part tests 

vith each f~"rt. ;:resenting a unique. pd.ttern of air traff~c. Part· 1 -

20 mi~utes. Pa== 2- 15 minutes. Forms, 4o6eJ 4e6o;_6e7o; 6o7e; 

7e4o: 7o4e: eae3 test 55 items. Structured multiple Choic~ . 

• 

60 

... 
• 

OEV?:LOP!'ENT OF THE AIR TRAFFIC CONTROU,ER .OCCUPATIONAL KNOWLEDGE TEST 

The ATC Occupational Knowledge Test (OK'l'} is ci subject -element, paper-p.encil, 

machine scorable._, oultiple choice type test. It measures importan.t ·aspects 

of knowledge assoc1.ated with prior experienc~ that are related' to successful 

performance as an air traffic controller speci-alist. The subject ma-tter 

areas included are Air Traffic Control ·Regulations, Terminal Air Traffic 

Control, Ei1route Air Traffic Control, Cominunications, Flight Se-rvice St~tions, 

Naviqational Aids, Weather, ~ Radar. · 

Experimeutal Test Oe'lelogmen~ 

The. initial A'l'C Occupational Knowledge Test was designed. to measure the 

acceptability Of an applicant 1 s claimed experience for qualifyirlg at the 

G~-9 level or above. The.r.e was such a wide· var.iety among applicant's claims 

of experience that they were difficult to evaluate, and_ the OKT would provide 

a:ores to verify the acceptability of cldimed experience .. The fir·st step in 

test development was a two-we-ek worls.shop in June-1970, in Which eighteen 

journeyman air traffic control specialists from Terminals, Centers and 

Flig-ht Service Stations participated. The workshop was to te"ach these 

people the specifications for the test and help them wri~e it~~s·~~der the 

tra,iriing arid superVision of representatives .of FAA Headquarters. This 

committee ex~~ined the exact types of qualifying experience that.were given 

cred.}.t, and d~tennined the SI?ec-ifications fOr the test to Permit its application to verify claimed experie~ce. Then over 300 items were written, and 

reviewed and edited by the Examination and Certification Section of the ATC 

Academy, Oklahoma City.. Two test booklets we-re created·,. Form 15 containing 

150 of the items, and Form 16 containing 160 items. 

These forms were a~~iniste~ed to two ~lasses of students on the-day they 

rqported for .. training at the ATC ·A,ca~emy, and to szunt.les of GS•7 and GS-9 

Air Traffic Contr_ol Specialists in Fli-ght Servi-ce StAtions, T~rminal~, and 

Center·s. The tests were foOJnd to correlate highly with past experience 

patterns, had acceptable validity coefficients with· training success for 

Center, Terminal, and Flight Service Station training, and demonstrated 

abil.j.ty to differen-tiate Air TraffiC? Controller from Flight Service Station 

job inc.umbents. The results _of the experimental·-·testing were analyzed at the 

individual item- level', and these dat.a used for creating multiple fonns of 

the test. 

Sine~ planned initial application was a test for use as a partial basis for 

determining applica...;ts' qualific;.atior1 for Te1.:minal and Center jGb entrance 

above the GS-7 level~ a first step was to eliminate items from consideration 

that had been found lacl<ing by experts in ehe specialty area. The perfo!'l1lance 

of virtuoso air traf.fic contx:olle.t"S was cOmpared to performance of en-tering 

ATe Academy ·students on an it_em by item basis.. Prime criteria for an 

item's selection won:e (l) its ability to Gifferentiate between the two groups 

and {2) the item's. difticu!~ l'evel •. Til<> eas!.es.t 100 of the most discr.!,l;<inating 

items were selected for inclusion in l"erm 101 - E)(pl!-rilnental and the next 

most diffic:ult 100 items were assel!lbled as Form 201. 

61 

' .. 

' . ~--
. :; A 'snnre detailed discuss_i.O.f1 of -tbe initial development o! -the Occupational 

"lcnowledge Test is found else\olllere (JO)." Subsequent to 1970, FAA discon-

\>Y·c'ti.r\ued hiring- ATC applicants __ at g-radeS ~ve GS-7 and work on developing 

'"the ATC Occupat.iori.al K!'lowledge (OKT)' was deferred until 1975, w-hen further 

· ,J:"e$ea.rch ~as in.itiated. to __ iJDpEove ATC selection. 

·.·:;,~:In -August, 1975, a. battery· of ~xper~ta:l selection tests, including Forms 

· :.:- ,~~·101 and ·201 of -the ATC Oecupa-ti91'lal JS,n-owledga Test, were administered to 

'-"' 109 stude-nts of the Navy Air Tr'afffc School, Memphis. Studer:.t scores on 

· ·~-.;~ Fonn 101 were dis-t.rihuted against their week in training when tested, and 

these sRowed increases in tes-t perfo"dn.ance levels as the students gained 

~ . .experienCe. Grades of tl'fes":- stude~tt;::~ere obta"ined as they progressed 

·, through. training... Analyses _includedt!htercorrelations and rotated factpr 

_ .. .,,-:__;~;> J.o,adings of :the various tests with_-~ -other and. with school grades such 

~--(-~~;_:.a.s :ac.ademic and labo~atory gt;~des ~:course average_. Distribution statistics 

·_-.'plus c.orrclations .with scores ·on selec_ted variables are ·presented in Table 35. 

Mean soor~s on torm 201 ·Are tell points low-er than on Form 101, and the· Form 

· 201-tetal score correlati~ wi~h other measures are similarly depressed • 

. ,. TAJ!Ll; 35 

., PERFOR:-1ANCE RELATIONS BETh'£ Eli ~D SELECTED '! ARIABLES 

~y ATC.Students 

.4u&ust 197.5 

,_-, __ ,._,., 

OKT 201 

Mean SO M~an SO 

TOXAL GROUP (N-109) 

. CORRELA.~'ION WIT!h 

~a ding 

·Educati-on !.eve 1 

MCAT 

ott IOl 

OKT 201 

-.. _ 

,:S.ehool Gra<les , ·,,.ii/i. · <Academic Average ~', -;~j_;_:·-- . -~-.' -~ 

· : i lllock II Ccm:posi. te · ·, •· •· 

61.6 

-.02 

.27 

.42 

.16 

.48 

.28 

8.9 51.5 

r 

-.01 

.10 

.22 

.16 

.25 

.16 

6.7 

~~:::.:::~ ~,.. ·. ·~ ::; :~ ··: .. 4.-.~n the ··factor."1lTialysi'S,·'S'tudent~cperfonuance on the ATC Occupational Knowledge 

· .: ; Test (OKT) showed little. or. no. i:ot\llitiinality with their ability to read and 

>---:~rcornprehend the materiu: and v.etY, .. "little.relatidn with their mechanical-spatial 

,:~,:;aptl.tude~. Qta'-101 had:hi.'lh 1,,,.· .. ~.on the Air Traffic Controller Perfor-

·';~ ·" ~ce. factor, the Cont:rOi''Towe_ _, r~\:Or fac~or,_ and the Laboratory factor. t: ~-201 had a similar .. fac}~or a· · ·Stiii,. but with less loading on each of the· 

~:!factors., perhaps as a_-£urictiott;,;,- _ · re~ri.ction in varianc-e, since it wa& a 

• ~ .ratbt!r .difficult test hJr J::ll4 .:.P"'P"l.ation • 

. _;;-

-~- -

. r,. },_,, 

-_;::. ...... ,. ...... - :i\ GZ 

,,. •' . 

;_;,_,.4 

-:;~ 

~~~-: {]1 

~ -~-
A battery of experimental tests includtng ATC Occupational Knqwlecige Test 

101 Experi:nental ~~·:ts also. ad:minlstered to forty-one Students at the Aj.r 

Force Air Traffic Controlle!" Sc:hool, Keesler AFB·; durL"'19 October l97~L 

Distribution statistics 'and correlations with selected school grades 

are presented ifl Table 36. The performance_ of these students were quite 

s~ila·X' to t.."lose of t.l,e ·Nav~; Ai~ Traffic Controller: School sampte. 

TABLE 36 

PERf\1R..,1..'~.-'~CE RLLATIO:\S B£T\~~r.~ 0K-l~ -A:\D SELECTED \'ARIABLES ------ ---. ·-------"-- ---------- ·oiZ'l--irlc-- --- rsAr ,\'!C S1t:DEnS ---- Standard 

Q.s_~o_l?E_r_l9? 5 Mean Deviat:i,on 

Total Group (N=41) 63.66 16.34 

School Grades, Final 

School Correlations 

Block I 

Block II 

Block III 

Block rv 

,; '- ... - ~"'-·"·~· Basic FAA C;rt~i¢~ion Basic Operatiodt-·~S{>ecialist. 

C~ntrol Towei Operator 

Radar Controller (6CA/PAR) 

E 

.42 

.15 

.32 

.Ge 

Based on the results of administering Fonn··101 ar1. 201 of the Occupational 

Knowledge Test t.o tl),ese g;roups, as well as to controllers at selected FAA 

tc::rminal and enroute facilities in December 1975, the best 100 items were 

selected to create Form 1018 of the OKT. 'This form was part of the experimental test battery administered to FAA facility person.Rel and ATC Academy 

students in the 1977 selection studY conduc_ted by Education and· Public 

Affairs, Inc (6). Distribution statist~s~ and correlations of OKT-lOlB 

with Supervisory Asses51nents obtained in" the 19"'7 study for field per'sonnel_ 

are given in Table 37. Correlation with supE;:rvisory assessrne:::1ts for fie·ld 

ATC personnel were among the highest foWld for vai:iable~ in the e>".,)eri.Jnental 

test battery. 

TABLE 37 

ntsnusrno!{ STATISTiCs A."'D cOIU.EL.ATIONS 

ATC OCC\:PATIONAL XN01..1...£PGE t.ES't - lOU-:-

(Fa~tlity ATC Specialist&- 100 ttecs. 97 keyed). 

ATC Option Corrcla~ion vit-h 

and Year Hi red S!!e!rvi&o!:I .U.~eumeot !! !2!! 

llR 1969-70 .1) 97 BS.l 

1913-74 .1-4 

~-,-~~·":.!?::;z~· _1:: es.o 

TOTAL '"l't·: 85.0 

~ .. vn 1969--70 .n "' 69 82.0 

1973-74 .29 .. .:···'tOO 80.6 

TOTAL .21 169 81.2 

AII.TCC 1969-70 .21 88 79.~ 

1973-74 -14 94 n.o 

TOTAL .17 18~ 78.4 

rss 1969-70 .OJ !-~ ,_ .. ·t:--:~ 75.0 

1973-74 .34 7J.S 

TOTAL .19 196 7].) 

ALL OPTIONS '1969-70 .08 3Sl 80.4 

197~74 .20 ~SJ 18.5 

TOTAL .lS 736 79.4 

63 

SD 

S.02 

6.4S 

S.7S 

9.71 

8.39 

8.97 

6.71 

9.33 

8.26 

9.1S 

13.01. 

1].55 

8.78 

10.76 

9.91 I 

I 

I 

l 

~1 I 

' ' 
Compara~le data for a group of n~ ATC trainees enter~ng the FAA Acad~/ ir. 

1976 are prvvid~ in Table JH. 

TM!..E 38 

DI,'"I'RIK'l"'-:1> S"IN~di· .1ND c::E'PF'~c:t:; 

"1i.n:-co':L'PJ.nci<>Tm:>-!ffi'iE .,.,., - 1ou · 

(~ ATC Trainees - 100 !.,., g·7 keyed) 

~ 0<J$SCS 4-8 

Jan- Jl.:ln& 19?6 

'lbtal te.tmiiiai an<! 

»=e-tc by s~ 

Aviation eJq:erie."'lle 

~ aviation ~enre 

HiOOrity 

N:m-Mirori ty 

Teiminal 

Fmoute 

!hrrelati01 with 

Z Lab So:trc 

.40· 

.27 

.31 

.11 

.)I; 

.29 

• 31 

.32 

~ ~ 

BOJ 67.3 

)4~ 76.6 

.(S2 60.2 

700 68.1 

101 57.5 

86 6'7 • .5 

710 68.7 

372 70.9 

431 64.2 

SD 

16.42 

10.88 

16.23 

l£. OfJ 

15,62 

15.51 

16.06 

15.44 

16.33 

As would be expected, ~ean scores on OKT for this trainee group were l~«er 

and the- standard devia~ions higher than fo_r the _more experie:1ced facili_ty 

laC specialists hired in 1969-7Q or 1973-74. Correlations with the nonradar laboratory training seores (Z scores) for the Acad2my were highes~ 

for those ~ith· prior aviation experience and for m1norities. Correlations 

fOJ;_ the terminal and enroute optiC?n were also· among the .highest found for 

variables in the. e)Cf.erimental test ba-ttery. 

Tlie ATC oKT· lOlB was -~lso· evalu_ated in relatl.pn to present m~t.hods for 

gr-anting extra credit for aviation r~lated experience. The ATC Occupe::~ional 

Knowledge Test 101B was administered to 784 ATC trainees_ who entered the 

FAA Academy's 16-.week A'It training p;ogram between .July and December, 1976. 

All trainees completed a pre-employment questionnaire. Based on respo~ses 

to the questionnaire, th~ trainees Were assigned to one of three experience 

grC'.lPS in acco"rdanCe with evaluations of claimed exoperience as made by the 

Office of Personnel Managem-ent ra·tinq proceOu.res. It was found that while 

scores on t:he OKT were: highly cor!""elated with claimed experience ·( .64}, OKT 

had a higher correlation with successful completion of the non-radar lab 

(. 251 than did claimed experience ( .12) • lt was determined that use of an 

OKT sco,t;e of 75 or above· ·to assiqn· extra credit would resul~ in a failure 

rate of 3.1 percent for those receiving· credit, while use of the present 

experience rating woui~ result in a failure rate of 7.6 percent for 

new hireS who now receive extra credit for experience. The results held 

up for a cross-validation sample of 432 trainees who ente-red .th.e Academy 

during 1977. The full results of this analysis of OKT is provided els~Nhere 

In order to obtain data on the ATC applicant gcoups within available testinq time, a 60 item form of OKT (lOl-C) was developed from form lOla. 

~is was administer~ to ~ groups of ATC applicants during September• 

November 1978, t;ogether with parallel forms. of HCAT after they had canpleied 

the existing O,PI!rational ATC t:est battery used by OPM. One g.roup cons-iste~ 

of 5331 scheduled apPlicants. 'the- second grOup was 66-9- "walk ia" applicants 

64 

'· 

I 

: 

I 

L, 
' 

.~"' ..... I 

\ 

in testing sessior.s .J.rrar.gt...-~ to encourage women ar.d minoritic:::;. t.o ar;.ply for 

ATC work. Mean scor~s for the schedul~d applicants on the 60-~te~ OKT (l01C) 

was 24.9 with a Standarj deviation of ll.SO a.ftd fo.r the "walk. .i.:-t" gro~p, 20.1 

and 9. 71 respectively. 0}-..7 Form lOlC was ~lso ad."!'linistered to groups a: ATC 

trainees at the ATC Academy during 1978. From these analyses it was concluded that the ATC occupat_ional Knowledge·Test Fom lOlB ha.s hiqh reliability 

(.91) and that t..~c slight loss of reliability with reduction in test length 

would not be of consequence. 

Oevelopoent of Parallel Test.For.nS 

The purpo.se of the Olcr' is to define the domain of ATC Knowledge th~t is 

demonstrated by applicants for the occupation as a basis for granting extra 

credit to those who successfully pass the basic ATC selection tests admini-. 

stered by OPM. For operational usage, multiple forms are needed to meet 

retest requirements ar:td provide some ca{:<ability for control over compromise. 

~ essential step in the deVelopment of alternate forms was to define the 

types of ~tems and number of items of each type to include in. each succeeding 

fo~, to structure the domairi·of knowledge to make it consistent with the 

knowledqe l;"equ~rements of air traffic control. This was ac.::omplished pr-i .. 

marily by dete~iriing what types of knowledge were ~etained after training 

and earlY apprenticeships experience and were known by nearly all full performance- level controllers. Table \tl identifies the steps involved in de..termining the final domain of ATC occupational knowledge incl.uded.. in O!CT a.."')d 

provide a basis for de~eloping fut~e co~parable fo~s. 

As the Federal Aviation Regulations state tha-t, to become certified as a 

Terminal ATCS, the person· must pass a wr-itt.en k..-.,.owledge test or. (l) Flight 

Rules in FAR Part 91 (Air Traffic Control Rules), (2) Termirial Traffic 

Control Procedures, (3) EnRoute Traffic Control Proced~es, (4) Communications Operating Pr.,cedures, (5) Flight Assistance Service, (6) Air Navigat·ion 

and Aids to Navigation, and (7) Aviation Weather, tAe FAA Basic CertificatiOn 

·Test was des!gned to test knowledge in each cf those subject mat~er areas. 

The original form of the certification te~t is listed first in Table 38, and 

the table shows that an equal percentage of items (14 percent) were allocated : 

to each subject Diatter area.. For development of the OKT, radar questions were 

added to the item pool (line 2), and the best items selected (lines 3-6) to 

measure.cartdiateS' possession of the knowledge required. The ~rcentage 

allocation per subject matter area is presented in line 1 for version 

1018 of the 0~. Each of the seven subject matter areas is further defined 

by subject sub-elements which· are .identified in Table ~0. Table ~ 1 shows 

the classification of items in Form lOlB by el~ent and sub~element. Numbers 

in the cells are item number. If the item number is underlined, that item 

vas found to be known by 90 percent or more of the 1975 sample of 50 fullperformance level Air Traf·fic Controlle-rs. 

65 
i 

TABU: ]I) 

'"" ocd.'tA:OfONAl DICIW'....t:DC.t:_ ttr. - s~ Ko\T'!'r'.i t:.n1Dr:s NJCDtr Of' ITDu. - """' """""" A.llrJICII't. 

1nflt9ht <-..u- Tb;ht A!.t 'l'taffl.o;- cat~ons biiUtuu::w lllav~qat.J.on 

·-· ., Tra!!lc Control ()perattor.a Sarvlc• ' A1da <o Avt&t!en o< Po= ::! Cit '!'•st. ,_\diU Pr'ocl'd\l-r•s Proc..siU"•• Procotd;u:e! Proc•d~•s NaYlOAtlon 1. "••t..,~tr ltCII,_ Ono;1n.1 r.u. kuc , .. "' "' , .. '" , .. , .. "' C•ttlho;-auon 'l'enc 

au ,_ 

'· ..... " "' 1ll " "' l!t "' lH '" 

•u ,_ ,_ ron 101 - .... 10 " " 1l ' " • .. All 

.. Tor. ··-101 • l:Jfp. "' • " " ••• ll ' " 

It-_ a_,. tty 

to.1. or ~;roo.ap .r f\1.11 Perto~c• 

'--' , ... 5e. J.9."!'SI 

•• ,_ 101 - t.p . .. • " u ' " • " It- ktlown Of 1~ 

ctt Cto.q ot hil 

fertoriauc• ~~ ..., (~16. 19101 

•- f'on loa ll " " " ' " ' lOO au .. _ ,_ ._ O,.l'aU-..l l! .!2 l! ll j Q 1 '" 

.. _ taaC-""'-4) 

S~~>•l-ntt l lO ' • s ' J ' , ' l • • u ' lS s l • 0 • 0 ' ' l , 
C'.AH().'fl.H":' (fl.Nl-!\-:-;.; A.'\r! $-l'l\-fL~~i~) (If 

Atr •'t\11 .-.:lv~Al. ·KN1'>."Lr~:t 

1. All\ TRAFFIC 1\tl.f~ 

{)1. Pert~;~lnin~: h• AircrHft 

0:'. PrrtailliU); l•' Atn;-,w 

0.1. 1'\•rl;un!n~ !•' t\U·Sj'•1n' 

0:.. Pt-ttllln11~ h' flvtn~ c.•ndtt!••n!< (Kat in):.~>, \.\·atho?r, Si't't'd 

~..IIU'\JV~·r,;, Al t II uJt•, f lij.:ht l.t>\'••!) 

05. ft'rt.li.nin•· t,• \'}'t'tatit•n.t~/lnstru.:-th•ns 

-·, 

Ot>. Pt'ttainin..: t,• A.h~l.nil'trati\'t' ~t'!osa~ii'S (Kt•J"e>Tt5, PhrascC>lc>.,:~·) 

Oi'. Pertaininf h' ~OIIt·ty/t:.mt•rt:o'ncv 

01. P~rtatntn.: t~· Traffic: lnt"rll'l,9t1"n 

02. Pe;taininr: tt• T1afftc !'attrrni!O 

OJ. Prrtainill.j.':. tc- Rum.: a,- l'i•rrat i,•ns 

().;. l'rr.taint·r.r h•·Flit-:ht ""-'\'t'lll<"nt (D.estinatiC'n Chan~·es, Arrivals, 

Dt>partiii(';HI') 

OS,. Pt"rtulnin$: t.• Adt~anililf,ltive H.-slj.a~efi (Phrase,•lur;y, Auth••rizatiC"u, 

Offh:lal 1<-t"r••n,;, O..•cun"-'t1ts) 

06. rrrtainii'IJo":. to..• Mt'tl"re>lCif_iclll ln!C>matte>n (Cciliojo:, \'1sil>iltty) 

07. Frnatoin;:: fC' S..!lrty (M~·t<~~o.a~··s, \Oarniu;:: ~dee~. Collisil)n~) 

3. HH"t l GIIT TRAFfl C CQ~':fRl'l PROCtflt'RtS 

01. Pt'rti!.lntn;:. t(' ~t·~·aratt••n (\'l'rtica-1, Latt·r;~l, t>tr.) 

0:'. Pt'rto11111'lt: t..• Cl.·.u.1~n· (R,•utill):' 

OJ. rertatn~11,-:. t..• ~·-rr••ach (fh_~h: l.t'''l'l, lk•ldinf::, Alt. Scttin1:) 

o:.. Pt•rtl\iutn~ h' R1d.n (CL•ntl\'"1, ldcnt, lnt~·rfl"'retlrt•, Traffic lnfontation) 

0~. Pert~t!:dl:'~ t~• 1'11~ht lnf,•l"lll,\tiC"n and Dilt-.J. 

01. Pl'rt.:tlnl!if:. 

0:;!. PertaiQtn;:. 

OJ. Pertainin_;.: 

04. Ferta1nin~ 

O>. Pntatnt~ 

06- Pertainin~ 

t•• 

"' to 

"' to 

to 

FI i!'bl t'.''''t•mt.•nt 11nd V..•nt n•l Hessarc!< 

A.dn:lnh;tfllli\'t' M<'Sfi3~t'S (rhl"d!>C'~'l<..l~\·) 

Meter,•l,1j:.\" lnlcn::.Jt 1011 

·£1earanct" 

$.afety 

Service /Ka tnt ('nance 

!fl. niGHT ASSlSTANCf SER\'lC£ rltOC£DURES 

01. Pertainl~ to Search and Reacu~ 

02. P~rtaininJi to Overdut' Aircraft 

0). "F~rtaini~ to Emeygenr:y C.•muntcation~ 

04. PertaSnif\8, to Heter"'lo~ical lnfonaat.ion 

05o. Perrain-lni, tCI Tactlitit:>S 

06. PertainlnJ to rroceduYea 

6. AIR KA\"ICATION AND AIDS TO NA\"JCATION 

01. Penatainc to P're-qvenCiea 02. Pertainl~ to lnatr\mL'ntatit'R 

OJ. Per't.at.utf).~ to Pos-t t ion/Dis t a nee lot rect ton 04. Pertaint.~ to Navt~ation (Tillll't etc.) OS. Pertaining to Navitt•t ion• I Aids 

7. A\'lATlM\ \o'Q.na:R 

01. Peortainin& t• \'tstbtl 1 ty tc.-u tn~ 

02. l'ertaini:n~ to ~eathcr Conditions (T~pcYature, Winds, Tornad~~, etc.) OJ. Pertainin~ to Clc>uds 

04. Pert aininj:. to AJmtni&tr•ttve H~ssag~s (f.hy .. ~}ogy) OS. Prrt.ainfn~ to Forrcaat~/Rerorts 

06. Pntllhiing to keaponsi bllt t lea/rrocrlur,•s 

67 

,,,. 
TABLE 41 

OKT lOlB A ITEMS COD'ED BY ELDiENTS _AND SUB-ELEMENTS 

(Underlined Item Numbers MeAn 90% + Right for FPL C'L -1975) 

1 11 Iq-Inflight IV V-F11g.ht VI-Air Vll 

Atrport Traffic Cot:aunicatione -Assistance Navigation Aviation 

Sub ATC Traffic Control Operating Service &- Aide to 

Element Rulu .Procedures Procedures Pro-eedures Procedure• Navtsatiqn Weather 

1 45, 'l 3, !!.· 10 l2..t. 18~ g. ll· .ll ll· 43, 44 

~~: ~~· 71, 1l 11 81, ~ 

z 47, ~ !· ~ 14, !!· 41, 17 12· 86, !! 

3 1· l! ~. 67 IS, 2!· 13, 27. 21, 29_, 

s~. $4, 

30, 11· )~. 79 

_!£, 8-2. ----- 55, H 

4 1i.9

S3

37 •• 42, 62 1!,., ~. 2'!, ~ • 

87. 88 

36, 46, 90, 91 

56, 13, 76 ll· 2l.· 94, n 

9!,.. 19.2. 

"' "' s 39. 21. !! 60, 9·8 24 !!.- 31, !!· !1 

6 59, !!· !! !! BS 

7 2, 35 SB 

67, 63 

21 iteu 12 tteea 28 itUIII 12 items 2 ite~~~a 19 items 6 ttema IOO 

\ 
The subject mette.r ar.eas or eiemfmt.s ~q, perceptaqe allocations per area 

as dete-rmined al::>O.ve wer-e used to describ:e the types and num,be_r of i tern's 

per type to ·be in.cluded. in new alternat~ forins of th~ test. Fourteen hu.-1dred 

m'ul-tiple choice items we:r;e drafted a.I)d reviewed, and items selected fro:n this 

pool tO· create eight lOO,..itemparallel forms of the test. Twenty items were 

common to all forms. These fo~s were administered experiment~lly, item 

analysis acc'9Ulplished;. and the eighty· best items in each form identified for 

keyinq. These eighty item forms are identified as ATC Occupational K.nowledg·e 

Test 102A, 102B,. 102C, 102!!:, 102F, 102G; and 102H. Use of a Rights Only 

seorinq procedure ha.s been practiced anQ. .is recottanended. 

The forms have been administ-ered tq incoming students on the first day of 

training a-t the ATC Academy since November 21, 1978. The mix of trainees 

attendinq the Academy varied during this period with regard to levels and 

types of avi_ation ... re1ated ex:perienc:e. In order to control these differences, 

each group of students was given two forms of the Occup~tional Knowledge 

te.st. Consequently. the ccmparison of .results for alternate forms of OKl' 

can only be Rt.ad.e for t,hoae tests taken by the same group o.f trainees. 

The descriPtive· statistics obtained for each trainee group which took two 

di.ffer--ent forms of the test a~'e ·provided in Table 42. 

(1'1=97) 

Mean 

Form A 41.95 

Form c 42.55 

(1'1=67) 

Mean 

Fom A 38.91 

Form B 36.25 

(1'1=86) ~lean 

Form B 38.35 

Form o 43.45 

so 

14.05 

11.9S 

so 

14.61 

12.15 

so 

14.80 

12.75 

Tl\BLE 42 

AL'J'FRN.ll'l'E FO~ OF Oh'T 

(1'1=22) ~lean SO 

49.59 

44.73 

13.35 

10.84 

(N=l84-386) CN=69-701 (N=22) 

Mean so 

Fbnn E 46.49 13.80 

Form F 44.34 u.otc 

(1'1=407.,.408 

Mean so 

Fom G 46.28 13.95 

Form H 48.38 13.95 

' 

69 

Mean SO 

45.51 

42.60 

13.98 

13-.13 

Mean so 

48.18 

44.50 

...... 

I 

I 

11.66 

12.16 
The means and standard. Ue\·iat.ions for the two f_orms taken by each group can 

be compareCil since the exPerience mix for that group was common across bot;l\ 

forms taken. However, the data cannot be comp~ed for the same form taken by 

different groups of trainees siri.ce the experience mix was not common. 

The alternate forms do show some differences which requ~re minor adjustments_ 

by shi-fting items of a coMmon type but ranging in difficul-ty le'IJei, frOm one form. 

to another. 

Reliabi~ity estimates for the variou~ forms have been· derived and are presented 

in Table !~ L These reliabilities are considered- satisfactory for operational 

testing use and administration. 

RELIABILITY OF AIC OCCUPATIONAL KNOWlfOCE. TEST 

OKT 102 Tak.l!n 

Testing :late Studer)t Ct"-"'U£!1 Fiut ... Second 

3-6-7CI Enroute E,F,C,H E,F',G,H .904 

N•66 (80 item) (60 item) 

Enroute, ."o!t'lllinal, t,F,G,H t,F,C,H 

FSS, N•R9 (80 item) (80 item) 

.893 

Enro~tte, Terminal A,S,C,t) A:,S,C,D .877 

N•lSJ ( 20 c.ore (iQ core 

items) i j:ei!IS) 

70 
STUDY OF ~TC JOB APPLICANTS • 1978 

Objectives. The re:;ults of the an_a·lysis of the experimental. tests for a 

group of 1827 new ATC trainees at L;e FAA Academy·during the period 

1976-1978 supported the conClusion that a test battery comprised of 

the Multiplex- Controller Aptitude Test, Abstract Reasoning (CSC-157), 

and Arithmetic R·easoning (CSC-24) would provide an operational selection 

test battery for the ATC occupation with significantly better prediction 

of success in ATC training. 

This conclusion, together with the development of parallel test _fo~s 

of M~T made it desirable to administer these tests to a new group cf 

ATC applicants in orde.r to· compare the results of the propo~ed test 

battery with the existing esc tests with resp,ec.t to the composition of 

the two groups who '"passed" each test battery and t.he effect on score 

distribution within each group. A change in policy also made it possible 

to obtain race, sex and ethniC data which had not been possible with 

the 1976-1977 applicant group. 

Arrangements were made between FAA and the Office of Personnel Ma.."1agement 

to administer the parallel forms· of the MCl\T and a GO-item' version of· 

the ATC Occupational Knowledge Test when the present CSC tests we::-e 

administered during September-November 1978 ·to ATC job applican.ts .. 

Sample Description. The present CSC te"Sts, together with M9AT and OK'!' 

were. admitl.istered to a total of 6,000 ATC job applicants. ~('!W 5,331 applicants 

were scheduled for examination through established OPH procedures·. In addition, 

FAA had arrang_ed with OPM for a series of "walk-in" tes.t sessions which did 

not requir·e advanced scheduling by applicants with OPM. A total of 

669 applicants took the test on a "walk-in" basis. T-hese 11w-alk-in" 

sessions were established as a means of encouraging more wOmen and 

minorities to compete for positions in the ATC occupation. Table :,!, 

provides the distribution by race·, ethnic groups and sex for 5295 of 

· the 5331 "'scheduled" A.TC job applicants. Table 4 5 presents these data 

for 664 of the 669 "walk-in" applicants'. 

I~!!-.!::_~ 

-r. 

~ ~ !!!£!! ~ Hisp. ~ ~ 

.... 42 (12\) 38 (16\t 6SO CSB\) 2860 (76\) 213 (79\) 21 (70\) 3!2.4 {7 2\.) 

w-.o ll (28) ll (24\) ~ (42\o). _!ll. (24\) ~ (21\) ..2. (30\) 1-411 (.(6\) 

TO'l'AI. so '") so Cl') 1113 (21\J 3775 (11\} ... (5') 30 (1\) 5295 (100\) 

1'ABT.E 45 ------

~ -

.!!!!!1 ~ Whit~e --. Hbp. ~ ~ 

M•n 2 3 14~ (50\") 1&0 (55\) •• (65\) 354 (53\') 

-·· .! .! .ill (50\) 132 (45\) 24 (24\) l ..!.!£ (47\ I 

TO'l'AL 3 7 291 (44\) 292 (44.\) •• (10\-) 3 664·(100\) 

71 
Ther~ are several· poin~s of in~erest regarding the racial and sex 

distribution of these tw' app~icant groUp$. For the "Schedul~d"· a:;?plicant 

group, (5295) 92,percent were black or white men and women, with 71 percent 

of the group represented. by white men and women. Generally meB represented 

between 70-79 percer..t of each racial ·group except for blacks; only 58 percent 

were men while 42 percent were women; almost twice the ratio of other 

groups .. 

For the "Walk-In" applicants the distributi.on indicates that the objective 

of having more minorities and women compete for the ATC oc-cupa.tion was 

fairly successful. some 47 percent of the total group were women in 

contrast tb 28 percent for the scheduled applicants and 55 percent were 

minorities in c~par~son to 29 percent of the scheduled applicants. 

Predictors. Tests used in this study were the existing esc selection 

test battery and parallel f9rms of MCAT and OKT: 

~M Test Battery 

CSC-24 

csc-s1 

CSC-135 

CSC-157 

CSC:-540 

Arithmetic lteasordng 

Spatial Relations 

·Following_ Oral Direct.ion 

Abstrac~. Reasoning 

Ai~ Traffic Problems 

MCAT OPM IO ODD/EVEN TEST COMBINATION 

Form 120 (4o6e) 

Form 130 (4e.6o) 

Form 140 (&o7e) 

FOrm 150 ('6e7o) 

Form 160 (7o4e) 

Form 170 (7e4o) 

OKT - 101c (6D-itell\ version) 

As in previous experimental testing of applicant groups, the MCAT and 

OKT te·st were ·administered .after completion of the regular CSC test battery. 

In addition te the tests, applican·ts completed forms provided by OPM 

on a voluntary basis to obtain racial, sex, and other information tor 

separate analysis by OPM. 

Critet"iQn ·and AnalytiCal Methods. Since thiS study_ er1:com.passed ATC applicants, 

it was not feasible-to establish operational· criterion. measures of validity. 

However, a number of statistic-al analyses were made based on pass/fail 

eligibili~y compa;ison, score distributi.on.s and mean· and· standard devia.tion:s 

on the vatl~s te•~ 

72 

1 

:; I 

·,.' l: 

' 
~~----------------............. 

Resqlts. Descriptive st~tistics for the "scheduled" and "walk-,.in'' groups 

were developed for each test. It shOuld be pointed out that of the 2€0 

"Schedu-led" applicants who identified themselves as "Hispanic", all but 

74 also identifl~d then!"selve~ with. another racial group. For example, 

139 were .identifi~ as "Hispailic~White". In thi_s analysis,. such cases 

were in~luded in the racial group_. Consequently.- 104 applicarlts in the 

•other" category s~ in Table 43 were comprised of 74 Hispanic· applicants 

and 30 of some othE:-r racial groups not identified. Descriptive statistics 

for the "Scheduled,. groUp of applicants are pr$:>vided in Table ~6 and 

for the "Walk-In" qroup, in Table 4 7 · (In Table 44, the 24 applicants 

in the "other" c:ateqory consist of 21 "Hispanic'• and 3 otherwise not 

identified.) 

TABLE 46 SCHEDULED ATC APP~ICA.\TS. 

AMtiUCAI< 

... , ... .. ,.,. ,.:~'i!,} NHlTE 

(W.foftl """" """'" ciio::-n-1 , .... Jir., (;;;-ni4) (II;'""TI)ll 

!!!!!! !A !!!!!! u,. !!!!!! ~ !!!!!! !:.£:. !':.!!! 1:.2:.. !!!!!! U:.. 

ere-,. .... ••• .... 7.t u.c • •• f2.4 '·' )6.1 ••• 40.7 ••• (""nl (U.ll (9. ,, (42 .21 (7.91 

-· U7.0! tt~~.J (4).11 17.61 

CS-51 .... ••• 28.3 ... 23.1 • •• 2,..1 ,_, 17.0 ••• 27.7 •• 1 

, ... "' U4.JJ (6.4) llt.6, (5,01 

·-· Ul.ll (6.2) (27. 71 IS.61 

~ 21.6 ••• 19.9 '·' 16.6 0.1 24.7 l.o 20.1 ••• :u.1 8.1 

INn) U6.71 (8.QJ c.a.11 (6.91 

·-· U6.51 (ll.ll (24.2) (7 ••• 

c-ore-157 21.0 ,_, )1.0 ..... 22.1 ,_, 30.9 ••• 26.1 ••• 28.8 .. ' (A;.1J U1.SI (9.3) (4::1.51 l8.9f 

'~"' •· (22.91 (9.)1 (32.1) 18.81 

CSC•S4B 29.-4 U.7 ,._, ~ 

11.6 22.7 11.9 ll.-1 11.7 25.6 u.s )0.6 u.s (2.).41 (11.9) (33.6.) lll.fil 

·-· (21.11 fll.t) U1.l) u;..t, 

'"'" Jl.l ••• .n.o ••• 22.1 , .. 14~2 0.1 .... ••• 31.4 

~I • •• 122.91 Cl.lt '"-01 CI.Ol 

·-· 121.U (6.1} (ll.ll ti.Jt 

- 24.J .... 24.1 U.J 17.4 .. , 27.2 11.7 2'1.1 11.8 .~ ... , 11.1 ~~ (11.11 ... , (21.11 U2.01 

-· US.SJ te.S) CU.OJ (I.IJ 

TABLE 47 WALK-IN ATC APPLl CA. 'ITS 

_,.,.,. 

=~ ULAN se· {!!:-;;:, ;;a;: I ...... (!!: 10) ~ (tt- 66t~ 

- e._o. -.!.:!:.. .... !!.!:.. !!!!!! ~ !!!!!! !.:!..:. !!!!!! ~ Cle-M .... '·' • •• • •• Jl.S '·' ... , -7.1 19.4 ••• "41).1 

...,.., '·' at-~ ••• Jl.l • •• JJ.o '·' .... • •• :11.1 ,_, .... 6.1 

CIC·US 1t.o '·' 24 • .6 '·' , .. , '·' ;11.7 ••• 19.7 ,_. lCI"f ••• QI:<-15-T JO.O 11.11 H·ol u.> zs.o ••• n.o • •• "·' ll.f ...... ••• c:ac-sco .... ••• 4S.l 1"2·.1 21.6 12."7 ll-4 u.s )0.4 U.4 l •• , u.s 

~- ..... . ,,, ,._ . ••• "·' '·' Jl,f ,_. n.t- .. ' 21.0 ••• 

- 19.2 ••• 111.4 ••• 111.2 ••• u.' 10.t" lt.S ••• 20.1 • •• 

73 
For the "scheduled" applicant group, the mean scores for women were 

exam.ine:d for both the "Blacku and "White't. groupS. to a.sse_ss their relationships -to the- scores of men in the same racial gr~up. Table 43 indicates 

that women score somewha~ higher on esc tests 24 and 157 than· m~n in 

each racial groUp, _bUt. .somew;:"'t lower on- CSC 51, 549, MCAT, and mer:, 

with the greatest difference between men and \J(Omen bein'g on the OKT. 

The most marked difference in te&t score~ between racial groups is for 

the "Black" and "White" groups, w.t.th the mean test scores for "Blacks". 

generally about one standard deviation be-low the mean for ;'Whites••. This 

holds for both men and women in each of the groups. 

Relatively few differences between mean scores of the "Scheduled" and 

"Walk-In .. groups are evident by racial groups. Within the "Black" group, 

walk-in applicants scored about 2 poin~s higher on esc 24 and 3 points 

higber on CSC-15 7. The "White" group walk-in applicants scored· 

almost; 2 points lower en MCAT and about 3 points lower on OKT. These 

lower scores may be a result of· the higher P.ercent~ge of women· in the 

"Wh1te" walk-in group (45 percent) compared to the scheduled grou~ 

(24 pe~cent) • 

With the applicant group tested in 1976-1977, two of the CSC Tests (51 

Spa,tia1 Relations and· 135 Following Ora). directions) showed marked ne-gative 

skew. rt was of interest to dete~.ne if this characteristic was replicated 

for these two test_s with the 1978 applicant population. Figure 4 provides 

a frequency diptribution for test scores for these two esc tests. It 

is evident that. again· they provide little differentiation between the 

.applicants. 

' -

-

FIGURE 4 

74 

., 
Vete:rans prefe_z:ent:e for the total group (scheduled and walk-in} was ident-it:ied 

by sex and racial groups. Table .:OtJ provide!~ the distribution by sex 

for 5846· of the 6.000 applicants. 

TABLE-4R 

_s_E_)(__!) I S.l_~_ll_l.clJ_()_:-;.::_I'Jif-Jij_l_'f_,!.'\)'J'_L l C.l.':l:_r.ROCP 

VET 

PI<EF 

None 

5 Pt. 

10 + Pt. 

TOTAL 

MEN 

240G (59\) 

1571 (38\) 

123 ( 3\) 

4100 (100\) 

~· 

1640 (94%) 

9~1< 6\) 

1746 (100\) 

TOTAL 

4046 (69\) 

1666 (29\) 

132 ( 2\) 

5846 (100\) 

Table 49 provides the distX'ibution of Veterans Preference points by raCial 

and ethnic group. 

TABLE 49 

VETER_""-.;s PREF-F.RiNCE 0 I STR I BUT!ONS 

VET. »>EPlON 

!!!fL. INDIAN AS!l.N· ~ !!!!m' !!.!E.t. cmu;R ~ 

No:ne 42 (70\) 40 {70\) 9~~1 nn1 2724 (68\) 223 (6SU 27 (84\) 4037 (69\) 

SP'l'S. 1 

~}30\) :~{30\) :nas (27,, 12!~(J2t) 9~]_(32\) ~}16\} 166-6 (29\.) 

10 + PTS. 41 ; 132 {2\.) 

TOTAL 60 57 1340 4017 329 32 SSJS 

Since Veterans Preference points axe added to the test battery score for 

those wh·o pass the test, it is evident that only 6 perCent of the women coUld 

benefi-t, and thus be ranked higher on the OPM reqister for appointment eligibility. This irt contrast to 41 percent of the men who could benefit from 

militar; service. 

Eligibility for Veterans Preference ·points by racial and ethnic groups is about 

eve~l~ distributed (30-32 Percent) except for the "Black1

' group (27 perceri.t) 

where the hig-her ratio of women applicants to men red aces the num.ber who receive 

Veterans Preference points. 

75 

I 

' I 

I 

i 

' I 

l 

l 
T4ble 50 ~.:ompares the total ATC applican-t q:r'?up to the civilian labor 

force (CLf) by racial and ethnic group based on 1970 census data with amendments through l977. 

TAHLE 50 

ATC Af'I'LICA~T~-Cl\'ILL\.'.: LABOR FORCE --------'-.-----------~---------

Civilian 

L<$>r \ ATC Force 

A.e,elicants ZUnerican Indi~ .3\ 1\ Asian • 8\ 1\ Black 

9.7\ White 24\ 85.0\ Hispanic 68\ 4.2\ 6\ 

In t"rms of ATC appHcant;s, blacks as a group are significantly overrepresented in comparison to their percentage of the civilian labor 

force while Whites as a group are underrepresented. As a group, women 

were also underrepresented; they comprised 28 percent of the ATC applicant · g:roup coa:.parea to JS percent of the c_ivilian labor -force. · 

Analysis of current OF!-J Test and 0f...7 

Next, the current OPM test battery scores were analyzed. in ~erms of 

"pass/fail" score distribution by ethnic or racial group and by men and 

women. Because of the snall number of Asian and American Indian applica~ts, analysis of minority groups has been directed to Blacks and Hispanics. 

Table 51 provides the results of those who took the current OPM ATC test 

With respect to 11pass./fail''". The minimum passing score· on the test 

battery is 70 which equates to the approximate mean score on the test 

~there <$>ut half of the total applicants pass and half fail.· 

\ 

Men 

Women 

Total 

TAHLE 51 

fAS_S/FAJL-0!'~1 ATC TEST 

P;rss 

Total OPM Test 

Aeflican:ts N \ 

4191 2236 (53%) .1785 799 {45%) 5976 3035 (51%) 

76 

Fail 

01':11 Test 

N ~ 

1955 {47%) 

986 (55%) 2941 {49%) 

·t-
\ 

Compared to the 7500 applicants tested in 1976•1977, this group had about 

the s.ru::qe pass rate for men (53 percent compared to 52 percent) and a socewhat high-2r pass ratio for wome_n (45 percent compared to 41 percent). 

Table. 52 shows th.e "pas!"/fail" rates by raci~l group.-

1'AhLt 52 - -·-· ----

P,\SS/Fl\!L BY kACIAl. CfWLT 

Total ~ Fail 

Group AE;~l..~..cants N ~ N \ 

White 4067 2556 (63\) 1511 ( 3 ,,, 

Hispanic 339 128 (38\) 211 (62\) 

Black 1407 264 (19\) 1143 (81\) 

Tot;al Sill ma (Sl\l 2865 T49\) 

In view of the disproportionate number of minoxities who did not pass 

the OPM test, an analysis of score distribution was ma4e by racial 

group (Table 53). 

TABLE 53 

OPN TEST SCORE DISTRIBUTION BY RACIAL GROUP 

P•i.l OPK T•at reM OPM Teat 

(JI - 2865} HI· • 29.:08) .. , .. ..,., . ~ 50 ·~ ~ 70-79 to-!!9 --"-

! ~ ! ~ ! ~ ! ~ ! ~ ! ~ 

..... ,., (4%} 406 llO'J ,. (23h ' 1128 (282:) "' (24\l "' 111\.l 

li.,U.lC ., (l!ll) 56 U"l ., {17%). 10 (21%) " fl1'1 " (5'1 

llaclc "' (321) 181 U1'41 "'' (22:) ' 189 (13%) 6l I 5'·1 ll uu 

~ 

.,., 

"' 

100' 

lt is evident from this analysis that for th~ Black group in particular, 

the hiqher fail rate is comprised of a large number of applicants who did 

very poorly on the test battery. The 457 Blacks who scored below SO on 

the test comprised 40 percent of the total number who failed to pass the 

test in contrast to about 30 percent for Hispanic and l2 percent for Whites. 

77 

I 
Figure 5 provides a gl-aph of the OPM A'I'C_ test score distribution fo·r 

selected racial groups, 

FIGtJR£. 5 

--~ 

' I ' 

-~~ -----"""--. -- .......:t- w------· ..... ~ 

~~~-

' . · ~"'2L .. 

. ;· 

., 

·--

·1( "'II' !fl tt M .,. 2t! :zc lrJ M !I! a.-.,. 

aa•Ar.~ 

Table 54 compare:; the d~str'ibution of A'I'C applicants, ATC applicants who 

passed the OPM test and the Civilian Labor Force (CLF} . 

TA!l_L_£__,>_4_ 

CIVIL LA80R FORCE-ATC Al'PLJCA.nS-PASS GROUP -----·---

Civ. Labor \ of \ cf Applicants 

Force ATC Applicants in Pass Group 

Americ-'ln Indian .n 1\ 1.0\ 

Asian .8\ H 1.1\ 

Black 9.7\ 24\ 8. 9\ 

White 85.0\ 68% 84.8\ 

Hispanic 4,Zi _a 4.2\ 

100.0\ 100.0\ 100.0\ 

78 

•, 
~ 

Men 

~ 

hhlte 

Asian 

Arrerican r~anl Black 

Adm.ini$t-r~;tion of the Al'C Oc.eupa.t.ion~l !::nowledg-e Test (OK'l'} prc;>vided the 

first oppor!1mity to evalua.t~ the results for a siza.bfe:. sample of ATC 

applic~.:'lts. Tr..; OK't used.wa~ version 101 C, a 60-itelu test. Ccnseq-.;ently, 

the results of thi~ test were review.·ed for th~ total grOup, men, wa:::en, 

racial and· ethnic qrouys and relationship qf the resu~ts· to thos•· applicants vith·veterans preference. 

Tables- 4) and 44 provided =eans and standard deviations on the OKT' Test 

results for the various groups. The •walk in" group of applicants t~d a 

lower mea.n scOre- tha.n the "scheduled'" group; 20.1 ·in co:a.pa.rf,son to 24.9 a.11.d 

thi:s diH'ere."'ice w.:s.s evi:!c..ot ir. .111 r.l::i.tl ;roup::; Cxccpt fer !;;1.::..:-ks ~.-!:ere: the mean score was 17.4 hcheduled;) a.nd. 16.2 (walk in). This would i~icate that there was reletively little. difference in ATC related knoWledge· for 

blacka of either c;roup but that other applicants in the *Wa1lt rn• c;roup had 

lea-s ATC related knowledge or expe.rien~ that those in the •scheduled• 

group. In vi_ew of the objective. o~ the "'Walk In• recr...U.tment program-. these 

differanc.es ~uld be expected .. 

Table 55 provides the descriptive Statistics for men and wo~en on 0~. 

Hen 

TABLE 55 

DESCRJP_TIVE STATISTICS, SEX ts. 01\T 

SchedlJled Walk In 

..!!!!.. 

(383S) 

(1473) 

26.9 12.2 

19.8 8.7 

..!!Q. 

(356) 22.2 

17.6 

~ 

11.2 

6.9 

Table Sh prQVides the OKT score diatributidr.. for men a.nC: "·:men. and for racial 

qroups. In deriving this_ distributi.o.l. the raw scores we-r"J cor.verted to a 

sc-Ale of G-100. I·a this· analysis, the W:spanic group was .li:itribt.~ted w-ithin 

the racial groupe shown. 

TiiBu: 56 

D:!stril11Jtion of OI(T Soores 

Below 

N 15 ~ 25-34 ~ ~.2.:2.! ~~~ - Abo<., - ~ ~ -22.... (4191) 3\ 11\ 27\ 20\ 9% 10% 9\ 8! (1785) 5\ 3% 20\ 43\ 19\ 5\ 4\ 3\ 1\ • 

(4242) 2\ 91 27\ 22\ 10% 10\ 9\ 7% 3% 

( 140) Jl 14\ 351 18\ 9\ 9\ 6% 5% 1\ 

11443) 8\ 32\ 39\ 13\ 2\ 2\ 2\ 2% • • less than 1\ 79 
/ 

Approximately 20 percent of the. men scoted ·ilbove 64 as compared to 4 percent 

of the women; 19 pe.rcent of the white .. grotip; 12 percent of the Asian/ 

Aarerican Indian·; and, 4 percent of the blaCk group scored above 64 respectively. 

Previous studies with ATC trai-nees at the AcademY- as Well as developmental 

and Journeyme·n ATC specialists. provid~ a significant correlatiOn between 

scores on the OKT test and succesS in ~C ~raining and work. These studies 

also supported the conclusion that the OKT provided a better basis for 

gra.."''ting extra credit to ATC applicants who passed the OPM ATC Test than 

the present rating guide used for this purpose. Consequently, for further 

analysis, this applicant group's OKT scores were given the following "extra 

points": 

OKT Score.s 

6·5-69, 

70-74 

75.-7!1 

80+ 

Extra Credit 

3 points 

5 points 

10 points 

15 points 

These point values are similac t6 ~h~~e presently p~ovided by the OPM rating 

guide for specific types. of ATC rel~ted experience which now provides 5, 10 

o~ 15 points· devending on the specific experience. 

In most cases, the types of ATC related. exp-erience for which an applicant 

is presently granted extra credit is ·obtained through military service 

and training. For example, Ai~ Tra.f.t-i!=' Control experier.ce w~ich is gr<il\ted 

the most points (10-15) is obtained -through. militacy experienpe_. This also 

tends to be the case, a-l.though to a le·s_s\!r extent, f_or pilot and other types· 

of credited experience. 

In order to assess the relationship between military experience and OKT 

scores, an _analys~s of th0se applicants with veterans preference, their 

OKT scores, and the extent to which they could earn extra credit was 

completed. The results ar~ provided in Table 57 below. 

TABLE 57 ----

Veterans Preference 

N<> !!!. ~ Credit N ' N ' ~ 

65-69 3 ptS 103 (4!;\) 126 (55\) 229 

70-74 5 pts 76 (33\) 153 (6"l 229 

75-79 10 pts 61 (24\) 190 (76\) 251 

l•O+ 15 pts . 26 (l~l 23~ (90\) 259 

26ii (28,) 702 (72\) 968 

80 

,. 

. ' 

1 

" 

' 
\. 

Prior an~lysis of the relationship between oKT scores an~· "p~Ss/fail." of 

ATC training for 182.7 students ip.dicated that the largest drop in total 

fAil.;u.;-e_ rate was betw~en those trainees who scored between 60-.64 (29. 6 

percent. fa-ilure rate) and 65--69 .(20.1 percent failure r!}te). Consequently, 

for this analysis comParisons were based on providing 3 points for those 

applicantS ·who scored between 65 and 69._ About 16 percent (968) of the 

total 6000 applicants could have ~en e~igible foi· additional point c.z:edi t 

based on O!q' test scores. A total of 702 (72 percent) also could receive 

poin~s for veterans preference. The iricreasing rium!:ler o'f applicants w~th 

veterans preference who scored .. highest" on the OKT indi_cate that most of 

them would probably have had military air traffic control experience. 

In contrast, tho$e applicants Who were not veterans com~tituted 28 percent' 

of the total group whO could receive extra credit based on OKT scores. 

While applicants -in this gro.up score "lower" on OlC'l', use of this test to 

grant extra credit would broaden the opportunity for non~eter~s who 

acquired; ATC related knowledge througl) other experience to al~o qualify for 

extra credit. 

Under current OPH ratinq and ranking proceCures, an applicant must make a 

pas~ing score of at least 70 On the Present ATC test_ battery before 

additional credit is given for veter~s preference or aviation related 

experien~e. Of the total 968 applicant~ who co.uld· be elig-ible· fot" extra 

credit·bap~d on OKT scores, 272 (28 percent) scored below 70 on the pres~nt 

OPM ATC test ba~tery-.. Table 58 shows the distribution by "point groups". 

TABLE 58 ------- -·- ----·- ELIGIBI LIT\', VETERMS PREFERENCE, AI' I ATJO)i_J~;I.,\J:Fil___f~\f'I~']:f~'CE 

Total 

Extra Failed OPM !lligibla for 

Points ~ ATC.Test ·Extra Cz:oedi t 

15 pts 259 58 (2211) 201 (7811) 

10 pts 251 74 (29\) 177 (71\) 

5 p.ts 229 62 (27\) 167 (73\) 

3 pts 229 78 (34\) 151 (66\) 

96'S 272 (28\) 696 (i2%) 

The 696 applicants who passed the OPM ATC test and would be eligible for 

extra credit based on on scores, represented 23 percent of the 3'083 applicants who passed the test. 

Next the data were examine:d in tenn:s of the ATC test score distribution and 

the effect of adding additional credit for veterans preference a.nd aviation 

related experience on the score d'istribution for those who passed. Figure 6 

provides· the distribution of ~aw QPM test scores for the total applicant · 

group. 

8l 

. ' ' . ' \ 

. ' 
--~--- ·- ----- ·--

~----- --.--

=:=;._~~- :~::-::::-:. 

- -s---- ----- -- -+ .. 

-..... :: -·j .. . .., . 

--"- -- ___ _£ 

\ 

FIGURE 6 

--- ·--·----- ----- --- -------- -~---- . ---------

. --------- --- ---

--------- 1"'0'7'Wtl'f?rA"'Crr~.-r------- --~--

c;...,. ..,...ecoo --- . --- -·----

/ 

-----~----

-·•··- ..._ _____ --- ---_ -

f---------

1 -----------

-- ·------------. --· -----~- -------------- -: ·-- ---

-'"-•--= 1111:1..,. 70 N &Q er_.a;z __ .,v.,. ______ _ 

-.------ ~'"::!!~=-~~~==-

Figure 7 provides. the score distribution for all ATC applicants ~ho passed 

the OPM test after adding extra credit for. ve-terans preference and OKT test scores. 

FI;;t.,'RE 7 

. 7o 11' AD M_.311Lft"--.Lft..t. ___ _ 

Oaf*' •rc T•~· 5coy 

It is evident that granting extra credit for veterans preferehce and OKT 

significantly shifts th~ distribution to higher scores, with a total of 

89-9 applicant~ scoring 90 or above ( 2'9 percept of those· passing) compared 

tQ 500 Cl6 percent of those pa-ssing') before· adding· extra ctedit. 

82 

•. 
\ 

Figure 8 shows the distribution of OPM scores after adding extra credit 

for vete1;ans pre.ference a,nd. OKT test ~cores for men arid women· . 

"'~ "I 

. FIGURt 8 

' I I ' 

' I 

' ' 

I ' 

' L ' ',., i" ' I ~- _j 

I ' i I 1": I , ' . l I 

While a sign~ficant Shift to hi9her score& is apparent for men, there is 

relativ.ely little change. for women. Of the twO extra credit factors·, 

Veter,ans preference accOunts for more of the shift than OKT credit. 

For every wom.n who receives extra credit for being a veteran~ 16 men 

also get veteran's preference credit. By comparison, for every woman. 

who would qet extra credit for aviation-related experi_erlce based on OKT 

scores, 9 men would also get extra credit. While both factors result in 

DOving more men tban women to higher total score ranges for appointne~t 

consideration, almost twice- as many men are moVed up based on veterans 

preference than on o~ scores. · 

The effect _of qrantinq extra credit on score dis'tribut.ion can also be 

examined: in terms of the r·elative representation of men, women a.nd minority 

qroups in various score ranges with and without the extra credit. 

Ta))le 56 shows the _pe~cen.t;::~9e of men an4 women in e~ch of four score range 

groups based on r4W O}'M te.st- s~ores .. {be-fQre v~t:erans preference o~ Ol<T 

ext~a credit is applied) and compares those_ to the percentage of men and 

wom'en. in each group after applying- extra. credit. tt .Should be noted that 

by adding. extt:a credit it would be Possible to have a- maximum total score 

which exceeds 100. · · 

·-------I 
70-79 

Raw t\it~llS.core Credit 

(N=1440) _(N=1076)_ 

Nen 747; &n 

Women 26% 33% 

1007. 1007. 

80-89 

Raw tiitJ;-" 

St...'re Credit 

1_N=ll06l _( N = 1 0.§.0_ 

73% 73% 

27% 27% 

100% 100% 

90-100 90-99 ~-Ra._.;-- ---·\iTt~il --

scu l"e Ctcd it 

_(N=4RJJ (!\:"""'~~-~} 

__ ?)_~ 

1oo;~ 

100<- \/it_h_ 

Credit 

_(~':1__:'9_) 

IOo;; 

Based on th~ raw O:n-f test scores, women comprise 26-27 percent of each of the 

score range_groups as compared to men. Since women comprised 26 percent of 

the t_otal pass gr0up, t.hey a-re also proportionately represented (as compared 

to men) in each of the score range groups. However, t,.•hcn veterans _preference 

and OKT credit is in-cluded, the percentage of women increased to 33 percent 

in the 70-79 -score group; remains the s<llfle in the 80-"89 group; and drops to 

21 pe-rcent ai1d ll percent in the t_wo "highest" scoring ~roups. Table 59. 

provides a quantative mea-sure of the extent to which veterans preference and 

aviation related experience, as measured by OKT, differenti.::tlly affect men 

and women fo~ appointment consideration. 

Table 60 compares the same 'data by minority groups. 

70-79 

TABLE 60 

OP!I TEST SCOR·E GROUPS 

(Pass SampDo- N=2948 

80-89 90-100 90-99 100<- ------- Augmente.d Augmented Augmented Augmented 

Rating Rating Rating Rat~ng Rating Rating Rat in~ 

Group (N~l387) (N=l042) (N=l079) (N=1058) (N=482) (N=622_) _ .2'::Z1.6L 

White 81% 80% 90% 89% 94% 93% 93% 

Hispanic 5% 5% 4% 4% 4% 4% 4% 

Black 14% . 15% 6% _I! 2% -~ 3'' 

100% 100% 100% 100:: 100% 100% 100% 

In the case of Blac_k and Hispanic minority groups, lt is evident that granting 

veterans Preference and OKT credit has very little impact on the propor.tionatt· 

represent-ati_on within each OP~I score group· for those nrplicant-S who pass the OPN 

test .. Ovefall, Table 60 indicates that ven•r:ms preference and CFt;dit for 

avi-ation experience affects each of the racial groups t..•ssc-nt ia"J 1 y the same \•:i l h 

res·pect t-o their score distribution. · 

84 

.I 

:~ 

' ' 

'i 

•• 

'' 

. '' 

. I ' 

' t 

\ 

i I .-N 

,h:_ 
Analys-is of Experin;lental ATC Te-st .and OKT 

Based on pr-ior researc;;: condu<;ted by t.he Civil Aeromedical I!lstitute on 

a qroup of 1827 ATC traine~~ at the ¥AA Academy, it was shOwn that an 

experimental ATC test_battery ..;omprised of the Multiplex Controller 

Aptitude Test (MCAT), the OPM Test 157 - A.b.stract Reasoning, and th~ 

OPM Test 2~ - Arithmetic Reasoning provided a high multiple corre~atiori 

(R=.5.4) with ATC tr~ining laboratory scores _(19). It was also shown 

that th~ use· of an ATC Occupational Knowledge ·test added. significantly 

to the statistical correlation (lb.) un. 

Since parallel forms of MCAT and ~: 60-ite.m version of OKT {101 C) were 

administered tog~ther with the current OPM ATC test, it was possible to 

construct an Experimental. ATe Test Battery comprised of MCAT, Abstract 

Reasoning, Arithrneti_c Reasoning, and OK'l' and compare the-se results to 

the ourrent OPM test battery for the same 6000 applicant group. 

Conseq.ueiltly, the analyses previously discu.ssed for the OPM ATC test were 

also completed using. this Experimental ATe test battery. Each of the 

thre~ tests (excluding OKT) were· equally weighted in deriving a compos_i te 

score for the exper~ental ATC test battery. 

The tcilile numbers used in this section wilJ. be the same as those used in 

the OPM test analysis followed by_ a suffix "E" where the same da.ta is 

presented for the Experimental ATC test. 

Table 0! provides the re·sults of using the Experimental ATC test with 

respect to "pa!'>s/fail". A minimum raw score of 100.47 was used which als-o 

equates to th.e" mean scor-e on the test batterj where about ha.lf of the 

applicants pass and half fail. 

!l!!!!t 

-

-Toed 

TAIII.L 61 

!:_ASS IF,\ J~i;__-::: )'XP_I-;!l_I~_IEti:lj~lc ATC ·r~ S~~ 

•••• Patl 

Total !xp. ArC· Ten ~· JJC T~.•t 

~21tc.aou ! .l!l. !! .l!l. 

4191 zm ('51). 1179 (4)1-) 

!.!!! ..!!l mll ...ru = 

5976 Jl64 UlZJ la12 (47%) 

With the Experimental ATC test battery (excluding OKT), a· somewhat higher 

perc:eitt of the total applicants passed compared to the OPM t"est (53 percent 

compared to 51 percent). 

With respect to sex group_s, both uten and women passed the Exp~riment_q-J ATC 

test at a higher rate, (55 percent men .compared to 53 percent on the Ol'M: 

test and 48 percent wetilef'!. compared te 45 Percent with the OPM test). How~ver, 

the mean raw score on the Experimenta-l ATC test waS usce.d as a ."pass". score. 

On the curren·t OP'M ATC test the mean raw sCore for this appl-icant group 

was 207.1 com-pared. to the established pass. scor'"-e of 2~0. Consequently, the 

''"highe·r" pasS rate is primarily a result of the pass scores utilized. 

85 
\ 

Given this general comparison of the two tests, one- key point. then is 

the· extent to which the individ'u41 applican.ts who passed the two fests 

in fact differ. Table 62 shOWs th·li number of applicants who: (1) paSsed 

both tests; (2) faile<l both tests; (3) passe<l the OPM test but faile<l the 

ExpeJ:'i..mental ATC test; or {4) failed. the OPM test bUt passed the Experimental ATC test. 

TABLE 62 

TJ:S_T_RES_U_l~f- _l_l_l!:_I'_(I~iS((<i_:~~)_xn II' I DUAL APPL I Ct.!':~ 

Passed Both Failed OPM/Passed ATC 

(1) (4) 

2727 437 

(46%) (7%) 

~assed OPM/Failed ATC Failed Both 

(3) (2) 

308 2504 

(5%) (42%) 

Pass OPM Fail OPM 

3035 2!!4£ 

Pass Exp .. 

ATC 3164 

Fail Exp. 

ATC 2812 

TOTAL 

5976 

From Table 62 it is evident that using the two diff-erent tests affected the 

pass/fail status of 745 (12 percent) of the total applicants. In terms of 

the Experimental ATC test, 437 different applicants (1-4 percent of those who 

passed) would be eligible for appointment consideration !->1ho would not be 

el1.g1ble· base~ on ·the current OPM test. ~'hile most of the shifts hetween 

pass/fail on the two tests occur around the passing._ score of 70, some 

applicants who scored as low. as 45 on the OPM test, passed the ATC Experimental 

test. Conversely, some applicants who scored as high as 85 on the OPM test 

failed the· ATC experimental test. 

Table 63 shows the pass/fail rat:.es on the Experimental ATC test by racial 

groups. 

86 

+ -- •• ----:--- -- -----,..----·-'--:-· ---· - J 

. ' 
TA!l.LE 6J 

PASS/FAlL RATES - EXPf£!!_!~="'\lAL RACIAL GRllt'PS 

.... total 1•U 1:9. Al't Tut ~ S:el:te. !!J!. .UC T II·H .. u ! .m ...... ! .m 4067 2671 (Mi) 1119 {)U) -· , 141 (42:%) 1 .. (S8%) II.ack. !!!! ...!.!! .l1!tt Toc&J.' .illl jl~%1 5113 l07> (53%) Z1ll {47%) 

For the Experi.mental ATC teSt, the perc-entage qf the l..'hite group who passed 

was somewh~t higher than on the OPH test (66 percent compared to 64 percent) • 

. The Hispanic group also increased in pass rate (40 percent compared to JB 

.percent} and the Black group p.ass rate decreaSed sJ ie:htly (_1 R percent conpared to 1~· percent). 

Again·, because of the disproportionate number of minorities \oo.•ho did not pass 

the Experimental ATC test, analysis of score distribution is proviQed in Table 64. 

.. ~0\lt 

TABLE .1>4 

SCORE DISTRJBU17iox5-:EXPERWENTAL 

(N•l7l8) ,\TC TEST 

Fdl bp" ATC T .. t 

,. ! 1"44• t.p._ 4TC te."'t 

so;. 59 60-!9, : 

' Croup !! ill ! .m 1G-79 ! .m : h1te J9 ( 11:) "' ( 8%) l021 (251) ! 

ilp&Atc : 2l ( 11) ' 74 UJ%) 101 001) 10<k ! "' ' on> .. , (l)%) 441 (32%) ' i 

' 

...... !! 89 J.!) ~ .m !! .m Total lr58 (4}%}• ,, (211) " c :l%) 1.061 105 01%) 3J 00%) ( 1%) 

m (16%) 21 "' C 2U ( • J 11.07 

It is· evident. from this analysis that, for all racial gro-ups, the. numb·et 

of applicants who di<i very poor on the ExperitRental ATC test (scores below 

50) is much smaller than on the OPM test. For example·, 32 percent of the 

Black applican~s scored below SO on the OPM teSt compared to 17 percent 

on the Experimental test. This also held true for applicants who scored 

very high· on the·OPM test where 11 percent of the White group scored above 

89 in contrast to only 2 percent on the Experimental ATC test. 

Figute 9 provides a graph of the Experimental ATC test score distribution 

for selective racial groups. 

87' 

. ',I 

.. , 

·'' 

' . '' 

. i· 

( 

' 

L 

~~ 

' - I '<j 

'' ' .. ~ 

' ~;~' 

' 
I' I 

' I 

,,.. 

i~. ' •l 

. I 

l' 

'· 

,. 

!'I 

. ' 

,. ' 

" 

' 

" I 

....: " 

7 ,, 

,,, 

' ; 

' ·;,;, ' ;;- ~~ 

' . , I 1 • I · 

l'\ef.ijft"N!Al.A~z. J:F.:If :SCOZ'Z:OS 

' ' 

. ' ' ' ' 

Table 65 shows a comparison .o-: the distribution of applicants who passed 

the Experimental ATC test with those who passed the OPM test, .and the 

dis~ribution of ATC applic5nts with the CiVilian Labor Force (CLF). 

~j'--~r_.r_.!~~ 

EXPERHIENTAL ATr. TEST - OPN '!'EST ~£StiLT~ 

---·----·-------------,-.;r-~---.. -' of ATC 

CiV'ilian \ of A're OPM Test Exp. Test 

Group Labor Force Applicants Pass Group Pass Group 

Am. Ind. • 3\ 1\ 1.0\ 1.2\ 

ASian .8\ \.' '\..'\., )..3, 

~laclt 9. 7\ 24\ 8.9\ 7.6\ 

White 85.0\ 68\ 84.8\ 85.5\. 

Hisp_anic ' 4. 2\ 6\ 4.2\ 4.4\ 

100.0\ 100.0\ 100.0\ 100.0\ 

88 
In- analyzing the relationship of the Experimental ATC Te:...·t_ aild the OcCu'"" 

pationa~ KnOW"ledge Test (OKT}, the dirc·ussion and descriptive Statistics 

for the total applicant group, a:s provided in Table SS·, 5-6,_ ~d· ':>7 also apply 

here. 

HOWever-, some appli.Cants who passed t·he OP~ Test failed t.ht.! .E:Jq:>erimental 

ATC Test (and vice veJ:sa). Cons-equently·, the m~mber of applicants (and 

i_ndividU:al applicants} whu could be eligible for ext;ra credit based on 

OKT scores w.tll differ for ·the t.-wo tests. 

Table 66 provide"~ the number ahd distrib1.1tion by "poi_nt groups" of the 

total M>Plicant gr_oup, those who failed the Experimental ATC Test, and 

those who would be eligible for extra credit based on their OKT scores. 

TABLE 66 

TOTAL APPLICAXT I'OI~T GROUPS 

Total 

Extra Faile<:! Exp. Eligible for 

Points ~ ATC Test Extra Credit 

lS pts •. 259· 60 (23%) 199 (7n> 

10 pts. 251 70 (28%) 181 (72~) 

5 pts. 229 59 (22'1.) 170 (78'1.) 

3 pts. 229 ~ (37%) ~ (63\) 

96B 274 (28%) 694 (721>) 

Of the tco ... al 9.68 applicants who could be eliiible for extra 0~ credit, 

a to-tal ..:~~ 206 failed· both the OPM and the E~riritental test. A total 

of 66 failed the OPM test but passed the Experimental ATC test. Conversely, 68 passed the OPM test -but failed the Experimental ATC test .. 

Consequen~ly, the two tests affected 340 applicants or 35 percent of the 

appli~ants·who scored above 64 on the OKT. 

Figure 10 provides the· distribution of raw Experilp.ental test scores for 

the total applicant group in comparison to the OPM test score dis.tribution. 

89 

'' 
' .. 

FIGURE In 

TOTAL ATC ~PLICANT GR0UP 

(N = 6000) 

_ •. / 

/ 

~--

PA~:S ~.0~1! 

,..- dct'" "Tt!!J.T.!J, 

_.(_, 

', 

' 

... ' ... ,"" 

·v' 

:-1:f . 1$" a .3S w ...- 10 :;s eD as- SIQl Sir"to: 

""'7if.:sr ~-QEC .D~~rR.IdUT/0/'J" 

Those applicants who scored below 45 on the OPM ATC t~st tend to cluster 

in the S0-59 score range on the Experimental ATC test. Table 6 7 also 

provides the canpa.rative distribution statistics .. fo_r the "pass group~' 

for each of the tests. 

Oftl t'eat. -·- 697 -·-(2XJ 

(401) 

•-'" CW)Up T£ST scoiW ~ISTJU..&UTJON 1$- .,, . 80-64. . 

....!!.....~ ..!....-. 15--89 

143 (241) 

19S (ltrl) 

....!!.. -·- 6lS (li'U.· 471 (161) 310 (10%) 

606 (lftJ l4t cu:n u < z:tJ 

••• ..!'...--!_ ~ 

179 (61) JOJS 

6 (*) 3164 

The Experimental ATC test provides a considerably greater clifferentiation on test 

scores between the appli.cants who pass aS cermpared to the OPM test with a much 

highe.r; clustering of applicants in the 70-79 score range (67 percent comT 

pared to 48 percent) and a much smaller number of applicants wlth scores 

of 90 and al>ove (2 percent com,Pared to 16 percent). 

90 

/ 

/~ 
The Experimental A'l'C test data were also ex-a:nJ:,ned in terms of test score 

distribution and the e.ffect ·of ad9.itional credit fot" veteran-s preference 

and aviation-related experience on the score dis-t:.:ibution for those who 

pa.$sed. 

Fiqur~ 11 provides the sc~~e distribution for all ATC a?pli'Cants who 

passed- t.he Expe_r,i..1\ental ·ATC t.~c:;.t after a.dding extra credit. for veterans 

preference and OKT test scores abo·1e 64. 

FIGURE II 

EXPERIMENT~ ATC TEST 

~at 

'~ \ I LJI.S7£J""AV1'10/'J 

:;: ,... I~/ PPOZ.AN 13 

~--;;;:-; \ .:> tetO - ~- _\ - oA './ _____ ~.--f'WI7N' AIJDEIJ C;..ftJir 

1"1 v ' 1 -;.,,-m P!u -on~-

~ ...... ,, 

\ 

"' l. 

ti: '-' :;: \\ 

" \ \ 

"' ~:! \ \ 

---;;~----'- . \ '\ 

\ - \ . \ -- \ ~ 

-- --,-

70 . 7S" eo .e,... oo 9 r tat:tt_ 

&.rP< r;zrc U~T :>'G ~$--'---

With. the Experimental ATC test, a total of 485 applicants scored 90 or 

above (15 percent of those passing:) with the addition of xtra credit for 

~eterans preference and o~ scores "compared to 69 (2 percent of those 

passing) before adding extr.a credit. This is ~h contrast. to the OPM test 

where 899 (29 percent of those passing) sco::-ed 90 or above after adding 

extra credit. 

Figure 12 shews the distribudon of Experimental A~ test scores after 

addinq Vt!terans preferenc:e- and .OKT test scoJ:.e credit for men and women. 

91 
···.' 

·I 

" 

FIGURE 

.lJI~ Tt!'ldVT/0 N'. ()P 

L". 1Mt:N /NITJ/ A~~D 

~:""fi:JJ.r M:.ttt 1/E. r. 

Pl?.t!F. ~ Q~T 

" \ •\ 

.. , . 

'\ 

. '\ 

..___ 

••'4 

.... 

t I 

I 

12 

. 'lD 7$" 89 86 te.. <JS' 'flO • '!0 7'S' .. 

. Ed. AT( T~ST. SClllell' .S. 

Table 68 also sha.ors the percent.;t:ge of mer.. md women irl_ each Of foll;' scor'il: 

r4n':)'e qroups. based on ri!iw Experimental ATC 1:est scores (be-~ ore veteran: 

preference or OK'I' extra ·credit is applied) aJ)d. et.r.npares those to the ;p·( 

centage of men and women in each qroup after ap.,t:1:lying ex-tra. cr-edit. 

TAB_I:L~ 

~At. f.'te TEST ~- GJWPs 

1G-,'7<J • -... S<l-l!!!! !Q:99 ... ., ... 100+ - ;11th' - .. .,. wm; lc-• Cr.., it ....,. Crlodit ac .. .- cr.a1t c ..... , lltWh ..l!!=.Wl) l!::ml. ~ .I!:W. (11-374) t"'"JH) 

·- ,,. ... 75• .... '"' ... ... - .l!! . ..!!1 ~ ..l!1 ..l!! -ll! --.!! 

""' .... ·- ,_ .... ·- ·-

The overall'effect. of combining the Experlmental A'l'C test score with 

veter~s :_:lreferer..ce .. _ and OK'l' scores is to ino:rease the pro~ortion o~ women 

in the loWer score group (7Q-79) and decrease their representation h the 

higher score group (90+) in CCllllparlson to use of the· current OW. t.est. 

92 

'·"<· < ,_; 

' . 

......~.· .. _,,,.; · .. ·· .. 

---~ .... .. 
Table 69 compares the same information by minority groups. 

to-, .. ... WHfl 

Score Cn.<l.lt 

(!<t•:090) {M•l415) 

Wl\tte ... : "' ll:.ap.~~n1c " " ll.oo;ll, JE_ llt 

,,., '""' 

L\fiLE 69 

lap. Art teat So::ore Crou.p• 

,, .... s...-"'h tl•l071J 

,_., ~ 

--·-- •u.!\ .... $eo,re Cn•.:ht Scor• 

(li•~L8) (N•\1.:'1) {Mo.fo7) 

9)1 92% ... " " " 

_;o_ .o.: ..!!. 

"'" 100\. ""' 

..._, 

w:itFl 

cr ... ht 

{N•l6-.) 

9/ot 

" 

" 

lUU\ 

IP<>-

"'iltn 

Cred:t 

{S•IO'i) 

" 

,,.. 

addition of credit for veterans preference and 0~ sc~res has a 

relatively small impact by racial group except in the 70-79 score group 

effect is to decrease the percentage of the white group by 3 percent 

percent) and increase the precentaqe_ of the black group by 2 percent 

percent). 

analyses of the Experimental ArC test battery previously discussed were 

on equal weighting ~f each of the three tests (CSC-2'4, CS-C-157-, and 

IICAT) included in the experimental test battery. 

Hu.ltiple regression analysis usil')g 

oriteric:m for a SaJ!lple of 1827 ATC 

a correlation (R) value of • 540'7. 

weights of·: 

the ATC -laboratory problem scores as 

traineP.:-s at the FAA Academy re.sul.ted 

Converting t:he beta . ..,e.ights tO unit 

1 x esc 24 Arithmetic 

2 X CSC '157 Abstract Reasoning 

4 x Multiple Controller Aptitude 

the 

in 

resulted in a correlation value (R) o.f • 53$4. In order to assess the effect 

cf usi.nq t.he waiqh-t_ed ttest scores, the 197B ATC applicant group tes.t results were further- analyzed-. 

Table 70 provides the pass/fail results using the Weighted Experimental 

Ate· test (excluding OKT) for men and women·. With the test weighted, the percentage 

of aen who passed increa&~d from 53 percent (unweighted) to 56 percent. Women 

4eerease<i from 48 percent (unweighted) to 43 percent. 

TAB):.E 70 

PASS/FAIL, WE!Glf!ED EXPERHIENTAL ATC TEST, BY SEX 

haaVd&ht.e4 rau Wet.sl:t.-.1 

toul !!!:• ATC Te•t !!1!:· UC Tut 

S!!l!l ippllcaau ! .m !! 1ll ... 41tl ,,.. (S6J) 18U (~lot) 

- . .UU _!!! ('Jt) .!ill "7:) 

'Toul .... ,, .. "'~ , ... -

93 

/ 

"" 
~~----~------~-----~--

pr<>vid~s the pass/fail rates for se1ective racial groups with 

Experimental ArC test (also exclud.ing OKr). Again, the mean 

·score on the composite weighted test was used as the "cut" score for the 

applicant group. 

TABLE 7l 

toul P.tu Wetah!f'll h1l \;et,!\ttc! 

~ 4rpUu,nu l•p. ATC '!"nt [x~. A~: ~ut 

~ .ill ~ .ill 

""" ... , Zfl98 (66~1 1)69 o.;.:> 

I.UP-.A!c: m m (4!%) lOJ (59:) 

Jlack ll2.! ....!.!! {1'-1) ill.! (1!6':1 

foul '"' )0J2 (S::.;) 2181 (l!:.} 

The pass rate for the White group remained the same (66 percent) as compared 

to the Unweighted test. The Hispanic group dropped from 42 percent to 41 

and, the Black· group dropped from 18 percent to 14 percent. 

Table 

~ 

"? ,_ provides the wdghted test score distribution by racial groups. 

TABLE 72 

.\!E_ IC:_I~rETl ~IJ~J:._~CO_~ !:_11Y_I~A.C:I "II:_!:_R(~V_P_ 

(lf-:nu) hU VetsJned laP. . .UC T••t .. ,~ 

so ! .L!J. 

lO (•.) 

lZ ( 7l)· 

187 (lll) 

So-59 ~o-69 

! .L!J. ! J!i 

Ul ( 7Zl 10.16 U7Z) 

61 (191) 100 (JOt) 

Sl2 (!1%) soo (l6t) 

(lf•l0l2) 

· Pu.t .\o'tlshud t.;p. ATC T•n 

70-79 

! .ill 

164.S (401) 

100 (lOS) 

167 (121) 

80!-89 

! .ill 

tl7 (241) 

12 ( 9%) 

ll ( 2%) 

.. _, 

.. ! .L!J. 

.. { 2%) 

" ( 1%) 

0 ( 0~) 

... , 

"' 

, .. , 

By weighting the test, more of the White applicants shift to the 60-69 and 80-89 

aeore ranges and more of t·he Black applicants shift to. the SQ-69 score range. 

As a total group, fewer applicants score below 50 with the weighted test as compared to the Unweighted test (20 percent versus 25 percent). 

94 

I 

'

I 
Figure 1, show's the score distributicn fo.r· selected ra.cial groups based on 

the Weighted tx:perimental Tes:.. 

zo~ 

·' 

n~u.n~ 11 

P..'!.c...Sc.o-e.e 

. - . . ,..-""' BLAOC. ~OLI Pl , \ 

(ii•/""DT) . ":'*/ ,\\ 

I ; \ 

I I ,I I \\ I ,, I I 

/ I 

I . '. ~I ) I . I I / ·; ----

I /;_'--. I 

J I 

I 

\ 

\ 

\ 

\ 

\ 

\ 

\ 

r---\-1 t..h~P'IoNI(. CrtovP 

I 

\ 

I 

\ 

\ 

I 

\ 

(N: 3..39) 

--rs- -4:> SC. SS f>O 6S :'1:> 7S 80 85' 

· £ IPI:i!IMi!NrJ}L A7C ltE.sr :5C(>l!I!':S (w.Et6NreD) 

Comparing the Weighted Experimental ATC test battery to the present OPH test 

in terms of pass/fail status of the applicants shows that: (1) 2606 passed 

both tests; (2) 2431 failed both tests; (3) 510 passed the Weighted ATC test 

but failed the OPH test; and, (4) 429 passed the OPM test but failed the ATC 

Weighted Test battery. Use o,f these tvo different test affect the pass/fail 

status of 939 (16 percent} of the applicants, With the Weighted Experimental 

ATC test, 510 different applican~s (16 percent of those who passed) would be 

eligible for appointment consideration that would not be eligible based on 

the current OPM test. 

ln comparing pass/fail results of the Unweighted and Weighted Experimental 

ATC Tests, 8- percent of the total applicants were affec-ted; 21.9 failed the 

Unweighted Test but passed the Weighted Test and 267 p.assed the. Unweighte<! 

but failed the Weighted Test. 

95 

• 

. ' 
\ 

Table 7! compares the distribution of the ATC applica,nt.s and those who 

p.assed the OPM, Unweighted and Weighted ATC test with the Civilian. Labor 

Force (CLF). T,\RLE 7J 

cm!Pt\R..\"11 n: D l SH~ I JH'llOXS ------ --- --·-

' of ' of ' o.f w.~tahud· 

C1Vll1art ' of .UC ""' hop. ATC Test h'P· .&tC tau 

Wbur Fqr.,a ~~~~~ .. ~~.u. Paa.w Groue P••• cr.,~ Paaa Croup 

.l% " 1.01 1. !1 1.11 

.ax u 1.1% 1. 3% 1.01 

their. 9. n: "' 8. 9'1 ].a:; 6.U: 

8).0: .. , 4lo .Bl ·~.)1 87.11 

...!.:A ...!! ~ ..i:.ll ~ 

100.01 1001 too.o; 100.01 100.01 

Figure 14 provides the distribution of weighted test scores for the total 

applicant group in comparison to the OPM test score distribution. In both 

cases .. the test scores exclude extra credit for veterans- preference or OKI. 

~-.. 

FJ(;(;RE 14 

PASS SC.OJU!f---_.1 4oTHT~~TS 

' , 

/ 

, , ' / 

, ' -

. - 4$ .,~ so ~ 60 6,. '" ,.,.. eo es- 9o 95"+ 

.. 7EST XOIU! p,~ r IUBvTttt'N 

96 

_________ , 

~ -
/ 

Table 7~ shows the percentage o! men and- women in each of four score 

range groups ba~ed on Weighted ATC test scores befo~e applying veterans 

preference or OK."''. credit and compares these to the perc~ntage after 

apply in& extra cred.lt. Table i) ?rovides the same information by 

racial grou~. 

txi'ERlKL>qAt WllCKT.W A:'C '!'EH· SCOF.E Clb:lt'PS 

iQ-7\1 110-@9 90-100 90~1111 100. .... With ·- Wuh 

~ 

Score Cre•Ut s~o'C'e Cre41t $cure 

• •• Credit 

Wt·.to ~ 

Cndtt (N•I .. :t.) (,. .. 107tt) (!'1•1111-~) l!h/1) lN...,.O!) (tl.•l ~f.,) 

... , .. . .. m "' ... ,. ... ·-· M m m m .ill -!! ..i! 

1001 100< 100% 1001 1001 IOOZ 1001 

TA!!'!.&_J .?_ 

!XPEltMtwl'AL WtlCtrrm Af,(; T&ST SCOit£, CIO.I.iPS 

1~19 11()..89. 90-100 ,.._,., 100. 

.... Wtch ... With ._ With• '"W'i'"th 

Score Cred1.t Score Cre'd,it Scot'e cu,d1t Credit 

(N•I9t~) (N•ilB\) ~ (N•II)b) (N•10) (N•)92) (NO.llJ} 

Vbtta .. , "' ... 91% ,., "' "' 

IUapanlc " " .. " " .. " I Jack ..l! ..l! ..ll ..i! ...ll ..ll J.!. 

1001 1001 1001 1004 1001 lOCZ 100:1 

Table 7-6 provides a comparison of the total score distributicin with extra credit 

and the pass rates for the present OPM ATC test battery and the Experimental 

A!C test battery, both unweighted (UW) and weighted (W). These distributions 

reflect the composition of the OPM competitive register from which selection of 

applicants would be mad·e based on the "Earned Rati.ng·" for each of the test· 

batteries with extra credit given for veterans preferenc·e and QKT sc_ores . . (See T~bles 59, 74, and 56.) 

TABLE 76 

EXTRA CREDIT EFFECT ON PASS RATES 

,. .. , total tot~.l 

lattuy 11)..79 ...... "'"" 1 ... , ... ~ltS;••u 

.. I'Ded bt!!!J .!! .ill. ! .ill. .!! ill l! .ill. .!! ill l! 

Oftl t .. t 1076 (lSI) .... ow ., .. (211) ,. (U) )0)5 {~lll) "" .... .uc t••t {IJfl uu . (411) l1S6 (.i61) "' (121) lfl (loZ) 3164 (HZ) 5916 

.... A.TC t .. t (W} lU6 (461) U6<1· (l7t") ... (llt-) ... (.\1) lll6 (Sll) 5976 

97 

\ / 

.; . ': 
Table 7 7 provides the descrLptive- statistics for· the .~pplicant group on 

each of the three composite test batteries by men and women and racial 

qro1.!ps. 

T,\BI.l 77 

ll-~~~-t~~-I_I'_!~J..Y!:_!~Jl_-)~1-~})~~~--(o~~~~~~ !.!'__E __ ~l~E_s_·l~ _g.:\T_Ttk I ES 

OPM 

Test 

Unwei_ghted 

Exp. ATC Te~t 

~ so Mean SO 

Men (N=4191) 

Women (N=1785) 

White [N=4067) 

Hispanic (N=339) 

Black (N=1407) 

A$ian (N=57l 

Amer. Indian (N•61l 

Total Group 

"Pass. Score" 

Transmuted Score 

210.0 

199.8 

220 •. 1 

191.7 

165.0 

210.7 

204.5 

-.--

207.1 

210.0 

70.0 

46.0 

49.3 

39.8 

49.7 

45.8 

48.3 

~ 

47.2 

101.9 

98.3 

107.5 

93.9 

79.6 

103.8 

99.2 

100.5 

100.4~ 

70.0 

*Transmuted scores were derived ~s tallows: 

Unweig.h-ted Experimental A.,TC test .battery 

Rs-RsX: Ts • 10.5.6(. RsSD ) + 70 where: 

Rs • Ra~ score on ~weighted ATC te~t 

Rsx • Mean of unweighted ATC test (i00.47) 

22.6 

22.7 

19.0 

23.4 

21.0 

21.7 

~ 

22.7 

Weighted 

Ex.p. ATC Test 

Mea.."l sn 

227.3 

210.7 

241.5 

206.5 

170.5 

235.8 

~ 

222.3 

222,2~ 

70.0 

55.5 

56.1 

46.7 

56.1 

47.2 

50.8 

~ 

56.2 

RsSD • Standard deviation of unwoi&hted test battery (22.67) 

Weighted Experimental ATC test battery 

Rs-Rsx 

Ts • 10.68( RsSD ) + 70 where: 

Rsx • 222.21 

RsSD • 56.22-

98 

---
table 78 provides the intercortelatio_ns fot' the three test batte·ries .. 

f',\BLE 7~ 

UIT£RCORRELAT!()~S-6)~fi~S-t"iE TEST BATTERIES --------·-p;;se·nt_< _____ E;cp-:-- ---- -··--· Exp. 

OPM 

ATC (Unweighted) 

ATC (Weighted). 

OPM ATC Test ATC Test 

Test 

-

1.00 

Unwe·ighted 

.92 

1.00 

Weighted 

• 88 

• 97 

1.00 

Conqlusions. Given the high attrition rate of ATC trainees and the objective 

of developinq improved metPods of seleet~nq applicants for the ATC occupation, 

attention is focused on the selection tests used in the hiring process. The 

results of using the Experimenta-l ATC test battery which was derived from the 

research studies previously discussed are not unexpected. 

The number of applicants who sCore high .Jn the Experimental ATC test (85 

and above} is much smaller than on the current OPM test. Based ori the· 

valid~ty studies which have been conducted, ~ose applicants who score high 

on the Experim-ental ATC test can be expected to haVe the highest probability 

of auccess in the developmentai training for the ATC occupation. 

Granting extra credit for military service is required· by. law. This does 

impact on the competitive appoiritment conside·ration for women but does not 

appear to differentially affect indivi,du,a.l mino-rity groups·. Aviation-related 

expe~ience and knowledge which is measured by the Occupational KnoWledge teSt 

has been shown to be- a positive predictor of success i.n ATC training in the 

research stud.i.~s pr~viously.- discussed. Granting extra credit for ·the demo·n..._ 

stration of this knowledge improves the· competitive. position of selected applicants but this is consistent with the objective of developinq .. .improved selection 

pr~edU¥es in order to hire those applicants who demonstrate aptitude for air 

traffic cont_tol work and whose potential for success is greatest. 

99 

,j 1 
STUD)" Ot m;\1 APPOINTEES TO THE ATC OCCL1'AT!D); ~ 1978 

Objective. Re-search su•dies discussed up to this ·p.oint Yet:e directed to: 

1) Identifying an improved AIC test battery; 

2) Development of parallel. test fonns for those tests which 

consistently prod,uced s-ignificant correlations with the 

various criterion measures of ATC success; and 

3) Evaluation of experim.encal and OPM tests in relation to 

applicants for ATC work. 

Re-sults of these studies supported the conclusion that -1 new test battery 

comprised of the Multfplex Controller Aptitude Test (MCAT), Abs.tract 

Reasoning (CSC~157), and Arithmetic Reasoning (CSC~24) provided. consi.cently 

high correlat;i.on with criteri.on measures. It was also shown chat the AIC 

OCcupational Knowledge Test prC'vided a better m_easure of aviation-rel~ted 

knowledge than the present Racing Guide used to evaluace experience. The 

Occupational Knowledge Test (OKT) also provides significant correlations 

with success in the ATC occupation which added to the validity obtained from 

the test· battery alone. 

With the developaH!nt of parallel forms of MCAT and OKI, the next step .:as to 

valida-te the. proposed test batterY with a new group of ATC trainees a·t tending 

the FAA Initial Qualifica.tion AIC Terminal and En route training program. Since 

this study has not b.een published elswehere, this section will discu.ss the complete ~ethodology and findings. 

Sample Des.cription. The sample of tiainees used for this valida~ion analysis 

conSis-ted of 953;- uew trainees ·attending the FAA Academy during th~ period 

Jun·e 1978 through December 1978. As in previous studies, tests were administe-red on a voluntary Msis on the first day of attendance at the Academy. 

the composition of the sample with respect to sex and ra:cial groups is provided 

in Table 79. 

TABLE 79 --~··-·--

SEX A.~U RACE ~ 1978 S!~:PL_L 

Men 800 (85.07.) Orient ell 10 (1.11:) 

Womt~n 141 (15.0%) Hispanic 23 (2.4;:) 

Uni,Ientif ied _11. Black 81 (S.SZ) 

Other 839 (88.07,) 

Total 953 Total 953 

100 

- ' . 

/ 
Table Sl) c001pares the composition oi this sample with the l97a ATC appli~ 

cant group who passed the OPM t-est and those whq passed the OP}t test wit.h 

total scores (earned ratings-) above 89. 

The representa-tioa of miri.orities in the ATC trainee sample is somewhat 

greater that\ would be expected if all trai,nees had been sel~cted through 

competitiVe appointment _processes from the OPM re>~istC'r. Becaus-e of the 

large nu:nber of applicantS in relation to the' rela:tive small number of 

vacancies, comp·etitive se.!ections of ATC' trainees are normally made in 

score ranges ·abdve 90. As shown in the analysis of the OP~ test scores 

for the 1978 ATC applicants, the White group comprised about 91 percent of 

all apflicants who scored 90 and above on the OPM test battery after inclusion of credit for Veteians Preference and aviation--related knowledge/ 

experience. However, in this group of over 900 ATC trainees, the White 

group is 88 percent of the total. 

Because of the difficulties in hiring women and minorities through competitive appointment proceSses from OPM registers, FAA established an 

alternate recruitment program in 1968 which provides fo-r non-competitive 

appointment. of individuals who are already fede-ral employees with status 

in the career Civil Service system.. These individuals are required to 

pass the same A:!C test battecy with a score of 70 or more as the applicants appointed from the oPM register. However, having passed the test\ 

they can be hired at the GS-5 level non-competitively. This alternative 

recruitment program is identified as the ATC Predevelopmental Program. 

101 

·, 

' 
\ 

Approxima.tely 200 Arc trainees are hired each year through this alternate 

recruitment pro~ram. this re.prese.nts about 10 percent of annua-l new ATC 

hires,. These employees rel;eive a year of predevelopmental training in 

air traffic control related subjects including 17 weeks of formal classroom instrtJ,ction at the Predevelopmental Traini.og Center at the University 

of Oklahoma followed by training assignments at FAA terminal, en route 

centers. and flight service station. ffeld facilities. Upon completion. of 

predevelopmental training. these eQployees are promcted to GS-7. (the 

normal entry grade for AIC trainees) and assigned to the FAA A~ademy for 

Initial ATC Qualification Training. 

In order to further deC'e-rmine the composition of the 1978 ATC trainee gr-oup, 

Table 81 identifies the number of predevelopmeotal emplcyees included in 

the sample by sex and racial group. 

TABLE lll 

PREDEVELOP}!E~TAL EHPJ,_~E£~1!.1- S~':_A:~D R_,\_C£ 

Non-Predevelopmental Predevelopmenta1 

(Colr.petitive Appt.)u (Non-Comp_edtive Appt.) Total -·--. 

,-of % of ~ of % of 

! Totc.l Croup .!!. Total Group N ! 

739 (92.4%) ( 88.9%) 61 ( 7. 6%) ( 55.5%) 800 (100%) 

92 ~65. 2%~ ~ 11.1%~ 49 ~34.8%) ~ 44.5%) 141 

m (88. 3%) (100.0%) Tili (11. 7%) (100.0%)- m 

9 (90.0%) ( 1.1%} 1 c1o.o:o ( 1.0%·) 10 (100%) 

11 (47.1l%) ( 1.3X) 12 (52. 2%) ( 10. 9%) 23 (100%) 

26 (32.1%) ( 3.1%) ss· (67. 9%) ( 50.0%) 81 (100%) 

797 ~95.0%) ~ 94.5%) 42 { 5.0%) p8.1%) 839 (100%) 

m (88.5%) (100.0%) llO (11.5%) (100:0%) 953 (100%) 

It is evident that the Predevelopmental recruitment program provides a 

major avenue for entry of women and minorities into the GS-7 Initial 

Qualification ArC training program. About 12 percent of this 1978 ATC 

trainee sample enter~d the A~ occupation through the Predevelopmental 

Progra:. Arproximately 35 percent of the women, 52 percent of the 

Hispanics, and 68 percent of the Blacks in this group of over 900 ATC 

trainees :me into the occupation tltrough Predevelopmental recJ:uitment 

efforts. 

102 

./ ~--: -"~- I 
.. Table ~--' c.ompare:s the composition of this group of AIC tr.dnees -with 

the Civilian Labor Force (CLF) in relation to both the ATC trainees 

hired· through competitive and non-competitive OPM procedures. 

T,u:u- K~ 

Cl\'11..!.\:\ L\:'.11R __ F~l_!u.:·t \'-~. -Al_t:_ 111.!\l'_r~~.f'·Y__C:}._r\S_~; 

CompeLitive 1978 Total (Comp. & Non-Camp.) 

CLF ATC Traintes 1978 ATC Trainees 

Men 

Women 

Amer. Indian 

Asian 

Hispanic 

Black 

Other 

62% 

38% 

.3% 

.8% 

4. 2% 

!'.7% 

85.0% 

loo:oi 

89% 85'! 

11% 15% 

1.1% 1.1"% 

1.3% 2.4% 

3.1% 8.5t 

~ 88.0%.· 

100.0% 100.0% 

Predictors .. The predictors used in t_his· analysis were: 

• Multiplex Controller Aptitude Test (4o6e, 4e6o, 6e7o, 

6o7e; 734o, and 7o4e) 

• CSC Test 24 - Arithm~tic Reasoning 

• CSC Test 15.7 - Abstract Reasoning 

• ATt Occuparional Knowledge Test 

Otcr 101!1, 100" items; Otcr 101C, 60 it=s, Otcr 102 

(Fo_r.ns A, B, C, D, E, F, G, -H), 80 items keyed 

In this s·tudy, tvo of the parallel ioms of MCAT were administered to each 

student. MCAT 1 was t\ie first form administered; MCAT 2 was the second. 

Table RJ provides the intercor:relatiQn. of raw s ... ores for each half or part 

test on each cf two parallel forms of.MCAT a~-nistered to 617 ATC students 

at the FAA Academy. TABLE 83 

tNTERCORRELATIOXS~~IC,\T PARALLEL FO_fi~IS 

First MeAl" Administered (MCAT I) 

1. First Half, aights 

2. Second Half, Rights 

3. Total Rights 

Second MCAT Admin1atered(MCAT 2) 

4·. Firil.t Half. Righ~s 

5. Seco11d Ha·lf, Rights 

6. Total Rights 

103 

1 

.58 

3 

.89 

.89 

4 

.48 

.48 

.89 

5 

.42 

.49 

.89 

•. 52 

I I 

6 

.51 

.56 

.60 

.85 

.89 

. ' 
A test-retest =orrelatiou of .60 bLtween comparable two-part 35 minute 

forr.s of ~r.AT was obtained. Test reliability was . 75 for this res.'ri~ted 

g-:ou?. Arl;ninistering tvo forms of XCAT would p·rovide an estimated reli-

_flbility of atout .88 ~Jith an unr~stricted _;>opu!.at.ion. Some 265 trainees 

in the t~tal sample (9~3} took forms IO:IB or lOIC of the 0!<!. For purpos_es of this analysis. the:s;e t~st scores were converted co a scale common 

to forms !OZ of the OK!. 

The descriptiVe statistics on test ~cores for this sample of ATC traineeS 

is provided in Table X~. 

_!/~B}.£ 8_4_ 

-~~~q. _U~II __ Y.L_ST,~~!: ..;.}_~ c;s-!-!~A~:__l_ ~-~-~ SA.'-1PLE ·--'-----

!!E! """"' IPT£ 

Q!) !!!!5 !!!. Q!) !!!!5 .!! '!) !!!!5 !!!. 

NCttl (lOCI) )1.4 ••• (l4l) .... ••• ,,,, l6.t ••• IICI%2 (1001 u.t ••• (li•U 40.7 '·' (95)) U.6 ••• cs: 24 C>l>> .... ••• ( 67) 47.1 ••• (StU 46.6 ••• esc U7 C>1Sl ]l.t 6.2 ( .,.., lt.a ••• (S9l) .... ••• ocr (1001 .... u.s- (141) 17.7 U.l (fl)) 6J.9 lJ.I 

... ,., ... EIU"!!C """' !!!!I! 

l!!!. Q!) !!!!5 !! Q!) !l!5 !!!. Q!) !!!!5 .!!! (!I 

-!!!. 

-· ' tO) )6'.2 6.7 ( 2)} J1.1 ••• c 11) 29.1 ••• (J)t) 31.6 '·' """ 2 . ( 10) .... •• 7 ( 2l) •2.7 ••• ( II) "·' ••• (1)9) u.: esc. 2• ( n .... ( 7} ... 7 '·' ••• ( ,, .... '·' UU) .... , ••• esc U7 ( 1) u.o ( 7} .... ••• ( ll) ]).) • •• (S4)) )9.1 ••• 

ocr ( 10) 61.) U.J ( ll) Jt.O "~' ( 11) .... U.l (I)') .... u-.1 

About 12 percent of the 953 ATC· rraine~s cnte~ed the occupation t~rough the 

Predev.eloptttenta: Program. It was ~f inte-r.est to establish the e.xtent to 

which mean sco~~s and standard deviations o'1 each of the tests differed 

between the Pt'edevelopmen·tal (non-compt!ti::ive ·appo-intments) and non-predevelopmentt.l (ccmpetitive appointments) groups. Table 85 pro•!i.des the 

descriptive ·statistics fo1: each of t~tese grou?S by s.t:...c. It. is ~.mportartt. 

to recognize that the scores for the Fredevelo~ental traine~d on MCAT 

aad OK! were obtained ..!£m. they had completed a year of training. Scores 

.,.. the CSC-24 and csc-157 tests were obtained for both. groups based "" th;. 

current ~C test battery administer~d prior to employment in the AIC 

occupation. 

~E 85 

DESCRIPTIVE STATISTICS J:Y SEX - 1978 SAMPLE 

k.-h' .... iopMatal h.S..el.,...C•l 

.a%C 1r. ...... AtC fnb.MS· 

! !!!!5 - illi ! 

·-

!!!!5 illi ! !!!!5 

-

.lW. ! ·-· ~ .lW. 

..... (1l9) )?.1 '( 6.6) (92) 35.2 ( 7.4) (11) )2.; ( S.l) (49) U.4 (7.9) 

IIW2 (13t) 42.] ( ),4) (tZ) 4l .. J ( 6.1) (61) ,. ( l.l) , .. , 39.~ , ... , 

£K..14 (4Hl ..... , ( 6.1) (61) .... ( 7.1) (ZO) ... , < '·'I (lt) 4S.O (6.7) 

csc-J.>l (4tS) .... ( 6.2) (&I) 40.1 ( s. 7) (ZOI :.s •• ( s. 7) (It) ,,, (l.l) 

... (Jlf) .... US.J) (tl) u.o (1S.6~ (6U 72.5 (lO.I) (41) .... (9.4) 

104 

,. 

. ' 

. ; 

.• 

'' 

., 
On all tests except OK:f. the me.an score of the Pre.deve1apcental group is 

lower ehan the non-yredeve1opm.ental group. On OKT it is evident that the 

year ef predevelopmental training for empl.oyees who are hir~d non-competitively 

is quite successful in providing knowledge on air traffi::. rule-s, regulat:ions~ 

and procedures vhich the OKT was desigped to .meas\lre. BOth :nen an.d wumen 

Predevelopmentals score signif.icantly higher than their non-predevelopc~nt;al 

counterparts as t"eflected by the mean scores on OK! for the tl."O gr-oups. In 

addition, the,standard deviations on OKT for the Predevelop:1~ntal group wa.s 

considerably lqwe·t; indicating a much more restricted range oi test sccres. 

On the two present esc tests (157- a·nd 24). women in both groups $COre SO:lewhat 

higher than men. This is consistent vith the pattern deri"\·ed from the 1976-

1977 AIC applicant &roup who passed the OPll. A!C test and the 1978 ATC appli~ 

cant group. 

On the MCAI. men and women in bo~h groups increased :heir :ean test scores 

en the second ad~inistration of this test. However. the· difference in mean 

acores beeveen the predevelopmental and the nori•predevelopmental groups reaained fairly constant for each of the tests. Men in the pred.evelopc.ental 

prosram scored between 4 and 5 points lower and predeYelopoental women between 

2 and. 3. ,oints lover on each of the tests as cowpared t;o non-predevelopmen.tal 

trainees. 

Table.S6 provides the ci.esc.riptive statist.ic.s for the predevelopmental an~ 

non-prede.vel.opmental gi'oups by race. 

TABLE S6 ----- -·. --

N,_tr.d-•lf~Mal Att Trdg••! 

Orlaa(al Hhpan1t l!ack Vhtt• :"ou.l 

.l!!l. !!!.!! ~ .l!!l. .'!!!.!1 .Jill .l!!l. iiUn .Jill .l!!l. !!!.!!!. .lEl .l!!l. ··~ .lEl 

11W I "' .... ( J.l) (11) U.l ( .. ., (24) lO.t { 6.4) {176) 37.' ( 6.1) (Ill) ll.S ( 6.1) 

,.,.. ' (I) 4),4 ( S.1) (11) 4l.:l ( S.l) (.14) .... ( 1.6) cnaj U.l < s.~> (.8::1} 4).1 ( S.S} CS04 (II .... ( ,, ''·' ( 6. Z) CIS) 42.4 ( 1.1) (512) 46.9 ( '· l) (Sl2) 46.9 ( 6. Sl 

CIC lSf (1) u.o ( 4) 42.0 ( ).6) (U) )).4 ( 6.)) ()ll) )t.t ( 6.2) Ul2) ]9.0 ( 6.2) 

-... .... (1!.4) (ll) 41.1 (20.5) (14) u.s us.n (Hi-) 6l.S (16.0) (121} U.l (lft.l) 

......... IOI!!Ifll·~ .UC Trai-a 

ctt-t .. tal ~ ·at-.c:k ""'" !£ill .l!l. ~ .l!l. .!!!5 .121. .l!!l. ..... .1!!!1. .l!!l. !!!!2 ~ .l!!l. ~ ~ 

lOCI% I (I) .... n.n n.t ( 5.2) (56) lt.S \ l.l) (4)) .... ( 6.9) (Jil) u .• ( 1-11 

""" ' <II n.o (U) u.s· ( 9.1) , .. , U.9 ( '.l. (43) 41.1 ( 6.0) (112) )9.0 ( 1 • .5) 

CSCK ( " "·' ( 4,$) (24) 45.4 ( 7.0) (I)) 4).9 ( ... ) ( 40) .... ( 6.6) esc,., . ( " .... ( 4.5) (14) lS.l ( 6.1) (ll) ll.l ( 6.5) ( 40) ll.l ( 6.U ... U) .... UZ) lt.l us.•l ($6) "·' (,lQ.4) (4)) 10.1 (10.3) (112) U.7 (lQ. 7) 

As a group, the Predevelopcental trainees scored lower on all tests as 

,,.,.pared to the nen-pre<ievelopcental trainees except on OK!. The greatest 

differ.ence in ••an scores were associated with the MCA! test which also had 

areatar score ranaes for the Predevelopcental group. These differences 

vera generally evident withill each .of ihe racial groups as vell. 

105 

I 

'·' 

; . 
On the OKT, each racial groupo~ Predevelopmentals gener-.lly scored signi• 

ficantly higher with <:!.more restricted range of scores .than their non--predevelopmental cour.t.erparts HoweVer, it does not appear that the academic 

ATC knowledge acquired during ;:l,.e one year· of training carrys ove·r, in 

applica~ion, to the MCAT for the predevelop~ental group. 

On the HCAT. thet;'e is a consistent pattern of "learning" be._tween the first 

and serond administration of the test for all trainees although the nonpredev~lopmental tra-inees genet'ally maintain a constant difference in mean 

scores on both tests as compared to the predevelopmental trai-nees. This 

pattern s~pports the use of a double length MCAT as a m,eans of measuring 

this "lear:'ling ability" on the part of applicants. 

Cri-teri_on. In this analysis, whether or not- the student passed the AIC 

Academy training course or failed/withdrew was used as the criterion 

measure against which the tests were validated.. Of the students who withdrew~ those who did so. for medical reasons or because of personal or family 

emergenc~es, were excluded from the sample since their training was interrupted through no cause of the student. In these cases, the student is 

allowed to reenter training after the medical situation or emergency is· 

resolved. 

Essentially all o- the students wQo fail or withdraw do so in the t.aboratory 

:-hase of training which requires the trainees to demonstrate and apply their 

kr.owledge and skill to air traffic control problems conducted in a laboratory 

enviro~ent .. The structure of the grading of the laborato·. phase of training 

is prov~d:d ln Figure 1 ), Estimated reliab.ility for the Six en route control 

problems ~s . 79 and .. 75 for th7 teraical control problems for students attending 

the FAA Academy dur~ng the p~~lod· June-December 1978. · 

The ATC laboratory training phase consists of six operational ATC problems 

designed· to replicate the non-radar air traff,ic contro-l environment. Trainees 

conduct identical control problems within a common air traffic control sectcr 

using identical equipment and evaluation criteria for each ATC option. These 

operational problems provide stimulus and decision-making conditions representative of the job environment and requi.re responses from the trainees in 

terms of actions that are both possible and required to control air traffic 

on-the-job with a minimum of restrictions. Students are given a series of 

practice problems before each of the six graded problems as part of the 

tra:!.ning and leaJCQing process. Each graded operational problem takes about 

~ne hour and is scored by an instructor using established grading standards 

for defined types of stud.ent errors or deficiencies. Additionally, each 

student's performance is evaluated by the· instructor on a rating scale of 

40 to 100. Each problem is scored and evaluated by a different instructor. 

After the operational problems, a Controller Skills Test (CST) is administered. 

This test was designed to measure the application of knowledge and skills 

taught durin& the first months of training. Three basic elements for evaluation are distributed within the tes.t: (1) Application of aircraft separation 

standards-students must, respond to control situations presented by flight 

strips and. cha.ru.; (2) responding to or forwarding information received. 

106 
which pettains to coordination of aircraft movement or info~ation with other. 

Controller-s; and {3) other ATC cOntrol it~s suc;h as board management, timeliness of actions and ·phra_seo.l_ogy. The CS-T is 4 50-i terr. ·rml tip lt.• choice test 

with an ado.inistration time of on·e bout. The correlation betwee.n, the average 

score on the six operat·ional prOblems and the CST for -this grOup of tra1Qees 

was .50 for those in the terminal option (N•454) and a correlation of .54 for 

enroute students (N•473). 

During the aqtdemic training of- the ATC Laboratory Phase-, a bloc·· test is 

administered for each suhject. A comprehensive knowledge test '~ also administered to measure the degree :o which students have learned the acadecic 

portions of ATC subjects taught during laboratory training. This comprehensive 

phase test is adm+nistered prior to giving any operati9nal laboratory control 

probl~s. Figure 9 provides a breakdown of the ATC laboratory training and 

the ba$1$ upon which a student's total score for this phase of training is 

derived. Students must complete this phase of training ·with a total score of 

· 70 or more to pass the FAA Academy and continue on-the-job training ar 

assigned AXC facilities. 

Analytical Methodology. The first step in the analysis was the calculation of 

descriptive sta,tis-tiJ;s on the ATC train-ee- sample using those students who had 

test and criterion data. It should be noted that the s~ple size was affected 

by the lack of complete information on all variables. particularly on esc tests 

24 and- 15 7 where sCores- could net be retrieved from FAA or OPM records. 

Multiple regressiOn analysis vas carried out using the exp~rimerit.al test- battery 

with and w:i_thout OK! to determine the multi_ple correlation of the test battery 

with "pass" or ''failn status of the traine.es. Separate regression analyses 

were conducted for the total gro-up and for subgroups; men, women and r~cial 

group·s· where subsam.p.le si:.es wa-rranted,. 

The selection utility of the test battery was examined to assess the impact 

on training costs; tmpact on the s~ple of AtC· trainees in terms of those who 

would (or would not) have been hired if the test battery had been used for 

se-lection decisions; and, an analysis of the pass/fail status of those who 

"ould have been either eligible or ineligible for hiring based on the test. 

107 

... 4. -..,. 

I 

i /I 
Lav Average 

*6 5.0% 

HGL'REI5 

CmtPOXENT$ r\~ll \..'EIGHTS t:SED 1 !\ CO~lPPT I ~~C TllE 

FAA ACADE~fY AI C L\EURATLH\Y TR.:\I:\ 1 :\G Tlll:\L SCORE 

I **Extra Cr<•dit Sixth I Extra Credit 

n.om: l'r<•b 1 <'m I Instru(·tor .\sscssr.::cnt 

I 

Pr0blern Errors 

\ I **Instrul'tor F lfth I .Extra Credit I Assessment Prubler: lnst rue tor .~\ssessment 

i 19.50% Problem Errors 

I Fourth Extra l.redit 

Probll~m Instructor Assessment 

Problem ErrOrs 

Third E.xtra Crcdi_t 

Problem Errors Problem ins.truct0r Assessment 

32.50% Problem Errors 

Second J Extra Cree! it 

Pr0blcm Instructor Assessment 

ProblPm Errors 

First Extr.c, Cn·dit 

Problem lnstruc-to·r Assessment 

Problem Errors 

Controller Skills Test 25.00% 

Com~rehensive Phase Test 8.00% 

Block Average 2.00% 

2.60 

3.90 

6.50 

~.60 

3.90 

6.50 

% 

% 

% 

2.60 % 

% 

% 

3.90 

6.50 

2.f0 % 

3.90 ., 

" • 6.50 '• 

I. 30 

I. 95 

3.25 

1. 30 

1. 95 

).25 

% 

% 

z 

% 

% 

% 

*The lab average constitutE:>R OS% of the total score for thi-s phase of 

ATC training. 

**On each lab problem the instructol gives a performance rating for that 

problem that is averaged with the student's prorylP.m_performance. 

~ir.ce the ratinp iFI 1'\f't ~tllowerl tn h~ he1ow 40. esseritially the student 

is given- a certain amoun-t of extra credi.t in the cornputation of the 

problem-averageo Each of the t:ix problems is gradea by a 

different. instructo-r. 

108 

'. 

\ 
/ 

lesults. The follo·.11n.g discusses the analysis and findings in te-rms of 

the AIC sample used. the d~~criptive statistics; multiple regiession, and 

utility of the tests. 

ATC Saeple Used. As pointed out previvtsly, the samp~e of 953 ATC train'ees 

was· affected ~Y' the lack of complete infon.:ation on all variab-les. As a 

consequence, the total sample was not availabl_e for some of the analyses, 

particularly the ~ultipl~ regression. Table S7 provides a distribution of 

the total ATC 'traint!:e sample (953) showing the number o.f tr2'.inees who had 

data available fo·r each of the variables. 

s .. 

.... 

Vartablu 

{

Heo 

""""'" ' total 

£

Black 

Hisputc 

Aat·an 

Other 

Total 

TABLE 87 

AVAILABLE DATA- 1978- ATC TR..Alr\EE S.\NPLE --------·----------·.- -- -----~--__:,._ ____ -------

Status As 

Predev. 

NonPredev. 

788 

133 

m 

80 

23 

9 

821 

m 

ltCAT 1 

HC.\7 2 

.2E: 

800 

~~! ~· 

81' 

23 

10 

839 

m 

esc 24 

c.sc lS 1 

SIS 

61 

ill 

39 

7 

1 

lli 

S92 

Pus/Fail 

Ct'1t.ar1o() 

790 

137 

m 

19 

23 

9 

821 

'ill 

,Availability of test scores on CSC 24 and 157 had the most impact on the 

AIC sample which could be "sed in s<lllle of the analyses. Table SH prov·ides 

a comparison of the means and standard dev..iations for. the grou.p of trainees 

with CSC 24 and 157 scores, the group withOut, and the sco·res on those: tests 

common to both groups. 

In addition to common test data, OPM Transmuted scores and Earned Ratings 

were available for many of these AIC trainees. The Transmuted OPM scor~ 

is the •core the trainee made on the present OPM test bt.ttery before additional 

credit for veteran•s· preference or aviation-related experience is applied~ 

Tb~ Earned Rating is the total OPM score including extra credit for veteran's 

preference and aviat.ion-relaced experience. Table :8B provides the means. and 

standard deviations for these data. The data for the pass/fail criterion is· 

also provided. These data were examined· for statistically significant 

differences. 

/ 

,. 

• 0 

' . . !' 

0 

.· 
TAllLF. SB 

DESCRIPTI\~ STATISTICS - CSC TEST SCORES 

Toul .uc s-.u .xc s-.11 "'· t••t t.t. IUect 

UC S•ple Wttll esc Z4/U7 w1t~t esc .:t./U7 ~ !.!2 

c-t .. t. '" aQd D•u '" '" .l!!.l. ~ .!.! .l!!.l. ~ !!! .l!!.l. ~.!! !! 

"""" ' (9Sl) )6.11 ( 7.11} (S9l) )7.07 c •• 94) ()61} lt.S6 ( '.40) •• l-07 .... 07 

"""' ' (911) 4l. 5:9 ( s. 96} (592) 4).01 ( ).11) (:..1) 41.90 { 6.0$)- .. 2.10 ••• 19 

or:r (tSl} 6l.92 (15.64) {592) u.u (U.U) (l6l) U.04 (1). )I) t • •1.15 .... u 

Of'M Scor• (l') (704} 17.51 ( ,,.,} (591) 1?.21 ( 1' .U) {lll) lt-.Oi ( 7,lf)U t "!' •).10 .... u ant (U) (6)1} ... 9) ( .5.66) (HO) 95.07 ( 5.4)) {Ill) 94.12 ( 6.61) ' . .... ... ·" •-•lr.u Gl (939} ·" ( .41) {US) ... ( .U) ()54) ... ( .41) • • ~ .61 " •• 04 

- '• ( .o-u 

At-test-was performed to examine the significance of differences on the tests 

and OPH s.cores. A z-test was used to measure the difference between the proportions for _pas:s/fa_il. The differen(;es are not significant operationally. 

A number of analyses were based on sex and race in addition to te£t .scores. 

Consequent~y. test scores on MCAT 1, MCAT 2 and OKT were examined for statistieally significant differenc·es by race and se:x. between the ATC t.raLnees "With 

CSC 2-4 an~ 15.7 .scores and_. thos~ with-out scotes on the latter two teSts. 

Table 89 provides the results. The uHispanics" and 11Asianu groups- were not 

used. because of the small sample numbers. 

As. sho"rn in Table 89, group differences were not significant for the "Womenu 

or "Black" groups. Statistically significant differenceS were fo:.nld fer 

the "Mer( and "Oth-er 11 groups where the sample sizes were large. Ho_wever, 

the estimated effect size ·in Table 89 are all below the Cohen's criterion (31) 

for small. 

110 

.,/ . 

~-I. . ~-· 

-- -·-----~---·-·---~- 1 

·--· 

'. 

,. 

. ~ . 

' ' 

. . 

(;· 

' 

' . 
\ 

TABLE 89 

HCAT A.~ALYSES BY SEX AND RACE 

TOTAL AtC 

s.wu 

HeAT l (800) )7.4 ( 6. 9) 

HeAr l (800J 4l.9 { 5,1) 

~t TOT (8(1')) 10.) 01.25) 

Qll' (800) "·' (15.5) 

·-··------... ------------- MCA.T 1 (141) )4.l { 7.8) 

IICA.T 2 (1-41) 40.7 ( 6.4-) 

• p(~.OS) 

"p_(i.OU 

NCAr TO"'' (141} 74.1 (ll.l) 

Oil' (141) 57.7 (15.1) 

---···-----------·---

K(:A.t 1 { 11) 29.1 ( 7.9) 

IIC.\T 2 ( 81) 16.6 ( 1..8) 

MeAT TOT ( II) 66.4 (I). 21 

orr ;: au 66.2 cu.l) --------------·-

1\CAf 1 { 23) l7.l ( 6,6} 

HCA.t 2 (. U) 42.7 ( 7,4) 

tc:.\1 TOT ( 2)) 79.1 (U.l) 

~:=----~·~·~'~'-''~·~·-~~·-·-··~' 

IICAT """'' 1 2 

IK:Al-

... ( 10) l6.l 

( 10) U.l 

( 10) 7t., 

( 10) 61.] 

(139) l7.6 

(119) U.2 

(llll} 10.7 

(1)9) 61.9 

( 4. 7) 

( S-1) 

( 9.S) 

(ll_.7) 

( 6.1) 

( ).5) 

(10.1) 

(15.1) 

ATC SNG'U 

VIttt esc 24/UJ 

m ~ !.!! 

(SU) l7.6 

(5Hi) 4),) 

(liS) 80.9 

($15) 6-41.1 

( 6.6} 

( '.1) 

(10. 9) 

Cl5~11 

-~--------

( 67) )').) ( 8.0) 

( 61) 40.1 { 6.6) 

( 61) u.o {1]. )) 

!~!~-~~--~~:~ 

(-39) 29.1 ( 1.7) 

{ 39) 36.3 ( 1.5) 

( :J9) 66.0 (l3.9) 

~~-<~::: 

(. 1} 33.4 ( 6.1) 

( 1) 40.1 ( 5.1) 

( 1) 1).5 ( 9.S} 

~~~ 

l) )\.0 ( - ) 

( l) 42.0 ( - ) 

( l) 13.0 ( - ) 

( .:ell__:"::.:·c:.• -· ( ~..l 

(54)) 11.1 ( 6.6)· 

(5o4.5) u., ( 5.4) 

(.545) !1.2 (10.5) 

(54.5) 63.1 (15.1) 

ATC SAKPU 

wtnot.'T esc 2'1U' 

m ~ ~ 

015) lJ.O ( 7,))., 

(28Sl 42.2 C 6.0}** 

(285: 7!11.2 (1L8>• 

(285) , •. s (lt.,8) 

-----------· ------ ( 14} )1,,8 ( 1.1} 

( 14) 1,0.6 ( 6.]) 

( 14) 15.4 (1).0) 

i2~~-.1!.:._• -~~:.~! 

< u> a.a < e.n { 42) )6.9 ( 6.1) 

( 42) 60.7 (12.-lli) 

!..:!~--~:!-'-' !:~! ( 16) 18.8 ( 6.1) 

( 16J 4].8 ( 8.0) 

( lf~) 82.6 (U.S) 

!~1:.·:.:•::.· _:<.:;"::·.:'' 

( 9) )6.1 ( 4.6) 

( 9) 4}.) ( 6.0) 

( 9) 10.1 ( 9.8) 

!.-!?-..!!:!-!~:.~~ 

(294) 

(294) 

(294) 

(l94) 

)1.4 

41.5 

19 •• 

6.5.4 

( 6.9) 

( ) .. ,.. (ll.l)" 

{U.S) 

SIC:. Tl.Sl 

IU:Si.'U$i 

t .. l.ll 

t • 6.11 

t • 2.46 

( - -1.85 

t • - .81 

t • .08 

t.- .21 

t .. - • )8 

t .. - .0) 

t • - • 2:6 ' t • - .01 

t • - .lio 

t • l.M 

t .. 6,25 

t .. 2.11 

'. -1.18 

u'r. tmcr 

!!,;! 

• - .09 

C.• .I~ 

d •. 15 

••• Ill 

d ... 19 

d .. • 02 

4 •. 11 

d ... 17 

d • • I 

d - .09 

II • . 0~ 

II • .09 

• •• 04 

4- .111 

4 •. 12 

.... 15 

The question as to whe_ther the two groups can be considered equiv~lent cannot 

be· answ-ered defin{te.ly. However, the analysis supports the assumption that 

th~y are, ccnsidering: 

No group differ-ences were found on f.our. of the 

measures (Table 8e). 

N~ group differences. were found for the "Women" or "Black" 

3roups. 

Where group differences were found, the esttmated effect sizes 

were small. 

Where group differences were statistical~y significant, they 

were in a positive direction for MCAt, but a negative direction 

for OKt. They do not consistently favor one group or the 

othe·r. 

Table 90 shows· the distribution of the totdl 953 ATC sample in compariscn 

to the. ATC trainees (592) who had scores on all test variables by Pre:developr&tttntal and non-.predevelopuKt:ntal groups. 

' 

111 ., 

\\ \ \ \ '· I - . I I \ 
I_A.B L.E _'t_\! 

DISTRIBU'r IONS- LNITIAL AND REDUCE!l SA!·tPLE~ 

Toul ATC $-21• ]tlo<lev•l~ntd No .. -ru·cl•v•laf:!•!'U •l 

t•uul .. ,h..u4 lnitld ... ~ ... llliU•l Ltclu:..d 

s-el• s-et• S.aplc. s-ol• S•pl• s .. pl• 

~ ! ! ! ! ! ! ! ! ! ! ! ! 

·- ""' ( I).OZ) m t U.5t) " ( )S.Stl 2Q (' Sl.l:t) "' { 1!11. 91) "' ( 91.211 

-· hi ..L!l.:.2ll " frl-itlli " ~·:.ill 19 .u!.:.lli " .L!l.:..!.!l " ~-8l! Toc•l "' (lOO.Ot} m ( ) Tii (IQO.Ot) 3i (lOO.Ol} m (lOO.OZ) "' ( .02:) 

lhck II ( 1.5!.) ,. ( 6.61) , ( 50.0!) ,. (· 60.0:} " ( ).11} " ( ~-1'U 

llhpa.o.ic: " ( 2.,0,1) ' ( l.H) " ( 10.9t) ' ( 7.31~ II ( l.lt) • ( .11) 

Wtaa 10 ( • l.U) 1 ( 

-"' 1 ( l.(tt,) ( -- ) • ( I. [1) I ( .21:) 

., .... "' t aa.o:n '" Wt " DL!!l II 

~ '" ( 94.5:1 lli .i...ll,:.::!l 

Totd ill (lOO.ot) m ( m (HiO.Ol) ;;;; ' ill (lOO.Ot) '" uoo.o:1 

Descriptive Statistics. Of the total 953 ATC trainees, 582 ~en and women 

and 592 traine-es based on rc:ce had complete data on all test variables. The 

next step was to c~pute a total weighted test battery score and descriptive 

statistics using };CAT 1, MCAI 2, CSC 157 and CSC 24 for the total group; 

those who passed the ATC training and those who failed or withdrew. In 

addition, the number of students, means and standard deviations for th~ 

Predevelopm.entcH trainees (non-competitive appointments) and ncn-predevelopmental trainees (competitive appoinrments) are shown in Table 91. 

At this point, OKI s~ores were excluded, the tests used were weighted as 

follows: 

2(MCAT 1) + 2(MCAT 2)_ + 1 (CSC 157) + l(CSC 24) 

Weight-s- were. deri·.,e-d from- the re-gression analysis which is discussed later. 

TABLE 91 ---- ---· DESCRll'TIVE STATlSTICS-1"1<0 DEVELO!'NENTAL CATEf.ORJES 

:W- Fa -:til --·~·--- ---~---

l!!IH J!1 !Ill! .Ill! Jl!l 

-"" - ""' zu.e UJ.J) u ... ,._, (!1.91 flit) zu.z (Z..f) 

- ~-· --· (U.t) "" ZU-J u.z .• ,_ ( Ul U$.1 tH.'IJ 

- I II) , .. _. (U.II I., ..... (U.I) "" UI.S u•.n 

-· I .. , .. _, en.» I • UI.O tu.n I " U•-• n•.u M- I " - I:U.O . -' 1-> 1-> I .. U).G 1-. 

- ..... ..... Ut.lJ fl .. , UJ.O trt .. ) 1$6)1 ..... (zt." ..... _ (..,..,, 1-' ~-· <-' Clt.JJ . .,_ .. ur.,, 

P:f!PF!!h!!I'!IIPJ.U. 6ilj !WW! 

---- aiililiii ------'-

!I!!!!I J!1 .... .Ill! J!l. !!k'!' .!.In 

- um Ut.J ,., .. , CUtl Ut.l '"·"' f•tJI .... ) cU.tJ 

- < "' 

,.._, Ul.tl I U) ns.s , •• 1) 1"1 u •.• (ZJ.U 

..... I .. nt.r nt.•l I .. MJ.t , ... , C U) II old (U.tl 

-· I " ,.,,, uo.n I " "'-' ,.._n I .. ,.,_. fU.J) 

M- c _u us.o 1-' 1-> ·-' I " ,,_. 1-> ..... UMI ..... en.•• u•• ur.• Uf.•l (Sill ..... (0.11 

-- 1-> 1-' c...:.) 1-1 OUt IU.t u•.u 

....ra&.m~~~ 

.... :;---' !ill -;;: -!iii-

.1!1 '"'' Jlll§i--,;;t" 

- I " Mr.• tU.I) "" ... .. (U • .Z) I"' IU-l UI.Ol 

- I .. utoJ" Ul.tl CUJ ,.._, UJ.Ol c 101 au.t UI-U 

., ... I " U).l '"·" "" .... , (lto,t) c ,., ..... Ul.'l 

- I • lll.O ' 1.1) I " .,_. 1-> I " UJ.O UO.GI 

M- ·-· 1-' 1-> 1-' 1-> 1-' ..... , ' " ... _, ,._,, ' .. UJ.t '"·" ( Ul ,,._,. UI.G) 

1 .. &1 .... ·-· 1-> 1-> 1-' t ... Ut.J c».H 

112 

I 
It should be pointed out that the total of passed and failed trainees in. 

s_Oitie cases do not equal the number shown for the total group. For example. 

the 320 men who passed. and the 189 who failed is ')09 compared. to 515 shown 

for the- 'total group. this is a result of having 6- trainees in. the total groupfor whom. information by sex was not available~. 

The means and standard 

derived, ineluding Olcr. as follows: 

deviations for this same 

The total weighted test 

group of ATC trai~ees were 

bat-tery score was established_ 

2(MCAI 1) + 2(MCAT 2) + l(CSC 157) + l(CSC 24) + l(OKI) 

Since the OKT is not intended for qse in determining if applicants pass the 

ATC test but rather as an alternative to the present OPM Rating Guide for 

establishing extra credit for aviation-related knowledge, it was given a 

weight of "1" in deriving the weighted toea1. The results are provided in 

Table 92. 

TABU 92 

EFFECT OF OKT WUGHTS ON VARIOUS SAH!'LES 

TOTAL .tTC TL\U!US . 

---·-tASs-- --FAIL----- ---NtA--·-----

CIDIIP ill ~ .!!! ill ~ .!!! .l!!l. !!!>!! .!!! 

.... (310) Jll. r (1~.6) (189) 194.0 (.2 9.!11 (51~) JU. l ()O.ll 

·-· ( )4) "". l (28.!1) ( )1) 275. 2 (ll. ]) ( 67) 291.4 {l_4.~) 

a}.atJt ( 121 110.2 (I B.O) ( 26} 261.8 (28. 1) ( 39) 177 .l ()).8) 

YUp.anJ.e ( 4) )11.0 ()4.3) ( )) 212.) (ll.O} ( " 295.0 (}2.8) 

Ulan ( 1) 119.0 ( -- ) ( - ) (-) ( 1) lt9.0 ( -- ) 

Other (345} 320.5 (26. l) (194) 29!1. 2 {29. 4) (!145) )) I. J 00.0) 

fatal (;;raap' ( -) (-) ( - ) ( -- } (592) 309.0 ()1.4) 

___ tkHI-PUDEV£LOPH!I'n"AL e_c· t~JKtES ,,.., __ ----F.UL ---- ---ror.t.:.._·--- CJDI!! ill ~ .!!! ill ~ .!!! ill ~ .!!! 

... (31)} 321.6 (25.6} (17~) 295.9 (21.6) (495) lU,2 (29. 4) 

- ( 16) )08.9 (29.0) ( 21) 27.1.4 {)4.4) ( .. , 294.1 ()4.fl) 

IlK it ( " 302-.0 ( 1!. 2) ( ., 259.9· (11.2) ( 15) zn.o (24.!) 

aJ.•,.U-e ( ,. llLO (55.2) ( ,. 271.0 (U,6)· ( 4) 296.) (43.5) 

... u ( l) )19.0 { -- ) ( - l (-) ( 1) )19.0 ( _.;. ) 

OtMr ()18) )20.6 (26.0) (181) 29S.6 (29,4} {5l2) lll.S {29. 8-) 

total Group ( -) (- l ( . ) (- l (552} liO.S UO.J) 

raG!VnDPKrlCTAL AT9; TV.l!CUS 

-PAS$- ---FML --- -·--toTAL---

...., ill !!!!!! .!!! ill !!!!!! .!!! ill ~ .!!! 

... 7} )14.4- (:11 .. ( lJ) Ht.l C34.l) 1 20-) lf«.l t•t.n 

- ., )02.3 (21.1) ( ll) 269.0 ()1.6) ( 19} zn.o (ll ... , 

llaek ' 7} )l6.0 (ll.l) t 17) 26l.t (3).4) ( 24} 271 •• ()1.9) 

UQ&ai.C ( ,. 103".0 (12.1) ' I) 271.0 (- l ( 1') 2tl.O (l9.5) 

..... ( - l (-) ( -) (- ) ( - l (- ) 

oc,..;. ( 7} >tM ..... 6) ( .. 115.-2 ,(29,0) ( ll) >OU {)7.1) 

tKal· c;;rcN, I - I (-) I '·) I-l ( .. , 217.-1 '"·* 

113 

~-- <. - ----~-·-...;...~_,..-_:;-:':h·~ .~ 

' •, 

I 

./ 

I 

,. 

. ·• 

' 

. j .. 

' .• 

I 

\ 

( 

, 
The di,fference in test performana:e betw-e-en those who pass and those who 

fail are. consistent, fo-r the total group and_ for the Subgroups. For 

exampJ~, the =ean score (Table 92) for the pass group is consistently 

abov£. 300 while the mean s~ore_s for those who failed is consistently beli)W 

300. Generally there is a separation of about one standard deviation 

between the mean scores of these who pass and fail for each of the subgrbups. 

For the total ATC sample, the sma-llest difference in mean scores is between 

the ''Other" ff111 group (295·. 2}- and the "Wome_n" pass group (307. 3). 

·This pattern generally hold's· for the weighted test battery which exc.ludeo 

OKT (Table 9r~. The most evident exc.eptions are the mean scores for the 

"Men" and "Other" non-predevelopmental "fail" groups compared to the "alack" 

pass group. However, the number in the Black group- is small (5) and the 

mean difference sligh~ especially when compared to difference in the stanqard 

deviations. 

Iatercorrelations.and Multiple. Regression Analy~is. The next step was tO 

relate test results on parallel forms of MCAT and 0~ together with CSC 24 

and CSC 157 to the ATC Academy rns or fail/withdnw criterion. 

Tabl~ 9-"l presents the 1ntercorrelation o·f thes.e tests for the total ATC 

saple and for -each subgroup by sex and racee the correlation of particular interest are those between the tests and the criterion. Thes·e 

ar-e zero or4er val.idity coefficients. 

,. 

' 

. ~' 

-I ' 

/ 

' 

\ .. 
_--:.-:_-·_. 

Tea~ 

IICAT l 

MCAT 2 

OllT 

tSC 24 

esc; U7 

Cr .l.'ul'lon 

... 

... 

"' 

!!!!. 

KCAT l 

Mci.t 2 

OK't 

esc 24 

.esC U7 

,Cr-iterion 

I 

."~· .... ;;• - ~' 

;" ~: ·.: .... 

!!..!! 

MC.I% l 

"CAT 2 

or:r. 

esc 24 esc 157 

Cr1tedOA 

T~.BLE 93 

nm:RCXJRRELATICNS-PCYl'l'l'TI'li'LTFST BA'I'I'ERY VARII\BLES 

TOTAL ATC TRAINEE SAMPLE 

(MinU.UUI N•585 -- Criterion- and CSC 24) 

MCAT l MeAT 2 O<T CSC-24 CSC-lSJ Criterion 

(N•IJ~)) (N•95)) (N•9.5l) (N•S92) (K•S~2l (N•S85) 

- ••• ••• ... .!1 ·" - • -iz .20 .20 .!7 

- - • 24 .... 14 ·" - ,10 .10 

- ,II 

le:M ATC TRAI~ SAMPLE WOKEN ATC TRAIN"EE SAMPLE 

(Mlnl.aua M-509 ..... Criterion apct esc 24} {Htnhua lf--66 --·Criterion and CSC 24) 

"""" l HCAT-2 OkT esc. z• esc 157 Criterion K<AT I MCAT 2 OKT esc 24 esc 1' 1 

(M•800) (N•800) (M•800) (N•515) .Q!.!ill!. (N•509) !!!!. .lli;!.!.U .1!!!!.'.!1 ~ (1'1•61) .!!'.:.; 'l 

- • Sl .OS ·" ·" • )4 KCAT l . ... .II .12 . ·.z 

- ••• • 21 ·" .!7 IICAT2 - .II -.01 " - -.23 -.u • 2l O<T - -.2S .01 

- ,10 .12 GSC 24 - ,13 

- .16 esc 151 - - Crlurtcm 

OTIP.l M!IT!l ATC TltAlNEI SAMPLE .LAC¥ ATC TRAIMEE sAMPLE 

(Hiniaua N•Slt -- C·~terlon and CSC 24) ~1ntaua N•l4 -- Crlt~rt~n ~nJ CSC 24) 

..... 1 HCAT 1. OllT esc 24 esc an Criterion HCAT I HCAT 2 OIIT CSC2ll esc JH 

(M•U9) {K•8l9) (N•&J'7) (N•54S) (~545) (Nio~)9) Tnt (N•dl) .1!!:!!.1 .i!i:.lli !.!!.:..ill. (14 .. }9) 

- .51 .10 .28 .21 .29 M.t".AT l - ,6) .)] ·" ~. 29 

. .13 .19 ·" .ll MCAT 2 - .22 .06 ~. 24 

- ... 2"' ~.11 .ll OKT ~.10 ... ll 

- ,10 .01 esc 24 -.01 

- ·" esc 157 

- Cr:tterioa 

•, 

Crlt•rloa 

(lt•66) 

.!7 

.38 

·" ."08 

.40 

Crtterton 

i!!.:.lli 

·" .45 

.24 

·" .07 
\ 

It sho.uld be pointed out that these v-aliditY coefficients are for-. a restricted population. Cor!'ec.tion for -restriction in range was not a- concern 

in this analysis since the objective was to com·pute regre-ssion equat-ions 

and determir\e the reliltive Contribution of the_s~ variables for predicting 

course pass/faiL However, Table.~:~ provides a comp.arison <?f these validity 

coefficients with the restricted and unrestricted coefficients obtained for 

the prior AIC sample of 1,827 trainees who attended the FAA Acad_emy- during 

the period January 1976 - April 1978. 

TABLE 9~ 

\',\LIDlTlES - POTE~aJAI. TEST B:\l'TERY VAl~L\BLE~ 

ltCI.1' b'r• (Z SCOUI) 

K~ l <-.v •coru) tiCAl' 2 (._ •cor .. ) 

t'loC 24 

:::.c U7 oa 

Ate Tr • .,_.,., •• 

J-.. 1976 • AP!U l9111 

.2& 

.10 

.07 

.n 

·" 

.34 

... 

AIC tr&Ul•••~" June 1?75 - ~sc. l'H8 

... U'ic:t..S 

' 

.n 

.)'l' 

.10 

.1& 

.n 

• Crtttiti- • AJC -lAb. Awn•s• Sceu (% •cora. See Tal>b• 2.5 *ld 2"6) H Crtt.r't.OII • Aud-r Tn11l141 ..... /fail (Lab. Traitiq) 

The restricted correlations were next used in multiplt! regression analysis. 

The first regression analysis employed MCAT l, MCJJ 2, CSC 24 and CSC 157 

as p·red"ictor variables. OKT was· excluded since it is not inte;nded as a 

screening test for applicants. Each of the four tests were included in the 

regression analysiS in ord~r to ascertain its contribution to. the multiple 

eorrelation coefficient (R). The results are shown in Table 95. 

TABLF S5 

REGR~:SSION ANALYSIS - FOUR PREDICTOR \'ARIAl\LES 

,., ... ·-- Al'C $Nt11.& - Jl!ll --..wotP-- ...... S) (M-SMJ (Oo66) 

!!!! ! !! !!!! ! !! !!!! ! !! !!.II. 

IICAt I .J41 .I:U .11)2 .»7 .114 .1&00 .315 .140 ,1430 

IICAt 2 .... ,164 .2470 .400 .160 ,2))4 ,4U .no .2259 

esc'' .... .164 -.0109 .... .160 ·~OUt .411 .115 .0)5) 

esc 1.57 .416 .17) • 09,7 .... .16} .069-l .m .21) .)240 

..... --(VIIrn<)-- -wa---

-lfl (¥-Jl) 

l!l!. ! !! !!!! ! !! !!!! 

ltCAt I ·"' .015 .Ull .56S .)19 ,4734 

IIC.U 2 .us .U6 .2.294 .579 .))) .1016 

esc 24 .us .tH- ... 0110 .... .... .28lt. 

...... , .lOI .ns .onz ·'" .41$. .2601 

llj; 

. ....., 

,. 

' 

I 
Of the four tests, esc 24 and esc 157 contribted to the :nultiple correlation 

j:oefficient for the "Wom~n~~ or "Blacku groups but made littl-e or no contribu-tion for the total group as shmm below: 

CoQtl'"ibution to R 

Group esc 157 esc 24 

--·-· 

!len .006 

"Women .104 .007 

Black .049 .061 

Total .011 

Other regresSion analyses YeJ:e- conducted for the total group which ~xcluded 

CSC 24; the multiple correlation and beta weight!: obtained were essentially 

the same as those de-rived when esc 24 was included. 

In terms of the total group. the findings for esc 24 are consistent With 

results for Qther ATC train~e samples and AT\; applicant groups·. For ... lR 

sample of 1,827 ATC students (1976-1978) previously presented, the CSC 24 

contributed -le;ast to the multiple correlation and had the smalles;: beta 

weights (Tables 27, 28, and 31). For the ATC applicant group (1976-1977), 

CSC 24 ce>ntributed least to passin!s the current OPM ATC test (Table 19). 

However, since CSC 24 and CSC 157 do contribute to the prediction Jf the 

criterion. for some: of the groups, the· decision was ·co include c:1~ in the 

test ba-ttery for further analysis·. Cori.seque'J.~l:;t, the weighted test battery 

used in developi.n'g descriptive s.tati.sttc:s and for examination of the tests 

consisted of: 

2(!1CAT l) + 2(!1CAT 2) + l(CSC 157) + l(CSC 24) 

Table 96 provides the results of the multiple regression analysis for the 

four test-s p-lus Olcr. In this case, the order in which each test entered 

the regression analysis was det.ennined by its contribu-tion to the multiple 

eorrelation coefficient. As shown for each group, the tests ~ntered in a 

diffeTent order, espec·ially in the C(lSe of "Women" and 11Blacks. u 

TABLE 96 

REGRESSION ANALYSIS-PREDICTOR VARIABLES A~~ OKT totAL 

-ATC SMCPU-- ·--wot!EM---· 

(01-51$) (llo>Ot) (••66.1 

I=!1 ! !! !!.!.! !!!!. ! •' !!!.! .!!!1 ! !! !!.!.! 

""" ' ·'"' .140 .2134 ICAt' _,.. .136 .2%64 esc U7 ... , .16Z ,2997 

-.ut .... .2471 ""' .Ul .117 .2341 ""' • .Sll .liZ • ]~52 

-· .u.s • 207 ,1547 liGAt 1 .... .200 .Ul9 IICAtl .592 .J'U .12~ 

esc u1 ,414 -~ .un esc ur .... .211 .1109 IICAr ' .60> ·"' .117) 

..... .471 .121 .Mil esc •• ,462 .ll4 .0)30 esc •• ,611 .376 .un 

Otml 

(VHITl}------ ·~....----- (IMlt) (11-Jil 

I=!1 ! !! !!.!.! !!!1 ! !! !!.!.! 

IICAt Z .na .101 • 1924 ..... .... ,)19 .3573 

oct ,391 .UI • 25-ll esc 24 .6ZS ,)91 .])60 

esc u1 • 4~4- .... • 1397 ... .... .... .l•U5 

IICAT.l .4')1 .191 .un esc ts't .744 • 55) .2791 cac,. .... .... .0476 II<At l .1 .n•-- • 1'974 

117 

' . 

11 
Tab1e 97. shows, in Sltmm~ry, the m4ltiple corrE::Lltion coeffi.cient (R) values 

derived by gro~-p and. the variance (RZ) accountet! for in the pass. or fail/withdra ... · 

c:r!terion bas-ed on ~he restricted zero-order cor~elation coefficients (r). 

T:\P.LE 97 

Sl~C.L\RY \'Al.l'ES -· ·!\·E·,;!£tss 1 u-:: ,\::;..LYSES 

Test. T .. t Bat t.e;y 

~ ~ 

Group ! !! ! .2 

Total Sample .416 .173 .477 .228 

Othe.r .368 .135 .440 .194 

Keo .406 .165 .462 . 21~ 

w-• .523 .273 .613 .176 

Uat.k. .689 .475 .759 .S79 

Test Ut:Llity. The purpose of this analysi_s va:~ to examine the prac~ical 

utility of the new AT~ ·test battery, relate the findings to th~. in!"omatio_n 

obtained for the 1978 ATC App lie ant Group and es.tiloate the potential FAA 

training investment costs which could be saved· by using the new ATC test 

for selection ~f applicants. · 

As previously noted, the weighted test scores for thie s ..,ple. of 585 ATC 

trainees was based on: 

WT1 • 2(MCAI-1) + 2(MCAI-2) + 1(CSC 1~7) + 1(CSC 24) 

J!owe.ver, for the 1978 A.TC Applicant Group, only one parallel form of MCA.T 

was administered rat!ler than the two which were given to this group of Al'C 

trainees. In order to be able to relate the two groups, r.he. weighted ATC 

test scores for the 585 ATC trainees were recomputed using only the first 

MCA.T administered and then weighting each of the tests on the same basis as 

was used in the analysis of the 1978 ATC Applicant Group. The weighting 

used was: 

wr2 • 4(MCAI-1) + 2(CSC 157) + 1\CSC 24) 

The total raw weighted scores were then tr&BSlll\lted for eacll trainee based on: 

Ta • 10 68{RWS- 222.27) + 70 " S6.22 

as was. shown with Table 77. Again, a transmuted score of 70 or abov& wa10 

e(juaeed to "pas·sing" the new A.TC test for this A.TC tr&inee group. 

118 

' 

I' 
ln or4er to assess the d~fference between these two weight;ing __ filctors C\.11 8lld \JT 2

). correlation coefficients were ccmpute.d betveen the two veighted 

~cores on the AIC test batt.ery and he·tween these and the scores on the 

present OrM test battery vhich ~Jere also available fo-r the ATC trainees.· 

Table 98 provides the d-escriptive statis.tics ah.d correlations. 

TABLE 98 

~OUPTIVF STATIST! C:S A."'D I}!l'ERCORREIATIO.·:S, ATC-<JPM TESTS 

T~st !' ~ so 

Oftt S~oTe (law) S~l lU.O 2). )1 

ATC rut ·(\011 J.a.w) >92 24S.6 27.S7 

Att Iut ~"t2 bv) S92 272.6 JS.l 

-oPM ATC Score ATC Scon 

Sc::or~ wt, •"I, 

OP!1. Score .62 .n 

ATC Score - \ll1 

.93 

A-TC Scon - WT 2 

Sine.:~ the ,,,.eie;:hting used to examine the utility of the n~· ATC test battety 

(t..'T2) has a -similar higher correlation coefficient \dth the 0?~1 test scores 

(.65 compa-red to .62), the results should tend to be similat". 

The mean tra.nsmuted score on the present OPM test battery for this group of 

ATC trainees was 87.2 with a standard deviation of 7.6~ Their mean transmuted 

acore on the new AXC test battery ) was 79.6 with a standard deviation of 

6.7. This lower mean score is consistent with tbe findingo derived from the 

1978 AXC Applicant Group which demonstrated that applicants who passed b0th 

the present OPM test and the new AXC test, generally score lower on the ATC 

test (see Figure 13). 

Por this sample of 585 AXC trainees, Table 99 compares the score distribution 

b~~~~-t~~~~~~-t~~f~tbet~l 

sample and for the Predevelopmental (non-eompetitive applicants) and No~­

predevelopmeotal (competitive applicants) &roups. These are transmuted. test 

scores without Veterans Preference or aviation-related knowledge/experience 

credit. The 585 AXC trainees for whom complete test and criterion data were avail 

able represent 61 percent of the 953 attending the kadei!>y durillll .:une throu2h 

December 1978. The 40' Predevelopmental trainees represents 36 percent of the 

non-competitiVe hires; the 545 Non-predevelopmental traineesrepre~ent 65 percent 

of the competit1ve hi;res. 

J.l9 

' 

.. 

- -·· I 
'ri'B!..E 99 

SC\:'RE DIS'l'RIBL'l'Ict:S-585 TRAH:sES 

TOTAl AtC TlAl~~ES 

Pr~~.~:lt 0~ Nev. AT( s~ .... r• fe-st Te~r--LJ::'T~) Rans~t ""! ~ --r- .;_ 

9h IZI 20.:: 2 .)1 9Q-9 .. u~ 2Lt.:. 27 4.6':. 8~··8~ l)l 2:.6: 98 16.8: . 80-to. .. llC 18.8: 161 27.6':. 

7S-7Q " 10.8: 14 7 2S.l!. 

1Ci- '1 .. B S. ;:, to::: 1 7 ... ~_ 

"Pass'' T~st 5S~ 10o.o~. re ~<:.rt•. 

6S-69 

)~ 1 60-t~o. 8.:!·! 

5S-S9 

"'tat: .. 'feat o ., m 100.01 m 100.0! K~nn 8).2 79.6 SD 7.6 6.7 

I«)~"Jl!D. TlWJ\Et$ Pfi.SlE\'. TltAl~"!£S 

!Cam.fetitive W.ru) {Noncoee~titfve Hires) Pr~s .. nt Or.! ~_. ATC Pres.-nt OP~ KN ATC Scot• Test Test ~\.."1'2) Test Tut (I;I:) J..llnu ~ .; ~ ... ~ .. !' ,;_ 

·~· 11• 21. s: 2 .4: 2 4.6 0 90-9.i !" 22.4: " s.o: ) 7.0 0 85-89 1 ;::q 23.7: ., 11. a: 4 9.) l 2. 5! 

80-8:. 10! 18.5! n: 28.8:. • 23.) 4 10.0:.: 75-79 ~:\ 8.8:0 1)6 24.9: n 37 • .:! ll 27. s:. 1()--7:. ,. ~s_; ..29 .!U3. 7 tiLt> " 30.0~ .. Pa:ss" 'tests 5:.5 lOQ.u:;. ~09 ~ 40 100.0';. 

65-69 " ~ )i1 -.~ 60-b4 6.6: :) 30.0! 55-59 

"Fail" Test 0 J6 0 I~ 

~·~ ~ m T"O'O":O"i ;;o ~ 40 100.0:. 

Two significant points are evident from the data i.n Table 99. 

First, 48 (8.2 perce11t) of the 585 A:rC trainees, all of whom passed the 

present OPM teat for t!lllployment eligibility, would not have passed the new 

ArC test and"therefore would not ~ve been eligible for appointment as an 

air traffic controller trainee. When the total group is further idenrifi=d 

by cOillpetitive hires fr<lll established OPM certificates (Non-predevelopmental) and non~co~etitive hires (Predevelopmental) who were hired through alternate 

recruitment procedures, other differences are apparent: (1) )0. percent of 

the Predevelopmental Group would not have been eligible f:>r appointment in 

contrast to 6.6 percent of the Non-predevelopmental hire~> (this 6.6 percent 

compares to 7. 2 percent of the 1978 A:rC Applicant Group who also passed ·the 

OPM test but did not pass the new weighted (VT2) ATC test; and (2) 23 percent 

of the Non-predevelop:nen_tal tra"inees score<i 8S or above ou the new AIC test 

in contrast to 2.5 percent of the Predevelopmen~al group. These dat• provide 

some further quantative insight into the de&eriptive statistics on the mean 

score dHferetl~es be !:Ween the"e two groups which were identified in Tables 85 • 86, 91, and 92. It sho~ld be pointed out that the trainees in this sample could 

nOt score lowf,!~· than 70 on the 0}1..1 and thus had b. have hi.gher scores. 

120 

' I 

,. 

;. 

,. ~ 

' t 

\ 

;... ;, 
' ' 

Seconds· the score- distribution of these A:tC trainees on the_ new AIC teSt 1$ 

markedly lower than on the present OPM A!C test. About 42 percent of the 

total trainee group .scorE"d: 90 or above on the presen_t OPM test _in contrast 

to o-nly 5 pe-tcent who scored- this high on the new ATC test. In the Nonpredevelop~ental group, 44 percent scored __ 90 or· more on the OPM test compared 

to 5.4 percen_t on the new ATC test. Of the Predevelopmental trAinees, none 

scored above 89 on the new test but almost 11 percent did on the oYM test. 

These find>ngs are consistent with the results obtained for the 19·78 ATC applicants where passing scores on the experimental ATC test wenr significantly 

lower than on the- OPM tert·. 

Given this marked dtffere:>ce in score distribution between the OPM and the 

new_AIC test, the next step was to relate the score~ on the present OPM and 

new ATC t~st to the paS$ or fail/withdraw criterion. Test scores and criterion 

data were available for 695 trainees for the OPM teSt and 585 trainees for 

the new ATC test (W!2). The results are provided in 'I able 100. Again, these 

scares do not include ext.ra credit for veterans p-refe-rence or aviation-r-elat-ed 

cxperieru;:.e or len~ ledge. 

TARLE 100 

ATC TRAJN"EES ATC TAA I NEts 

-------------cPM rtsr--~----------- -----------NEW ATC TEST (\lf2)-------

"TC trng FaU/\lD ATC Tl""n~ Fail /'WD 

~ Fait/1..1> R~u.· Tot~ hil!WO ltatt> 

!! (%) -r !! ~%.) --.- -;:-

··~ (10.0'%) ll 22.8! 2 ( .)t) 0 

~4-8 (11.3%) 47 ll.Bt 27 ( 4.6%) 5 18.5% 

156 (22.5%) 62 )9. 1% •• (16. 8%) 20 20.4% 

Ill (18.8%.) 59 45.0% 1&4 (27.6%) 46 28.01 

76 (10. 9% 38 5-0.0% 147 (25.l%) ., 42.2! 

)9 ( 5.6%) 22 56.4% _,)92 (I7.4t) 52 51.0! 

....!! ~ . --- ~ _u,m ]! 19.!,2! 

69> 100.6% 261 37.6% 585 1()0.0% 223 38.·1% 

Table 100 clearly identifies the difference br.tween the present OPM test and 

the new ArC test with regard to the percentage of trainees who successfully 

complete the ATC training in each of the ATC test score ranges. For example, 

with scores between 9Q-94 almost 32 percent of the OPM test group fail or 

withdr"" iD cor.tparison to only abcx.t. 19 percent of the group with test scores 

in this range on the new ATC test. Of particular interest is the fact that of 

the 48 ATC trainees who scored be.low 70 on the new ATC test, 79 percent did not 

successfully eomplete the trainiog. 

Table 101 provides the score range distribution and failures/withdrawal rate 

fO-r the 5.85 trllinees io relation to the new ATC test identified· by Non~predeVl!lopmental and i'·redevelopmental AtC trainees. 

121 

' i 

/ 

. : 

·---. 

I .;-

.. , 

I 

I 

. ' 
TI\BU: 101 

FIUUJRF.;\,'I1,UlfW·'.'IL Rc\TF. DUTRII'l.. "l"Ia-5 BY DFI:El.DP~:n:rn CN!IXXl?Y 

HCHH'MI:.:.P. ATC Pl\f.DP'· A.TC 

----~----TkAJNEES------------ --------· ·-TllAl :fi:ES-- ----------

N.-u t'al tt..:n Fa.il!Wil 

ATC TO!S[ T •>t..tl t',lll {Wll ~ lot a I Fatl/t..'D R.o« 

Sl..'Oh' R..antj.L' ~ E- ~ ! -y- ~ ! 1 

9S+ ' Q Ol 0: 0 0 01 

90-9.'. " ' :L~! 18.Y; 0 0 01 

85-89 97 20 10.1! 2'0.6~. I 0 01 

so-st. I'> 7 •• 1).1~ 29. 3:~ 4 0 01 

75-71} I Jb S7 28.6:. 41.9: II ' zo. e: 4S;S2: 

70-74 •o " 2l.b:!. 47 .Ht I 2 • 37.51 75.01 

Below 70 -1!! ~ 14.1: 17.:..81 !~ !2 41.7: ~)..:_);. 

Total "' 199 aoo.o:. )6,5: 40 ,. 100.0.: 6o.o: 

these data focus attention on the 48 trainees who did not Pass the. new ATC 

tes·t battery and who would_ not have been eiigible for appointment. if 

selections had been baSed on the· new test. Of the 48, a total of J;,. either 

failed or withdrew from the training; a loss rate of 79 percent. For NonpTedevelopmental trainees the loss rate for this group· was 78 pe_rcf:nt and 

for Predeveiopmentals it was 83 percent. 

This has a. direct and important impact on both recruiument for the air traffic 

control occupatio.n and th~ cost of training for FAA. Had these 48 trainees 

not be~ h-ired", the overall los·s rate would be reduced from 38.1 percent to 

34.4 percent and by t!on-predeve-lopmental and Predevelopm_ental groups, from 

36.5 percent to 33.6· percent and from 60 percent to 50 percent respectively. 

For each Predevelqpmental trainee, FAA invests about $28,000 !n Salary and 

training costs and about $10,000 in each Non-.predevelopm.ental student by 

th<> tillle their FAA ACademy ATC training program is comp}eted. For each 

student who fails or wit.hdraws from training, this investment ls lost and a 

new trainee has to be hired. 

If the 38 trainee• had not been hired based on their failure to obtain a min:!Jllum 

arore of 70 on the nev ATC test, FAA's loss in training investment costs would 

have been reduced by $560,000. Extending this for 2,000 new trainees entering 

Academy training on an snnual basis would result in a cost avoidance of about 

$1,350,000 eaeh year. 

This projection of the cost avoidance utility of the new ATC test battery 

addresses only the benefit obtained if ATC applicantswho &cored below a 

llin:!JIIUlll of 70 on the new ATC test battery were not hired. The other significant area where benefits could be obt:<tned is to determine the differeace 

in the group· of AIC applicants who would be hired under competitive selection 

procedures ~ith the current OPM test and the new AIC test with allowance of 

ext~a· credit for veterans Preference and aviation~related experience OT 

knowledge. 

122 

/ /\ 

' 

.:-i 

.. , '. 
./ 

In order to exam-ine these differences, the OPM Earned Rating which._ is 

derived (rom the:. ATC tr::tinee' s score on th'f present· OPM ATC. test batt(!:ry 

plus additional credit for -.-~terans preference (0 to 10 point;s} and -credit 

for aviation-related experienct: !lrovided by the orM Rating Guide (D-15 

points), was available for 621 of c~e 843 Non-predevelopmental ATC trainees 

in this sample. The 110 Pre-developmental ATC trainees w-ho were appointee! 

non-compl:!.titively were not included in this analysis since :::xtra credit 

for veterans p.refer-ence or aviation-related experience is not a factor in 

their appointment eligib~lity; they need only to pass tht! OP"!'t test battery 

With a score of 70 or above. ThP. "~quivalent" of the OPM Earned Rating 

usin£; the new- AIC test battery (WT2) and the ATC Occupational Knowledge T~st 

(OKI) ~as derived for the Non-predev~lopmental (competitive- hires) ATC 

trainees. The 36 trainees who did not achieve a minimum score of 70 on the 

new AIC test were excludeQ from the analysis since they would not have been 

eligible (or appoin~ttient. The 11£arned Rating11 for these trainees was based 

on their scOre on the new ATC test plu-s additional credit provided for their 

OICT scores as follows:· 

OICT Additional 

Scores Points 

80+ 15 

75-79 10 

7Q-74 5 

65-69 3 

The difference between the OPM transmuted score an~ the OPM Earned Rating 

for each trainee was the additional credit given· for vetera.""ls preference and 

aviation experience based on· the OPM Rating Guide. Since the veterans pre~ 

ference points were not separately identified in the total of extra credit 

given in the OPM Earned Rating, it was not possible to specifically id<:ntify 

those ATC trainees who were 5-point or 10-point veterans. 

In order to derive an "equivalent" Earned Rating wfth the nev IJC test which 

would reflect extra credit for veterans preferenc·e and aviation-related 

knowledge based. on OKI seores, the following procedures were used: 

OPM EARNED RAriNG 

MINUS 

• 

.!!= 

v1 is less than 5 VET 

OICT 

v1 • 5 VET 

OICT 

vl • 10, 15, or 20 VET 

OICT 

v1 is more than 20 VET 

OICT 

123 

I . 

~ 

points 

credit 

points 

credit 

(Mean • 95.1; N•540) 

(Mear. • 87. 2; N-591) 

(AVr::v1 • 7.9 points) 

• 0 

• 0, l • 5, 10, 15 

• 5 

• 0, 3, 5, 10, lS 

po~n·ts • 5 

credit • o, 3, 5, 10, 15 

points •· 10 

cr·edit • 0, 3, 5 ,. 1(1. 15 

-·--

. ~ .... ---· ' 

---------~ 

,. 

---------· ., /· 
The net effect of this pr-ocedure w-ould be a somewhat higher disttibuti,on of 

"Earned R4tin..;" scores for the new ATG tes"t tha_n WO\.'ld be the ca_se if specific 

veterans credit h:ad been available for e"ach ATC trainee. 

Given th-es_e conditions. an e'st:im~ted· total Earned R.atihg could be· coo.puted 

based oD. the new- AI'C teSt for 486 of the AIC -trainees a·pp·otnted ebm.pe-titively 

(Non-predevelqpmental) from OPX ·registers. the 4.8· students who scored below. 70 

on the new AI.C Test were excluded fro:G the. samp.le. table 102 compares the distribution of these Earned Ratings to the Earned Ratings for 621 trainees based on 

the present OPM ATC test battery and Rating Guide. The 486 trainees for whom 

estimated Earned Ratings could· be computed using the ne~ A!C t~~~ is a subsamole 

of the 621 trainees who had OPM Earned Ratings available. 

Earned 

ltatiD't 

Score &an&e 

100+ 

!l<J-.. 

eo-89 

7Q-79 

Tot·al 

TABLE 102 

NOt<;-PR.E:O£\'El.DP!'tENTAl. ATC TRAU.'US 

(Competitive Bires) 

.!! .ill 

:zos (33.0%) 

ll.C. (.S0.6%) 

99 115.9%) 

l L.lll m (loo.o:) 

£st.. lC~W ATC 

Earned Rating 

!' .s.;.t 

S4- (11.1%) 

143 (29.4!) 

229 (4i'.lt) 

60 (12.6%) 

'486 (100.0:.:) 

About 84 percent of the 621 ATC trainees had OPM Earned P..atings of 90 or 

more .. Since competitive selection from OPM reg.isters requires hiring ·those 

applicants with the higheSt Earned Ratings first. this dir:.ttibution of OPM 

Earned Ratings- would be expected. However., the Earned Rating.distributi.on 

bas~d Oll the new /JC test. for· the 486 trainees who are part of the sample of 

621, is quite different. 

_ Only 41 percent of the trainees have Earqed Ratings of 90 or above and almost 

13 percent ·are below 80 as cOmpared to les-s than 1 percent for the Ol'M Earned 

Rating. Consequently • many of the ATC trainees with Ea;-rned R.atings below 

90 on th·e new ATC exam would not have been hi;-ed comvetitively since other 

applicants. with higher ra~ings wou~d have been selected in their place. 

In tems of practical uti~ity of the new IJC test battery and OK!, these 

differences in Earued RaUng score distribution are of significance only if 

the pass or filil/whhdraw rates also differ between the OPM and new IJC Earned 

Rating. These differences are provided in Table 103. 

\ 

101}+ 

tJ()-9.9-

80-89 

7()-79 

To-tal 

/· ....,.,.~ ...,.,., .. ! . 

__ .... -

TABU: 103 

ION-PJUl)EVD..()PMDltAL ATC TRAINEES 

(Compettttve UJres) 

OPK Earned 

.,.,,,. 

!ot81 

-r 

205 

314 

99 

l 

m 

l'aU,ND R.n<t~ 

!. .! 

S4 26.lt 

122 38.9% 

47 47.51 

->· 66.6% 

m 36721 

En. New ATC 

lt:frned "lt.aJ:inJ, 

tot•l l'a11Am lay l! -. !' . ! 

S4 • 11.11 

143 33 23.1% 

za9 •• 37.6% 

60 ...M 66.3% 

... IU l):-5% 

124 

-- --~ ~----~---------·---- -· 

-15.2% 

-n.a: 

- 9.9% 

- .lt 

/ 

/ ' 

' -

---

--

.- . 

•/ 

/· 

, 

'· • •• 

"~/:,.-_" 

.. -__ ,,, ' 
\ I 

It h clear from the data in Table 101 that the probability of successfully 

completing th~ ATC training program is great~r in each of the Earned R.atin.& 

score ranges ab~ve 79-for the new ATC test battery. 

It i$ eviden~ .that an OPM reg-ister ba.sed on the new ATC Earned. Rating voul4 

result in hiring a somewhat different group of JTC trainees than was in 

fact -hired with the _present OPM register .. It is possible to relate- the data 

in Tables 102 .1.nd Hn tn t!n.' n·~ults ~-'btaillL'd f\•r tilL• iY7H .\TC ,\ppl iC'ant Gt<'llP 

on both the current OPM test and the new ATC test. This information would 

provide a basis for assessing the differences in selection of groups of 

appli~ants and the potential effect of these selections on fail/withdraw 

ratas and on training investment losse:s. 

The previous analysis of the 1978 ATC Applicant Group iden~ified (Table 76) 

the Earned Rating score diStribution of applicants who passed both the 

present OPM test, and the new weighted ATC test battery (liT ) • If this group 

of 621 ATC trainees had been hired frolll the group of 1978 Aic applicants bas.ed 

on Earned ~tin:gs ·obtained on the presen-t OPM exam, the relationship would be 

as shown in Table 104. For the 1978 ATC Applicant Group, the number in each 

Earned Rating category in effect represents the OPM register from which applicant 

selections wou14:1 be. made. 

TABLE 104 

PROJECTED EARNED RAT! ~CS 1978 AtC TRAH;EES 

OP!t Earned. oPH Eu-ncd lla t tna 

Qudtfied OPK Eam~d Rating bUn& Score 1978 ATC 1978-ArC Tr*inee~ "Selection" R.anse App.licaat• (CO.petltive lliret) ll.atia 

.!! .ill. .!! .ill. .ill. 

100.. 229 ( 7.5%) 205 ( 33.0%) (89. 5%) 90-99 644 ( 21. 2%) 314 ( 50.6%) (48.8%) 8G-89 1086 ( 3$. 8%) 99 ( 15. 9%) ( 9.U) 76-79 1076 ( 35.5%) Toui 3035 (100.0%) m 

l (__,J!) 

(100.0%) (20. 

~ 5%) 

The registers maintained by OPM for ATC applicants are by separate employing 

jurisdic'tions. ConSeq-uently, each of -the eleven FAA r-egions s-elects from its 

own certificate of applicants based on the geographic preference. identified 

by the applicant and the -Earned Rating. As a result, one reg.fon (Southern, 

for example) may be able to fill all of their ATC recruitment needs from 

applicants who have Earned Ratings above 100. Other regions (Alaska or 

Great Lakes, for example) may recruit a number of applicants with Earned 

Ratings in th£ 80's or even 70's. However, Tablel04 shows that essentially 

all of the 62~ ATC trainees scored 80 or above and, with a mean OPM Earned 

Rating of 94.9 for the tot·at group, selections. were mad-e in mos_t cases fram 

the "top" of the r-egisters with scores of 90 or. more. The selection ratio 

of 89.5 percent between the 205 1978 trainees a:nd the 229 1978 applicants who 

would have·· been avAilable wit~ Earned Ratings o.f 100 or more, is somewhat 

125 

,;'---_........ 

f. ;/ .-, /._. -·. 

·-

\ 

' . , •. 

\ 

/ 

l 

'I ""':.J .· ... 'il--

,., 

' 

. I 
----! '· 

higher than typfcally experienced by FM region$. Geperally, of those 

applicants offered ATC positions, about 80 percent ar.e selected; the other 

20 percent decline or are dropped fr,om consideration because of medical 

or se~urity reasons. 

The next ste_g was to project the :-omposition for a group of 6-21 ATC trainees 

by their Earned Rating if they had bt.~n selected from a compet'itive :register 

derived frQ:U!. t.he new weighted ATC test exam including _extra credit for veterans 

preference and OKI scores. 

The number of 1978 ATC applicants in each Earned Rating score group derived 

from the new ATC exam was used as the "selection register11 as shown in 

Table 1.05. This is the same distribution as previously provided in Table 76. 

A maxinu.m selection ratio of 80 percent in each Earned Rating group Was 

used as more representative O.f actual selection experience. Since fewer 

a,pplic.~nts have Earned Ratings of 90 or more based on the maw ATC exam, more 

selections would be made fr0111 alliong applicants with Earned Ratings in the 

So-99 range. Based on these factors, the projected distribution of the 621 

ATC trainees based on the new ATC exam would be as indicated in Table 105. 

TABLE 105 

PROJECTED EARNED RATING oJ:S:fii)BUTioNS, 197t! ATC APPLJCA.'\TS 

Q.M. . .Ufhlll _,, h-t. 1918 S.Lecd011 lt71 AIC 

Ka""'. r..tiJII AppUc.uu:• gc 'traioeu 'Ut10 

Score bus• ! .ill. !! .ill. .ill. 

··- 126 ( 4.01). 100 (16.1%} (110:) 

90-99' 402 ( U.91} m ( ~L9t) (80:) 

.._ .. U6l ( 37->'U 192 ( 10- 9%} (1•71) 

lo--79 1426 .i..V.:!ll 1 ..Lh!!l 1=1 Total 1I1'6 (100.0%.) m- (100.01) (201) 

With the OPM Earned Rating distribut.ion for the 621 ATC trainees and a projected Earned Rating distrib~tion based on the new ArC ~xam, the fail/withdraw 

rates provided in Table 88 can L·1 applied to each group to estimate the difference 

in loss rates between them as provid£~ in Table 106. 

TABLE 106 

DIFFERENCES IN LOSS RATE ESTHIATES BY TEXT EXAHINATION 

- ... ·---oft« ATC txNt-- --~-·-· ----ATC !UK-----·-- .......... 1971 Al'C fait/WD ... "'c Zit 19711 .UC ht. PaU/VD 

....... tl'dMU ! bt•..ru. tal'l:lecl 'tratn••• !aauja Scor•• ! J!l bttaa Sc.orea !! .!ll. 

.... ,., ( :!l.Ol) .. (26.3%} ·- .100 ( 16.11} ll (11.11) ...... ,.. ( SO."-) 122 (lJ, 9%} 90-:-99 321 ( Sl.9'l) 14 (2],11-) 

-· .. ( 15.91) 41 (4'1.5%} ...... 192 ( 30:·.91} 72 (37.6%) 

7D-7f ) ~ 2 (66. 61) 7o-79 ' LJ..,Jll . ...., m (100.01) m ()6. 21) To~d m (100.01) 1~ ~ U6..ll) 

126 

I I 

\ 

\ I. 

\ 

·t 

.,. 

.. , liJ 

i :~ 

.. .; 

; 
)· 

Table lOt demons-traces the potencial utility r,f the ""'" ATC ei<am in- terms, 

of an overall reduotion of the fail/withdraw rate of about 10 percent (from 

36.2 to 26.1) for this sample of 621 ATC trainees. 

With respect to FAA's trainin·g inv,..:.ttne-nt lOss, project1n.g this reducti·on on 

an annual basis for I ,800 ATC applicants hired through COtllpetitive selection 

procedures- wou~d resul~ in a cost avoidance of about $1,825,000 since significantly· more of the tr"a.inees should successfully complete the ATC training 

program. By ·combinin-g thi-s with the $1,350,000 reduction in investment loss 

resulting from not ·hiring those -applicants who pass the present OPM ATC test 

but fail the new ATC test, the potential cost· avoidance which could be obtained 

is in the order of $3,000,000 a year; 

127 

' ' 

'\ ..,.,.. --~· 

-----~------~"-.- I 

,, 

' 

'• 

I • 

t ., 

I 

I 
~~fOaM GUIDELINES ON EMPLOYEE SELECTION REQU!~ENTS 

The UnHono Guidelines ( 1 7 l jointly established by the Equal Opportunity Commi$s1on (EEOC), Departilient c>f Labor (DOL), Departmeont of Justice (DOJ), and 

the Office of Personnel Manag.ement (O?M). provides four basic requirements 

to.r selection .proce.ctUres in relation to equal employmeD.t opportunity. Basically, 

tbe~e·requirements are: 

• to determine if a select"ion procedure has an adverse impact- on. 

employment opportunities ·of mino"rities or women 

• if there is an .adverse impac: t, the need to validate the 

selection proCedure 

• when empirical data demonstrates validioty of the selection 

procedure (that is, it is predictive of or significantly correlated with important elements of job performance), the need to 

e~ine the fairness of the selection procedure 

• consideration and investiga.tion of alternative se-lection procedures.. \olbere two or more selection procedures which are substanti'l-llY equally valid for a given purpose, the procedure 

which has the lesser adverse impact should be used. 

The various research studies which have been discussed encouipassed a wide 

range of select·ion considerations and devices both of a cognitive and noncognitive structure. The analytic methodti applied combined new devices 

With present ()'PM $eler.,tion devices and evaluated their interaction with 

varioas criteria of performance to identify those tests and procedures 

Which provided the highest relationship between selection procedure scores 

and per·forinance crit;.eria whic:h are fun~tionally relevant to the air traffic 

control occupation. Based on this, the requirements of th~ Uniform Guidelines 

and other elements of EEO programs will be addressed in relation to the new 

AXC exam and selection procedures derived from the research. 

Adve{se Impact. For enforcement proceedings. the Uniform.Guidelines state: 

"A selection rate for any race, sex or ethnic gi;"oup wbic.h is 

less than four-fifths (4/5) (or eighty percent) of the rate 

for the group with the highest rate will generally be re:;arc:ied 

by the Federal enforcement agencies as evidence of adve~se impact 

This definition was adopted as a practical "rule of thumb" and not a legal 

definition, 

" 

ln the selection prc>cedures for hiring applicants for the air traffic control 

occupation .ad-dressed by this re_search, adverse impact e.an ot:cur. at two points 

in the selection process. The first point is es.tal>11shing basic eligibili~y 

fo• ""'Ployment aons.ideration by successfully passing the test battery. The 

aecond point is ~lie Earned Rating score. (including credit for veterans preference 

128 

I 

\ 

\ \ 
and aviation-related experience or knqwledge) which determines the c~petitive 

ranking Qf. applicants who pass the written test. 

The information base against which measures of adverse impact cqn be ic'enti,.-fied 

consist of the group of 5,976 applicants who took the present OPM and •~perimental ATC test batt.ery ir. 1978. Ethnic, race or sex data has not been obtain.ed 

l>y OPM for other grcups of <ip,;'icants. 

For this group· of 197& ATC applicants, the reutive selection (!).ass) rates by 

race and sex on the present OPM ATC tesc· battery is provided in I able 1 ..J7 · 

Comparable data for the new A:rC test {\ITz) is provided in Table 10~. Both 

tables exclude any extra credit for veterans preference or aviation ~~owledge 

since the purpose is to evaiuate the relative rates of basic eligibility status 

f~.r the applicants. These Selection rates identify the proportion of eac:;:h sex 

and rac.e group who passed the selection test battery with a score of 70 or 

GlOre. Based on these rates, the 80 peJ;""Cent "rule of thumb" w.as a?plied to eacr. 

group in rel,ation to the majo.rity group selected. to derive the "adverse impact 

.. ration which would occur at this point in ::h~ selection process. When this 

rAtio is less than 80 percent, the Uniform Guidelines state that this will 

generally he regarded as evidenr.e of adverse impact. The Guidelines also 

point out that Where the ratio is less than 80 percent, it may not constitute 

adverse i.mpac·t where: (1) the differences are based on small numbers and are 

not natistically significant; or (2) where special recruiting or other progr11111s 

cause the pool of minority or female candidates to be aty!)ical of the normal · 

group of applicants. Sin"e these 1978 applicants include a. "walk-in" group 

resulting from· specia~· recruiting efforts fo.r minorities and women, the adverse 

impac:·t ;ana.lysis was done for the total group (including. 664 "Walk-in" applicants~ 

and th~ "Scheduled Crou.p" which would be representative of the normal pool c·f 

&pl>licants. TABLE 107 

~ 

... 

-...... 

u..-.u 

...... 

..... 

....-r. w. 

·-

-

""'" ......... 

..... 

...... 

....... 1 .... 

A•h•a-rae lapact Analyda 

P:taaent Ol'M Teat B..attery 

---Total Croup---- Adve·rsa 

'total 1918 ..... l•p•ct 

Al'C Ap£Uccu !!!!!! a.dg 

4191 Sl% 

~ ... .,. 

4067 ... "' ... (:591) 

u.o1 ... (><1%} 

" ... ... __!! 511 ... $Ill 

TABLE 108 

Mvat"ae X.pac.t Aaalyet.a 

... &K 1 .. t ku..ry (W&z) 

___ ..,.,'f-ocal Croup--·-- . Advlt"H 

Total l91t Z Who blpac-t. 

~tUc:uh P!!ae4. ~ 

U91 ... .... 

1M .,. (77%) 

.,.., ... ))i 41% (62'S.) 

U07 14l (21%) 

51 611 ... .. ,,. ... mr 

129 

~----Scheduled CrC114'----------

Scta.dul-.:1 Advc"r•a 

1978 % Who l•p•.:·t 

I.TC Afpl1UA~I ~ t.;Uo 

,., ... ~ ... ... 317S 

111 '" ,... (60:) 

1116 llll (:9%f 

.. ... . .. mo •• SO% (19%) 

---Scl\ltd.W.ad Crc-:·p---

Sc:hftule4 M-r" 

1978 t Who t-p.¥:t 

.UC AnUuttta Paaaad a.uo 

3135 m 

~ ... (l'lt) 

l7l5 •n 21i 411 (61t) 

Hl6 141 (lit) 

so ... ... ,. . 

l3iili 

;,. (79%) 

·--··-- -------------~---· 

I 

I 

' \ 

' 

< 

1 

• 

\ 
The d:sta on the preceeding two· tables shoW that includi~g the "walk-in" 

applicant group has relatively little effect on the raties derived. Generally, 

there is only a difference of one p~rcent Oetween the Total Group and the 

Scheduled Group except for Asians and here the nucber is quite small. 

The present OPM AlC Test has an adverse impact on both the Hispanic and 

Black GJ:"oups. In the light ot past FAA ~xperience in hirino minorities from 

competitive OPM registers, this result is not UI)eX:pected but Table 107 pt"(lVidt·.s 

a quanta-tive measure of the degre~t of advers"': impact_ for these t'WQ groups. 

Tahl~ ., 0~ show~ th.a t the n-ew ATC tes·t also has sdme adverse impact on women 

applicants in addition to the Hisp~ic and 'Black Groups and that the degree 

of adverse tmpact on the Black Group is somewhat higher. 

For .American Indians, the ~ddit.ion of one or two more applicants to the "pass 

gr~p" would. result. in a selection ratio of 80 percent or more on ei~her the 

OPM or ATC_ test~ Consequently, evidence of adverse impact based on this S3J11ple 

, is not clear. 

The analysis was also completed for those who passed the OPM and Weighted ATC 

test to determine if there 'Was adver.se impact in each of three score range 

groups. Candidates who pass the test ba_ttery are ranked on competitive OPX 

registers bas·ed on their total score and those with the highest scores seleCted 

first. This ••Sa,rne"d. Ratin·g", which includes additional points. for veterans 

preference and Otcr credit.-· was used for .the a.n.aly~;is.. The re-sults are provided 

in Tables I 09 (OPH Test) and 110 (\,'eighted ATC test}. 

TABU 109 -------~-· 

ADVtrlSE Ufl>ACI" A.'iAJ..YStS 

Pk£SE!ft Ol'tl TESl Un'El('f 

WITK _V!:I"DlAlCi. PUFI:'lbtlMCE AND £Xl'EUE.>te£ (:.:t£1)-t'f 

.... or·: .... T0-t•l 1978 E.rne4 R.&t1ns EunH bc£.na E.raed ltilt1n1 ----70 - 7~---- ---so - 119-------- --'--90 ... --~--- AIC App a~•"u 

Inlet r. • ...s Altvt:u.e Mver•e Advene OPH Te•c ! !!!!! I .. p.u ! !lli£ l•p•tt ~ !!lli I•p••a ... 2236 711 "' ,., 

-· .2!! 36\ .. 722 "" l!! '" No n2 ... .. !ll 191 Tu ,., 1076 , ... 073 

Wlitu- '"' an m ,,. "' llt•p-.tc lZI SO "' 311 

..... >n .. 47 l7: •• " "' ' , .. "' "' •• 77 "' ' " "' y., ...... " 12 '" .. • m ... ,..r ..... _!!. " 43: •• ....!.% 321 .. -.!1 ... •• --! 201 

lou .... 1-011' ... 

130 

'-----~~­ ~ .. ~ . 

. :;_ 

\ 

•~' 

' 

' 

,, 

.. ' 

' '., 
.,...,..-; 

 . 

•rL• ,.-~ -

.. ~-,~~1'·:-t .. :,.•'t 

--. to-(al !978 

-4.tC Appltc ... IHI 

'$ .. _ .. Who Pu~4 

... 

- - ;:: ...., Ate l_ .. t 

41btt.• 2691 

H1.,~ 136 Jl.C.. 191 

. Mt. 3S 

...... '""!~-- --!! 

"'" 

TABLE 110 

APVOU tKPACf ANI.LYSIS 

N~ I..TC !ES-T ~Al"t!JtY I'll'~) 

WITll Y£r~ rHFDEKt': ,um txPEllDCE CAE:IIT 

ATC 

kned Lntna ~nHid' '" K.t11l& --.--t-o - ],._. _____ ---so- a9--

Adv•n• Mve~sc 

! l.uto 1•p•ct ! IUlo ~ 

"' "" to> 

£1 "' 61% .. ..ll! ,., .. 1416 1162 

U7l "" , .. , "' 16 '" •• " 29t. '" "' ... •• •• m , .. 16 "" .. " 311 .•. 

_J.! , .. .. _! "" f 

14ft lUI 

ATC 

!..~~tin&- ----90 + -----

Mvu·•• 

.!. &at.lo Iar.p.ac ~ 

492 Zl! 

..1i " '" 

"' 

ao 

II 11% "" 

•• " yi,, 

• "" .. ~ Ill f 

"' 

'While ~lle:"e is no evid.ence of adverse impact in the score ranges of 71)-89 for 

wceeu.. -!ladl· the OPM test: and the At:C test have advers~- impact in the score 

TaDge of 9G or more. As identified earlier (Figure 8), a significa.nt factor 

iS -the esb'a c~redit: granted for veterans preference and, to a les,se-r extent, 

credit fot; aviation-related knowledge as measured. by the Occupational Knowledge 

test (OKT). It is also evident that the Weighted ATC test results in a greater 

advers-e -:iBip:ac·t on women in- the 90 and above, score range than the present 0~ 

test. This is pr-'iJDarily a result of the fact that raw.- test scores for all. 

applicants are lower vn the. AIC test than the OPM test-. However. rela"tive to 

men; vom .. score lowe~ on the Weighted ATC test. As shown in Tables S9 and 

14., women >CO:tpr-is~ 27 percent of the group .tdth ra"'• scores of 90 or more 

on the 01!1. test but only 12 percent of this group on the Weighted ATC test. 

.k caap~ racial groups on the two tests, Tablesl09 and 110 show a "?" in 

S¢Ge·of ~score ranges. In these cases, the number of applicants is ~all 

and the .6iition or subtract~on of one or two ayp1icants in the score range 

woul<> ..-It in either meet1.ng or not meeting . the "80 percent rule of thumb". 

Ccnaeque.tly, evidence of adverse impact is n~t clear based on this s~ple of 

1978 ATC applicants. 

It: is elt!« that there is all adverse impact on the Black Group em both the OPM 

ad the ,.v& test in the score range of 90 or more. In the score range of 80-89 

on the~hted ATC test, and possibly on the OPM test, there is also an adverse 

impact. .taain, this is primarily a reault of the fact that as a group, Blacks 

,.vho pasJII!II tha test, score lower than ti:\e majority group. 

Ill the - of the !Uspanic Group, there {s 

rllllp of· to or more on the Ol'M and ATC test 

the OPK uat. 

131 

-. 

possibly adverse impact in the score 

but not in the range of SQ-89 for 

' ,1 ( 

. _________ , _____ _ .. .. .-· '. 

,--' 

' ~~ t I , 

' 
In terms of competitive employment opportunit_ies, baSed on the Present OP!i.f. 

ATC test, between ·so and 85 ?ercent of th~ appointments are g~neral~y made 

from amcm._g applicants with Earned Ratings of '10 -or m·ore. Consequently. 

elllploymeo.t prospects cf &pp1i··ants with scores in the 80~89 ra.'1ge woulq be 

ch.aracter-iz·ed as "poo~" to. ufair". Based on the Weighted ATC tP.st, h,owever, 

about 40 percent of the applicants hired would· hav" scores of SO or more. 

Conseq:uen:tlY, employment p:-ospects of applica.nts wlith scores in the 80 .... 89 

range on the ATC test would be 11 tair11 to. "-good•·• (Tablv !02). 

llh«!n the scor·e range of 80 or more Ot). the \oo'"eighted ATC test was examined, 

there is an adverse impact for the wome:1 and Black Groups. For the. Hispan.ic 

and American Indian·Groups the numb~rs a~e small and the add!ticn of one 

or tvo more applicants in the· score range would me-et the "80 pa.rcent" 

criterion. Th" results are provided in Ta!:>l£ Ill. 

{ ,-i:>' TABLE 111 

tAY\'ERSE l}{PACTA!i:ALYSIS 

NEll ATC TEST BATTERY - SEX A.'W RACE 

total lf1-l 4ft Al'C qpUcaau hr11~ lat.tnc 

Who luHd f2 or ~nt 

._ ATC Ten ! l.&d"o 

... ,,., l)tf .. , 

- ....!!! ...!!! , .. lilt .... -.. , ... . .., m 

lhpaaic ,,. "' llaeio; 190 '"" ... ~ .. " 19 54-: "'Alh-riciU:r.-"ln41c -21. _u "' 

"'" . .., 

... 

•• 

This analysis shows the .Weighted ATC t.es.t has an adverse impact on women and 

some minority groups with respect to both initial eligibility (in passing the 

test) and by using· the total scores, including extra credit, for ranking successful candidate5. In· this situation, the Uniform Guidelines require evidence of 

the validity of test, the basis for establishing the cutoff Pass.seore and the 

'- se of scores ·for rank crde.ring of candidates .. 

Test Validity, Pa!~;'_nore ..m<J Candidate Ranking. The validity of the new ATC 

test battery as well· ~ individual test components of the battery has been 

demonstrated in relatk·n .to a number of ATC .,erformance criterion measures. 

In sUIIIIItary: -~·-

• 1977 ATC Study. For a &8111Ple of approximately 2,100 full 

performance, developmental and newly hired ATC specialists, 

MCAT, Directional Headinga, Occupational Knowledge (Otcr) and 

the I'd·~~: 'i<"'rerienee Questionnaire (PEQ) provided significant 

' 

~-

t .f : . ..,--"" 

/· ··- " ' \ /. /; ... --~"1:_ '/ 1\( /\. 

/ .·· ''· ' 

! . ~· 

>.-·-

.... 

l\ 

I ;\ 

. .., 

. ' . 

~I 

.)'r\,. 
correlations (p ~ .01) with the Aggregate ATC Suceess criterion 

for each of four ATC options (FSS, \erR, lFR aod ARTCC) and for 

all options· cm::bined (see Table 5). In addition, mean test 

scot'es for each of the four ATC· options vert' different at statistically significant levels of confidence (p £: .01) or tp £:. .05) 

Vith the FSS option. average score lowest and the ARTCC s.co.re 

highest (see Ta·ble 6). · 

• 1976-1"978 ATC Studv. For a sample of 1,327 ATC trainees, experimental forms of ~CAl together with Abstract Reasoning (OPll 157) 

and Arithmecic Reasoning (OPll 24) provided a multiple correlation 

(corrected for range restriction). of .54- with the ATC labor_atrry 

p~oblen average score criterion (see Table 28). Cross validation. 

using the weighted test scores derived from the multiple regression 

provided a multiple correlation value (R) of • 5381 for one sample 

and • 52~l for the other (see table 32). llhen test scores on the 

Occupational Knowledge Test (OKT) were COillbined with the experimental test battery (liCAT, OPll 157 and OPll 24), an es·ti:nated 

multiple correlation val.ue of . 60 was obtained with the increase 

significant at the .01 level of confidence. 

• 1978 ATC Studv, For a new sample' of 585 AIC trainees, parallel 

fo=s of MCAt together with the present OPM tests 157 and 24 provided a multiple correlaticn value (R) of .• 42 for tire tor<1l sample. 

The R values by sex and rae., ranged from .37 to .69 (see Table 95). 

The criteria used. was the p(.i.SS or fail/withdraw status of the 

trainee at the completion of AIC Laboratory train~ng. Validity 

coefficients (r values)· used in the multiple regression were not 

corrected for ran.ge restriCtton (see Table q~ ) • . When test scores 

on parallel f,~ ""lS of OKT were combined with the weighted test 

battery, the multiple correl~tion value for the t~:al sample was 

.48 vith R -values by sex and ra~e ranging from .44 ~o . i6 (see 

Table 96. The increase in the variance (R-') accoooted for by the 

weighted test battery with OKT was significant at .the . 01 level of 

confidence for the total sample ·and for each race group·. For 

women the increase was significant at the ,05 level (see Table97 ). 

The utility of the new ArC· test battery (as compared to the present OPM 

battery) was examined with respect to appointment eligibility (pass scores) 

and ranking of successful candidates based on total scor~s ir.cluding credit 

for veterans preference and ATC-related knowledge, 

To do this, the information available for the 197·8 ATC trainee sample was 

related to th* 1978 ATC applicant group. A co=mon test battery, consisting 

of parallel forms of MCAT, OPY. 157 and OPM 24 was administered to both groups·. 

For the OKI, a 6o-item test (101-C) was administered to the 1978 ATC applicant 

group and parallel forms of OKT 102 (80 item) administered to most of the 

1978 ATC train.;.e sample. The common test battery was weighted' 

4 x MCAT Total Right Scores 

2 x OPll· 157 Total Right Scores 

1 x OPII 2·4 Total Right Scores 

133 

\. 
/ 

This ""'~ighting differs sOQewhat from the weight's derived for the test battery 

which included two paull~l forms of MCAT for the 1978 ATC trainee .group 

(see page 114). However, the correlation coefficient between the. two· 

weighted ATC.tests was .93 (s.,~ TaHe 98}. 

In the analys-is of the new ATC test battery. the passing score was set· at 

the mean raw score for the 1978 ATC applica,lt group· (222. 27). This was then 

equated to a tr:an-smuted pa._ssing scOre of 70 (see Ta-ble 77) • With this b'a-se 1 

equivalent scores on the same weighted ATC test ba-ttery were computJ!d for 

585 of the 1973 ATC trainees. A total of 48 AtC trainees (8.2 percent) failed 

to achieve a score of 70 on the new ATC test. Of these- 48 trainees • 79 peicent 

(38) failed the Ate Laboratory training or withdrew (see TablelOO). So01e 39 

of the 48. tt·aine.es had scores in the 65•69 range and 9 scored belo.w 65 (see 

Table 9g). The score range of 7G-74 on •he ATC test included 102 of the 585 

trainees (17 percend. Of thu group, 51 percent ( ~2) failed or withdrew 

-(see Table 100). Consequently, use of the mean tes·t score resulted in· a 28 

pe.rcent reduction in th'! fail/withdraw rates between the ATC trainees who 

scored in the 70-74 range in contra$-t to those who scored below 70 ·on the 

ATC test& The· difference between a fail/withdraw rate of 51 percent in the 

70-74 Sc:o.re range and a rate of 79 percent below scores of 70 cle.a1·ly supports 

the "cutoff" score developed in the analysis. 

The use of scores on the ATC test. with and without· e~tra credit for veterans 

preference and ATC-related knowle~g.e. as a basis for ranking successful ATC 

candidates was also· examined. Since extra credl.~ is a seleCtion consid~ration 

only for tho·se candidates who are appointed from competitive OPM registers. 

the analysis included -only those 197'8 ATC trainees who were hired ·competitively 

(Non ... pr·edevelopmental) and who scored 70 or above on the ATC t.est. On this 

basis, 509 of the 585 trainees were included in the analysis. 

For the: ATC test battery Yithout;. extra credit, Table 101 ( th,!'! Nori-predevelopmental ATC Gra·u·p) shows that the fai~/withdraW rate in each score range progressively decreases as the test scores increase. In the 95-lOO rang~ there 

were no failures or vithdr.aw·als- (Only 2 trainees were in this. range). In the 

70-.-74 range. 47.8 percent failed or withdrew·. The largest increase fn the 

fail/withdraw rate is between the 75-79 unge (41. 9 percent) and the so-as 

range (29.3 P"rcent); a difference of 12.6 percent in the fail/withdraw 

\ : 

rate. Table 110 c0111bines the ATC test score ran.ges and provides the failure/ 

wit.hdraval rates ·for this S'llllple of 509 trainees. 

..... 

!el! 

TABLE 112 -----

1111 Ate TU.lRU Al"C ust SO'JUi 

(lfmDin" aru t:UJI)tt) 

.. .,, 

;:~ 

.. • 

100 

m 

134 

\ 

17,1"1 

26.01 

u.n· 

1>7•f 

.' 

/ \. 

,. 

·'· . 

<. 

' 
,- .. · ' ' 

These data clearly shoW" that ranking ATC candidates 1 .based on their test 

scores and sel,ecting those w-ith the higher scores first, will re·sult in 

hiring applicants with the highest NObability of successfully completing 

the ATC training_ progrruo. 

Ranking candi~ate-s based on total u£.\rn·ed Rating" scores which include test 

scores plus extra credit for veterans preference and ATC-related kn~wledge 

based on ·OKT. scores wa-s examined next. From the Sample of 509 AIC r-;:-ainees 

who were hired competitively, the equivalent of the OPM Earned Rating for 

the ATC test could be computed for 486 trainees. Tablti!: II i provides the 

fail/withdraw rates by Earned Rating score ranges and compares these to the 

rates based on the ATC test score range_s without extra credit~ 

kou ~1• 

..,. 

..... 

1~79 

tottil 

1918 ATC !IAlmJ 

CN-Pred...,t!1o,.etltel-Cc:.,eUt1v., Ml-n11l 

___ __._._,TC T••c:----- -----A!C THe---·-•----

(VS.thout lieu·• Cudd.t) (\11th Extra Cudit' 

f•il/'llichtuw F•11."\01thd.Uiol 

! fat.l/Vttht-r,w !!!! ~ f•!l/\IJ.thdr- .... " u.z: 191 " 19.1),: 

m .. zt.o: m .. )7 •• : 

E! .!.!2 44.21: ...!9. 1!. !l;:)! -~-

••• 111 ~3.6: ... 163 )).5~ 

It is evident from Table l.l3 th~t ranking candidates for selection consideration 

on the bas.is of the Earned Rating (which includes extra credit) also results 

in selecting those applicants first with the highest probability of succ.eSs·fully 

. completing the ATC training. It is also evident that se-lection based on ~he 

Earned Rating results in a higher fail/withdraw rate in each score range than 

if capdidates were hired solely on Lhe basis· of their test battery scores without 

granting extra credit, 

In deriving the Earned Rating score for this sample of 486 ATC trainees. extra 

credit for veterans preference was included since it ·is granted by law. Extra 

cr.edit based on OKl' test scores was included based on .the positive and signi-

·ficant correlations with ATC ·performance criterion ·used in the various research 

studies •. 

Conse-quently. it was hypothe-5ized t'flat extra credit for being a veteran. 

wh.i.ch has not been· examined for validity with the criterion mea-sures U$ed in 

the research, does not predict (or negatively predicts) success in the ATC 

Laboratory training. In order to examine this, scores for the 509 ATC 

trainees were computed W-hich exciud~:! any veterans. credit and included only 

the weighted ATC test battery score and extra credit for OKT scores_. 

135 

' 

I 

I 

·. \ 

:I "'-. .. 

'·. 

~'"-~_ . 

.. . . ~ ".:. 
Table Ill, provides the score distributions for the group of S09 1978 ATC 

trainee-s before and after the applicat-ion of extra cred.it for air traffic 

control related. kpowledge based on OKI points~ 

TARLF 1 14 

.!. 9_~s_.1rc_ T-''.fi t~_rx,s ---·y:r:r}j}_Qf: Jc,'i'~RJ\_ .£R.EP.'.I 

---~178' Aft Tu.tuu---- -tt78 Ate ha.&.n•••---

(DhuU>\U.lOO. W1~~~~~ 0«:!: CrH1t) (1)tnr1buc.1CN W1:lh· Ol:r Ccad,t) 

! .. ! .. ... " ). 7: ... 139 %1. 3I 

~ .. 2S4 49.9% 8<>-89 "' u.n. 

~0. 7" .ill ~ 7G-79 ill l!..:.!! 

Tt·:-'1 ""' li,X).Ql total ... 1oo.o: 

It is significant to note the addition of OK! poi~~= result~d in moving 

110 of the ATC trainees to a score range of 90 or more;....-an incre·ase of 

about 2Z percent. It also resul.t~ad in reducing the number of trainees 

in the lowes-t. scoring group (7G-79) by 91-from 226 to 125-a reduction of 

about 20 percent. 

Based on the validity data previously established for t'-~ OK!, it would be 

expected that thoSe trainees who moVed to the higher scc·re ranges as a 

result of OK! extra· credit Would have lower fail/withdri!w rates. and that 

those who remained in the lowest score range Would have higher tail/withdraw 

rates. The comparison of the fail/withd·raw rates in the three score ranges 

for the 509 ATC tra:!.nees before and after applying Ot:T extra credit points 

is provided in Table 115. 

TABLE 115 

1978 ATC TRAIKEES FA!l./WITHii'i(,\\~ RATES' EFFECTS OF OKT CREDIT 

-197& lt.TC TraUM .. -- --lt71 4TC traa .. •-:--

(VtthcNC 0C CU41t) (V:I.tll 01! C.-...d:it) 

la:ll/ TIVD ld.l/ T/VO 

! Vs.th4rw. !!>!. ! VJ:th4rMI" !!.!! 

- ,. • 17.11 Ut lt 1].71 

..... "" .. 16.01 245 ,. )1.1% 

,._, & lB!! i4.=.2! .ill .!! 19.:..2! 

Total .. , Ill 1).61 ... Ul )3.61 

Sinc:e the fail/withdraw rate for the to~al group of 509 trainees is a constant (33. 6%), the rates can only vary between the different score ranges 

as shown in Table 115. In.orcler to assess. the effect of OK! extra credit 

on the· total fa.il/Withd~aw rate, one can assume a selection mad~ from amOng 

the group Clf 509 trainees. For example, if one were to hire 300 tra-inees 

136 

·,~ . \ 

' \, / 
i 

from the 509 in rank order {highest score range firs.t) with .an esti.m.ated 

selection ratio of 80 perceqt in each scOre range, the fail/withdraw rate 

for the 306 hired before application of OKT extra credit would be 30 percent. 

The faH/withdraw rate f..,~ the 300 selected after application of 0!-"T extra 

cro:dit would be 25 perceO.t. If o.t.lly 100 of the 509. were selected. the 

fail/withdraw rate for the group ~o.·~·thout. OK.T extra credit would be 24 percent 

in contrast to 14 percent for the gro~..:.p with OKT cred·it. These data support 

the conclusion .that ranki:l.g applicants for selection cansid~ratian base"d 

on test scores and- OKT credit will result in hiring applicants with th~ 

highest probability of successfully completing ATC training. 

TC'ble 116 show.s the fail/withdraw rates for the 509 trainees based on ( 1) the 

ATC test "'lone; (2) the· ATC test plus OKT credit; and. (3) the ATC test plus 

credit for OKT and Veterans Preference points. 

TABLE 116 

1978 ATC TRAINEES FAIL/WIT!IORAl.' RATES - EFFECT OF. C_REDITS 

----ATC f<~u----·-. ---.,--·-ATC Teat·------ ------Ate T~•t·------~--

tl'o btu Cud'i.t) UWa or.r Eat:u Cu<lic Only) (Pllll OICT aU Vet. I'Tcf, CredHI 

5C01"il : Fail FIWD : FeH . , .. • hi1 Fl'oJP 

ll.&a•• ! !. With4r•v !!.\! ! ! Vitb!!rew !!.!.!· ! ! ~ !.!!..!. 

... " ( .., ' U7.2U U9 ( 27t) " (1],71:) ,., ( loll) " (19.81) 

80-89. '"' ( ,.., •• U6.0U ,., ( 41t) .. Ol.IU Ut ( 4·lt.) .. {)1.61) 

71)-79 .ill ( 4-lo:} .!!!2 (4o4 .lU !:.!.! ( 25:1 ~ ll!:..ill ~ i.E.!! 1! (6).)%) 

Toul '" (100:) l7l ()],61) , .. ClOOU l7l 01.61) ... llOO%) h.l (3).51} 

The use of OKT credit resulted i:n movirtg 110 trainees into the 90 an,·\ above 

scor~ range compared to the ATC test alone~· The number of fail/with·i···aws. 

it,creased by 14 for a rate of 13 per.aent for this group. When vetera;o.; 

pr~ference credit is also applied fo~ 486 of the 509 trainees. an addJtional 

S8 l:.~.~tinees moved to the 9'0 and above score range and th.e number of fail/ 

withdra,;: ~ncreased by 20; a fail/withdraw rate of 34 percent for this group of 

58 students. From the data_presented in Table 116, granting extra c.re.dit for 

veterans preference increased the fail/withdrawal rates in the higher score. 

ranges and therefore has a negative relationship to potential success in 

ATC trainin&. · 

Test Fairne.ss. When empiric·al evidence of validity for a selection procedure such as presented for the new ATC test has an adverse impact on a race. 

sex or. ethnic gr.vup, the Uniform Guidelines. require examination of test 

fairness where technically feasible. 

In establishing the Uni£o"t1D Guidelines on Em:ploye·e Selection. it was recogniz.ed there is Serious debate on· the question of test fai'l"ness. There are 

several competin& defini·t.ions of teat or selection fairne6s J each of which 

incorporat;~s a diffel;'ent· set of social values •. Reference {"12) ·pro.v.ides. an 

analys-is of three incompatible ethical positions in regard co fair and unbiased use of. testa together with differing st·ai:istical definition-s of "test 

fairnes-~n and their re;lationahip· to specific et-hical positions. These- three 

137 

' . -....... . .. 

.. - -· .' ~ /I '\ / / i '.:. 
\ /,.' 

\ 

ethical definitions. are characterized as: 

• Unqualified Individualism 

o Qualified Individualism 

• The Quota Ethic 

The General Ac.coun-ting Offic~, in examini~ .g Federa.l selection tests and ex.nninations (33) stated with respect to these three definitions: 

"The published literature-on t·est va~idi_ty indicates that most tests 

are either fair to minority groups or slightly biased in their 

favor by the second definition, (Qoalified Individualism) which 

according to OPM, is the only concept of fairness consistent with 

merit syst.em principles. The lit~rature also indicates that by the 

first definition, (Unqualified Individualism) tests are slightly 

biased against minority groups, and if one subscribes to the la&t 

definition {Quota- Ethic) tests have alwaYs been biased against 

minorities." 

The "Qualified Incliviclualism" definition of fairness adopted by the Office of 

Personnel Management (OPM) holds that tests are biased when those· with equal 

chances of success_ on the j-o_b have unequal chances .of b.eing select.ed for _the 

job. This definition relies solely on valid meast~res of aptitud'e, a.chievement 

and experieti.ce; maximizes productivity .:i.t:ld pr.-:rvides equal opportunity consistent 

with merit principles. The statisti~al mode~ upon which the Uniform Guidelines 

~ppear cO be basecl, was· stated by Cleary (33) ~ Basically, -chis model requires 

the regre·ssion line which Predicts the criterion (e.g. performance) from "scores" 

be the same. for ~11 cultural groups and, absent thi:s, separate· regression lines 

should be used as a basis for selection decisions. Statistical tests developed 

by ~ulliksen and Wilkes (34) provide a prOcedure which m.eet.s the Cl-.~ary ~odel. 

Basic.al~y. three· independ-ent and sequential tests are involved: 

(1) population vari.ances (standard error of esdmates) are equal 

(non-significant differences), then, 

(2) slopes of the population regression lines are .. .,qual (nonsignificant differences), then 

(3) interceptS of the population regression lines are equal 

(non-significant differences). 

If ea~h of the thre·e statistical tests is met. the regression lines are the same 

lllld tl\e "fairness" definitioe of "Qualified Incl.:lvidualism" is satisfied in that 

"Scores" used for selection. predict· the criterion equally for the various groups 

considered. On the other hand., if a statistically significant difference is 

foun<l at azry point in the sequential analysis, the test is considered to treat 

the groups unequal~ 

138 

-~ --·----~ ----··-: __ __,___ -

.·. 

/ 

I 

•. 

/ 

.,/ ... 
, •. ; ' .-· • - #. '""f ' r.· .. •· '' ., 

In examtntng. the new ATC test battery i;t accordance with tlle-· "Qualified 

Individual.ism'-' model of ~.iirness·, test score·s and. criterion data obtained 

for the ••mple of 95'3 ATC trairieu attending the FAA Academy during 

June 1978 through December 1978 were used in the 3nalysis. Th~ nu~ber of 

Hispanics (~3 total--.·7 with OPM test data) was too small for, a~lysis. 

There were no American In~lians 'in thi$ s.Jmple. The.re was no evidence ·of 

adverse. impac.t on selection Of Asians with the new ATC test bat.t·ery. Con·· 

sequently. the analysis for tes·t fairness was. com"pleted with re$pect to 

men and women Or for White and Black groups. 

Sample sizes varied depe.nding on the availability of data on 

tests (OPM 157 and 24) included in the new ATC test battery. 

criterion data wei:'e available fot: the following ATC trai"lees 

TABLE 117 

DISTRIBUTIONS BY SELECTED SAMPLES 

... "-· wtli.t-e 

(11-800·) .!!:.!.ill. (N-8)9) 

._ .. ._ .. S•pl•· 

!!!1. -·- .-.!!...... ....!!.-

to:.\Y l 790 U7 ... """' "'"'" , .. 1)7 ... 0'" U7 ,., •• .... ..... ,., .. "' 

the two OPM 

Test and 

(Table 117). 

ll.cll; 

(N•81) 

s-pt• 

-·-" 

" ,. 

,. 

In conducting the analysis of tes.t fairness •. the tests were weighted in 

accordanCe with the· values derived for the- 1978 ATC· trainees. The-se 

weights. are: 

2 x HCAT l (Total Right Scores)"\ • HCAT Total x 2 

2 x HCAT 2 (Total Rig~t Scores)) 

1 x OPM 157 (Tot·al Right Scores) 

1 x OPM 24 (Total Right Scores) 

The. test for significance for all anal.ysea of fairness was the .05 level: of 

confidence .. 

Since almost all of the 1978 ATC trainee eample had test scores available 

on the two parallel forms of HCAT, an analysis of fairness of the MCAT by 

iuelf was completed. Table 118 shows the results for the first form of 

the test administered (MCAT 1) and the total score for both forms (MCAT 1 

~Ius MCAT 2) by race and sex. The test means and standard deviations are 

based_ on raw test scOres. Since MCAT was the-only test used in this analysis. 

the raw scores were not weighted. The criterion value is based on a mean of 

"011 with a standard deviation of "1'1 and refl~ct·s., in effect. the· percent 

of trainees who passed the l.aboratory training; the validity .coefficient (r) 

is the. co"(relation val4e of- the ':est with the, p.ass or fai-1/withdt:aw criterion. 

/ . r I /.· . i 

139 

' 1

--- .'·'· ·--- ... - . -::--~~.: -~ .. -~. 

' ' 

·--,; 

<= 

>-:· 

.:;.} 

•.:,;;, 

·;~ 

'{' 

. {· 

J 

> >...:'' 

/ ·' /:. 

f . 

·~ ·., 

·~ 

•1\ 
. /· , ~~':":/' . 

"\. 

L.-: >· ' /'\ 

;:< -r•: 

' . t· 

1

: .. ;;.. -·~· 

,..-t.;\' 

·~-!· . ,. 

.(~ 

-. 

:1 ···; / 

;/ 

r 

t ,. 

V/-

I 

' 

\.' ) I 

.. ·. ! 

~>i~· 

' ' i

.· _,, I ( ,i. lr· -~ 

c • I 

' ' 

· .......... 1'~-·\.·· ·; .-, .•. :}.--

ID! 

NCAT l 

KCAT TOT 

till 

KCAT J 

HCAT OOT 

.... 

.. 

<> 

' .. 

TJIPU: 118 

ANALYSIS OF TEST FAlRNESS 

.............................. ~ ... -HEK------------------------ -------·------------woKEN-------------------·- .............. fEST•----- ---cRITERION--- ~ .............. y£ST------ ·--CJUT&IUON·--

.!! !!!M! !!'. !!!M! !!'. ,!; !! !!!M! !!'. !!!M! !!'. ,!; 

1'10 j1, ~0 6. 90 .6J7 .483 ·" 1)7 )4. 20 1.'10 .571 .1196 .370 

790 80.B ll. 27 .632 .48) .40 137 74. 91 13.20 .sn ~496 ·" .................. ~ .................. ~ .. WKtT£---·-·-----------··-- ·--··-·----------·--·aLACK-------·------------- .. ............ TEST·•·---· ·•·•CRITERION--·- .............. fEST----•-•• -•-•CIITERlDM-·-

.!! """' §.Q !!!M! !!'. ,!; !! !!!M! !!'. -· !!'. ,!; 

. &28 )7. 61 6. 71 .647 .1.78 • 29 79 29. 82 7. Y2 ,]54 .481 8~8 ·" 80,81 10.76 ,6Q7 .478 ·" " 66. 3 5 lJ. ll .354 .481 .51 

~ ." . /:..'j/ ) .. , }<,..,_ r----~-- I I ~o\' ; 

-------TEST OF-------

sro 

W<OR 

NS 

NS 

SlGNlflCJ.HC! 

lNTERSWP!i S£!_ 

"' 

NS 

•• .......... _fEST or-----

Sl..:NIFICANC£ 

STD ltn'ER• 

~l SWPt ill! 

HS 

"' HS 

NS 

SLO 

HS 
In terms of sample sizes- used :ln this analysis,. the 137 wom..:..a comprise 

14.8 percent of the combined sample of men and women (927) compared to 

15.7 percent of the 7,894 AIC trainees who were w~en hir~d from January 

1976 through October 1980. The 79 Black trainees represent 8.7 percent 

of the combine<! sample of the 'llhi e and Slack group (907) compare~ to 

8.3 percent of the 7488 White· and Black ATC trainees hired during this 

same period (see Table 123), 

For- m-en and women,. the test of fairness shows no significant diffet'ences 

(NS). Fo< .the White and Black groups, the .<lifference in the intercept of 

the regression line, was statistically significant (SlG) for the first 

MCAT t~st administered but differences in the standard error and. the slope 

of the regression line were not sig.tificant (NS). When the total scores 

for- both MCAT forms ad.rQ;i.niste'red were analyz·~d, there were no sign"ificant 

difference's between the two groups. This ar.alysis indicates that the total 

MCAI scores. predict the ATC laboratory pass or fail/withdraw criterion 

equally for men- and women and for the Black and- White groups_ .. 

The next step was to develop differ2nt t~st batterJ combinations wit~ the 

MCAI test scores and OPM iS7, OPM 24, ana OK'l' test scor::..s and .:'.nalyze 

these various .test batteries for significant differenc'is l)et-ween tren and 

women and. the White and;Black groups. The intercorreJ.at:lons betveen the 

variOus test com.b.inatio-ns U$ed and their cor-relation with the pass or fail/ 

withdraw ~riteriun ~re provided for each group in· T;lble 93. SL'lce a num.ber 

of trainees did not have test scores on OPM 157 (ao<l OPM 24), the sample 

sizes were reduced as shown in Tctble 119. The cor·:elations. sh(f.o-n are based 

on weighted test scores where appropriate using r2M test battery scores (RS) 

as well as transmuted scores (TS). the correlati•.)ns which combine a test 

battery with OKT use both the raw OKT score (RS) as well as OKT extra points 

(PTS) derived from che raw OKT score (i.e., 0, 3, 5, 10, 15 points). 

The descripCive statistics for the samples used in the fairness analysis a·re 

also provided in comparison Lo t.be total sam_ple fot' each group as was. given 

in Td>le 84. 

I41 

~ ........ '"" . .,__ 

~·--·------------ ·-~ -·- - . .-:_- ,....- -- -· .. 

__;.-.;/ .. \ ~ ........... - /" \. ., . . \ 

\, I ~ ''-lr '','' / 

' 
Tl\BIL 119 

'l'!iST FAIR'l!'SS ANALYSIS OF TrnT EATI'ERY OPTictS 

~------------------~~------------------- -----------------W.QKEK--------.,..;. _____ 

(t.tlle S4 (T.-ble 84 

.!! ~ so .!' ~ so ! ~ ~ ! ~ ~ 

KCAT 109 )7.6 ••• 800 37.4 ••• •• ll.4 ..1 141 34.1 1.8 

KCAT 2 , .. 41.) '·' 800 io2.9 1.8 .. 40.7 .. ' 141 40.7 ••• HCAT TOt , .. 80.9 11.0 800 •• 74.1 11.4 141 

OPt! 1S7 , .. "'·' 6.2 Sl5 l$.1 6.2. •• )9.9 • •• ., 39.8 6.0 

OPH 2:. 109 46.7 .. , '" 46.6 .. , •• 47.3 '. 1 67 U.2 7.1 

OltT (IS) ... 64.1 1S.8 800 64.9 ts.s .. S6. ~ 14.4 141 s 7.7 lS.l 

---------------~~------~------------------MEM------------------------~----------w-----~-----·-----• 

(N•SO•)) 

(l) (l) (}) (4) (>) (6). (1) <•l t•l (10) (11) (12) (I)) 

(tS) 

(TS) ICAT ... u 

MCAT KCAT HCAT tOT. HCAT to-r. 

IICAT t«.:At rur. rur. TOT. "' ·ror. 1S7 

rur. rur. "' "' ,,, ,. lS7 24 P.ts• 

MCAT ••• OP!t OICr lS·1 1>7 ""' OICT 24 OKT " OKT f.:IH 

W:. ill ~ (IS). .l!ll .!ill .llli {lli1 .llli .llli .!ill .1tlll.. .2!L 

(1) HCA.T TOT. .24 .3) .03 .91 . 97 .OJ .82 .os ••• .9> .82 ·'" 

('2). OPK 157 .09 -.16 .48 .48 .32 .):! ... ;)2 . .. • 30 ·" 

(l) 01'M 24 -.2) .32 .32 .14 .15 ·" ·" .Sl .n .12 

(4) OKT (ItS) -.OZ -.0-2 .14 ·" -.07 .46 -.07 ·" .21 

(~} HCA.t TOT + U1"(RS) 1.00 .8) .. , .91 ... .91 .82 • 38 

(6) HCAT Tot + U7(ts.) .83 .83 ·.97 ... ·" .82 .38 

(1) MCAt Tot + lS.7 + Ola(RS) ••• • 78 .98 .18 .92 .44 

(I) HCAT tot·+ I ,7(TS) + O~(PTS) .78 • 92 .18 ... .44 

(9) MCAT. TOT + 1~7 + 2:4(1tS) .81 1.00 .82 .31 

('lO) ttet.t TOt + 151 + 24 • OKT{II.S) .ss ••• .44 

(11) HCA~ TOT+ lS7 + l4(TS) .82 .37 

(ll) JtQT TOT + 1)7 + 24o(iS) + OICT(PTS) ·" 

(ll) Crite~&on (La~ Paaa or F/UD) . 

142 

-·-·----------.. 

• ; //. J 

-~·~.-· . --~ _:.,·· -: J f 

. -: .. , ' I . ,, 

--·· 

I 
l 

TA!l!J'_ll9 (Cbutinu<-dl 

(I) (Jl (4) 

OPH 

ill 

(I) MeAT lOT. ·" . o.. .17 

en oPM l)l 

U) OPK 24 

t•l- (IS). 

(i) ttea.t 101' + U7(a$) 

~) NC.\t tot +. UlUS) 

(1) MCAT TOt+ 1:07 • OICT(a5) 

.12 .Ob 

(&)· MCAT Tot + lU(TS) + OU(nS) 

(9) ~T f9T + U7 + 24(R5) 

(lO) tiCAl TOT + U 7 + Z4 + Hf.{IS) 

(11) ftCAT TOT+ 1\1 + Z4(TS) 

(12) MCAT TOT..- l)J + 24(TS) + O*HPTS) 

(1}) C1'1tert·on (L.a-b Pau. or FIWO.) 

(>) 

o«:AT 

NT. 

illl. "' 

... 

••• 

.11 

------.-- ----..WHltt· ... ----~----------

(Tab'Je 84 

! ~ ~ .!! ~ !,11-

HC.U 1 

IIC.tt 2 

Sl9 17.7 

Sl9 43.6 

••• 

'·' 

MCAT TOt SH 11.2 10,6 

OPM ISl Sl9 39.1 - 6.2. 

01'11 ,. )}9 46.8 ••• 

ocr (IS) -Sl9 6).1 lS.t 

8)'1' )7.6 

8)9- 4). 2 

8)9 

S4~ 46.9 

••• 

'·' 

6.2 

6.i 

1)9 63.9 IS.t 

t«:A1' 

lOT. 

'" !!ll 

• •• 

. r.z 

.... 

.11 

1.00 

'·. " 

ti('A.f 

TOT. 

"' 

• •• 

. ,. 

-.06 

.>6 

• 91 

.91 

143 

'"' 

(1~) 

~.'I.C 

TtH. 

o.:r "' 

~ 

... 

.01 

·" 

.u 

. '' 

"' 

tll:Ar 

tO-t. 

'" ,, 

i~ 

·" 

·' l 

·" 

••• 

••• 

tiOl 

i'tCA.t 

l~lT. 

"' 

OKT "

illl. 

••• 

·" 

·" 

.91 

.91 

... 

... 

.91 

( t I) 

"CA.T 

tor. 

'" 

.illl " 

·" 

.41 

.29 

.ll 

... 

••• 

. .. 

••• 

1.00 

.91 

(I •• , 

ITS I 

lOUT 

tur. 

"' 24 

Ol."T 

.11m. 

.20 

.11 

.. , 

.91 

_., 

.98 

_., 

_., 

·" 

--~--------------ILACK---------------- (Table 84 

,!!~SO_!~~ 

ll 29.7 

)8 )6.1 

7.1 

7.S 

,. 6~.8 14.0 

••• 

38 44.7 ••• 

81 29.1 7.9 

81. )6.6 ••• 

II 

39 ]~.) 6.1 

)9 44.2 '·' 

11 66.2" ll.l 

(I)) 

.Jl 

.Oti 

.19 

.l9 

·" 

.41 

·" 

.42 

·-.a... .... _·\~ --.; . :-

' ;-47 

\ :¥j_ .I ,, 

'r-- ' t: .""' • I t \ - ... ' • "'; .... -

. . 

' 

\ 

' ' . -· 

-·\·,, 

··""\ .. 
\I) HCAT TOT. 

(2) OPH Ul 

(J) OPM 24 

(4) m;;T (ItS) 

tl) 

(St MCAT TOT + IHUtS) 

(flo) MCAt TQ.'t + U7(lS) 

. ! l 

(J) MeAT TOT • n1 + Ol;t(RS) 

lll 

Oi'H t'~>.i 

...1!! .i!2l 

• .;~ .Oi" 

• J.J •.llo 

-,24 

' (_ .. I'ICAT T01 ., U1(TS) • OKitPrS.) 

(9) MeAt TOf. + U7 • 24(AS} 

(10) tiCAl' TOT • I Sol • 24 + Gn(IS-) 

(U) MCAT TGT + 1S7 + 24(l5) 

(12) !'CAT TOT + lSI • 2.:.-(TS) + OKT(PTSJ 

Ull C'C'itulon (Wb r-• or F/\10) 

'" 

tti,_"Af 

tuf·. 

.tru '"

• 97 

••• 

. ,. 

.03 

TAB!F l!9 (Cbnt '"'-",d 

"'" Mt.

.iill "' 

·" 

... 

.18 

.J] 

1.00 

lOT 

tUT. 

OKT "' 

1!'21 

.8) 

• )I 

.09 

.8) 

.n 

, .. 

(T5l 

'""' TVT, 

U.:'f "' 

l!.!51 

.10 

... 

.az 

.az 

.94 

HCAT 

TOT. 

"' 

.!!ll " 

.9.0 

·'' 

.50 

-.0) 

·" 

• 97 

·" 

.11 

llU) 

lOCAl 

TvT. 

'" ,, 

.!!ll "" 

·" 

.JZ 

.31 

.50 

••• 

... 

... 

.• 92 

.I> 

(ll} 

HO,t 

TOT. 

1',1 

ill.!. " 

.94 

. 50 

-.0) 

.91 

·" 

.72 

.71 

1.00 

.05 

(I:') 

('I''S) 

"""' tor. 

"' l4 

U<T 

..lllll . 

.&2 

. J l 

.10 

·" 

.&1 

.82 

·" 

••• 

.II 

.94 

••• 

.}) 

.u 

,01 

.2J 

.:14 

.41 

·"' 

.ll 

.41 

.)j 

.40 

--------------------a------~------·BLACK---------------·---------------------------~-------~---~---

(I) (}) (4) 

... ..... 0<1" 

.ill _!i .IDl 

(I) IICAT TOT. -.28 .17 .27 

U1 OfK lSI 

()) .... 24 

-.n -.u 

. 14) oo:r (IS) 

(,) HCAT TOT + 1l7(lS) 

(-i) JICA1' TOT + U7(TS) 

(1) MCAT ·TOT'+ 1)7 + Ollt(r,s.) 

(&) HCAf TOT • IS7(TS) + OU(nS) 

(9) IICAT tor+ 1S1 + . .Z'lo(~l 

UO>- MCJ..T· tOt + ts 1 ... 24 + _o1Cl' (R!l.} 

(Jl) HCAT TOT + 151 + 24(t5) 

4"11) M9f TOT + U1 • l4(TS) • OICT(n-s) 

·(U) CduriOA (uta- hu o.r f'IWOt. 

. \ 

'J-

-.09 

(~•)8) 

IICAT 

TUT. 

.IDl "' 

.98 

(6) 

MI:AT 

TOT. 

ill.!. "' 

... ..... 06. ....06 

' 

·" 

.25 

-~- . 

f ... 

• ·\. •"_J I. . ...... 

.IS 

.2> 

1.00 

(7) 

~t<:AT 

TPT. 

O<T '" 

jlli 

.90 

(8) 

(b) 

Mt;AC 

TUT. 

1>7 

•~r 

<rfs) 

.92 

-.to .... 1a 

.08 ••• 

.62 ••• 

.92 

.92 

.9S 

144 

'" 

ftCAl' 

1'0l'. 

157 

l4 

.IDl 

.... 

-.09 

.)) 

·" 

·" 

.91 

.88 

.89 

UO) 

Tot. '""' 

~~-1 

,. 

Ol(t 

.lli.t 

.91 

-.12 

.27 

••• 

.n 

.91 

... 

••• 

.92 

(ll) 

K(;A'f 

TOf, 

"' ,. 

ill.!. 

• •• 

-·"' 

.)) 

.22 

- .97 

.97 

••• 

••• 

1.00 

... 

(11) 

ITSJ 

~u.r 

Tur. 

1'~1 

24 

on 

.1tW.. 

... 

-.19 

.2> 

.41 

.90 

.90 

.9) 

·" 

... 

·" 

• 91 

OJ) 

• •• 

.OT 

.24 

... 

... 

.62 

.61 

••• 

.61 

.68 

... 

<; 

.. 

; 

I 

; 

. 

; 

.; 

' • 

I 

"' --_. 

r . .. l --:, ''· ... . I ' -. .-! .. . _~I 
The first test battery analyzed for fairness consisted of MCAT Total and 

OPM 157 raw scores. In this analys.fs. MCAT Total was weighted by "211 and 

OPM; 1_57 weig~ted by "1" in acc.rir-dance with the weights prev,iously derived. 

Then the OKT raw s~ores (weighted 11 1") were combined with the MCAT and 

O?M 157 raw scares and this- cOmbtnation sepa_r~te.ly evaluated. Th~ res_t.~lts 

for eaCh sex a~d ra1=e group are pr·av{ded in 'table J 2r1. It sho.uld be not~d 

that at this point, a "passing" scor-e for the combi-ned MCA'l' cip.d OP!-1 157 

te&t scores was not established. Consequently, the additton of the 0~~ raw 

score was indepen·dent o.f whether or not_ the ATC trainee "pass~d" the MCAI/OPM 

157 test battery and also independent of extra credit ~~ints (0, 3, 5, 10, 15) 

which w~uld be derived from the raw OKT s~ores. 

The combination of MCA> and OPM 157 raw test scores show that this test 

battery predicts the ATC laboratory pass or fail/withdraw criteria eq~ally 

for men and women and for Black and White groups. The same results were 

obtained when MCAT, OPH 157 and OKT raw test scores were analyzed separately. 

Use of raw scores in analyzing the test battery comprised of MCAT, OPM 157 

and OKT does not take into account the proposed use of Ctcr in the selection 

process. Since O!cr is a job specific test, U is not intended for use in 

the initial "pass/fail" determination of appointment eligibility for ATC 

applicants. Its use would be limited to determining if those applicants 

who passed the test battery (in this case MCAT and OPH 157) would also be 

give·n extra cred-it for air traffic control related knowledg_e as measured by 

O!cr. F11rther, the amount of extu credit (0, 3, 5, 10, 15 points) would be 

based ou the ra\7 OKT score with no credit for l-aw scores below 65 and maximum 

credit (15 points} for raw Otcr scores of 80 or above. 

In order to evalua·te the fairness of a test batte-ry in combination with OK'I 

in r~la.t.ion to_ its int.et:lded use, it is necessary to (1) establish a "passing" 

score on the test battery ·£or' an ATC applicaut group, (2) transmute the 

raw test bat·tery score in relation to the paRsing score, and (J) add ·the 

approp'ciate extra credit points based on OK7 t"llW score for those who passed 

the test battery, 

145 

<~·-r- ··'-/, 

\ \ 
<~ 

-r 

' 

~ "• i 

' 

i 

I 

. I 

~ I 

II 

'j I 

' 

'-

t: 

~ 

"' 

UST 

MCAT tot1 

OPH 15 7JC15) 

teAT~ 

~ l$'J!l<Sl 

mi 

MCAT TOTJ OPH 15 7 (,PS} 

>rAT~ ~ 157. · (RS) 

tRSl - Socres 

--------------------HEM------~---------------

------TEST·-------

! ~ ~ 

509 200.62 24.19 

5Q~ 2G4.7l 28.68 

... --CRITER.ION---

~-

MEAM SD r - .629 .484 .38 

.629 .484 .4.4 

-------------------·--wHlTl-••·----------------

------TEST·------- ---CktTERlON---

!! .... m ~ ~!! .!. 

539 201.54 23.37 .640 .• 480 • 34 

539 264.62 28.62 .640 .lt80 .41 

TAlliE 120 

NWJ!SIS CE' TI:ST FAIRNFSS . 

---·---------------WOMEN---------------------

------TESt-----· 

.H. ~ !!! 

66 108.02 28.80 

66 2~4.47 34.33 

---CRITERION---

~~ L 

.515 .504 .39 

.515 .504 .46 

-----------------•-••BLACK----·---------------

------tEST-------

!! ~ SD 

18 166.9) 26.88 

)8 232.41 33.03 

,.., -· ' ... •:'. ~ ;I 

-··CRJTUlON---

~ so ! 

,))6 .47i .64 

.)l'& .47l .62 

\' 

-------TEST OF-------

SlGNIFt-CAHCE 

STU lNTER-

!:!!Q! .§.!B!! f!!! 

NS NS NS 

NS NS NS 

-------·TEST OF--------

SIGHII-'ICANCE 

STD INTER-

~ SLOPf ~:£IT 

"" NS NS 

NS NS NS 
·--~ -· 

\ 

--

In. order to transmute. t-he total raw scot'e for the ATC. trainee Sa.Dlple. it was 

nee~ssary to est;ilp~t.e the mean sco.re which would have he~n· obtained by an 

ATC app-licant ,gt"oup on the ATC test tat.tery and equa~e this mean score to· 

a passing score of 70. Fortunately i the test,s used in the ana.lysis of ~he 

ATC trainee S'llllple were i~entical to the tests taken. by the5,931 ATC. applicants in 1978 except for the second form of MCAT which w•s not administered 

to the ATC applicants. Consequently, given the mean scores on each te·st 

which wer.e available for bo.th the ATC t-r.ainees and thf' ATC applicants. it is 

possibl~ to estiJD.a.te a Clean for the AIC applicants on the second MC.\t form. 

By combining this estimated mean for uMCAT 2-" with the actual :~e.an scores 

ob~ained by the ATC applicants on the other tests and weighting them, an 

estimated aean score of 161.2 for the 1978 ATC applicant group on a test 

battery comprised of MCA! Total and OPM !57 vas derived as shown below: 

1978 1978 

ATC Applicants ATC Tu inees 

~ X :!!!· • ~ 

TOTAL 

MCAT l 30.6 2 61.2 

*(KCAT 2 35.6 2 71. 2) 

OP:i 157 28. 8 1 28.8 

Est. Mean. Raw Score lhl:'2 

(1) ATC A~plicant Mean - MCAT 1 

ATC Trainee Mean - MCAT 1 

(2) ATC Trainee Mean - H•::AT 2 

ATC Trainee Mean •• liCAT 1 

Difference 

36.9 

42.6 

38.8 

2 

2 

1 

30.6 9 36:9 • . 82 

42.6 

36.9 

5:7 (6.0.} 

*(3) Est. MCAT 2 Mean for Applicant Group • 35.6 

(6.0 X .829 • 5.0 + 30.6 • 35.6) 

73.8 

85.2 

38 .. 8 

197.8 

Since the correladon between the two differently weighted ATC tests for 

the ATC trainee sample was .93 (Table 98), an estimated mean score for the 

Ate appUeaat group of 159.5 vas also derived using the .:atio of the mean 

acorea for the 1978 applicant and ATC trainee t"<oupa for the eommon weighted 

testa as .ahc!lln on the next page. 

141 

' I. 

,. I· -

! 

•. 

/ 

1 ' . 

\ \ 
.. -..-.:.. >-

\ 

' 

(1) .Ac~ual Applicanl. Grou\ l!e·an (N • 6000) 

(4 X KCAt ~ + 2 X !57, · 

(2) Act\141 Mean ATC Trainee <;r.oup ~N • 592) 

(4 x KCAT 1 + 2 x 15 7) 

(3) Actual Mean ATC Trainee Group (N • 592) 

(2 x MCAt 1 + 2 x MCAT 2 + l x 157) 

• 

-

-

., .. .- .· 

-. 

181.6 (Table 43) 

.225. 2 (Table .69). 

197.8 (Table 69) 

(4) 197 •

8 X • 159.5 Est. MEAN for Applicant Group on ATC Test weighted 225.2 X 181.6 (2 x KCAT 1 + 2 x KCAI 2 + l X 157) 

Given thes<o two estimated mean score" for an applicant group on the ATC test 

battery compriseji of MCAT and OPM 157, a mean value of 160 was used as a 

"pas,ing" score a~d. equate4 to a transmuted score o-f 70. The. total raw 

weiahted scores tor each ATC trainee on MCAT and OPM 157 were then transmuted 

by: 

I • ( (RS.- 160{ x 30) + 70 s !lAX RS - 160 

RS • Total weighted raw score 

!lAX RS • 255·. 0 

MEAN. RS • 160.0 

An analysis of fairness was completed using the obtained transmuted test 

battery· scores for the ATC trainees. A separate analysis was completed using 

the transmuted scores plus extra· credit points (0, 3, 5, lOr 15) derived 

from OKT raw scores for those ATC trainees .who "pass·11 the test battery with 

acores of 10 or IIIOre. The results are prcwicled in Table l2l. 

.. 

; I 

1411 

-~ . ' ... ., \' '{' 

j/ 

.'/ 
.-/ l ., .. , 

/ 

'\ 

j 

I 

,/ 

' ·l 

1 . ' 

' 

I 

_/· 

~. 

,. 

i 

=~ 

,,-

/ 

:4 

' .; 

.1 

~· ',, 

.!'·!· 

,. 

:l· 

j, 

I r.' 

1/. 

---·----------------~---------------------

-~-._·TEST----.-.-

ll§! ! KEAIC SD -- - )tt.\t TOtAL' 

Ql'H- 1~7 J (TS) 509 82.83 7.64 

KCAt .TOTAU1 01'11 157 J. (TSl 509 

QICT 1 (Pl'S) 

87 .. 50 9.92 

-·-CRITEIUON-'-• 

~ ~ .! 

.629 • 464 • )ij 

.629 ,484 ,44 

-··---------·WHITE-·_.. ......................... ---

~--T£$T-••...,.•• 

T!ST l! !!1M! !!! 

HCAT tofAi1 

Ol'K )57 ·-f (TS) 539 tU.l2 7.18 

MCA~ TOTAL}} . OPM 157 · (TS) 5l~ 

GgT } (PTS) 

87.48 ~.-68 

(TS) • Transmuted Test Ba:t.tery $cora 

(fTS) • GKT. Po1nti 

.~ 

... 

"' 

-·-CklTEUON--• 

~ ~ !. 

.640 .uo .34 

.640 \~0 .!.') 

TT>~IF 111 

ANALYSIS OFTt:.S'f" FAIRNESS 

------------·------WOH!N-··------------------

............ T£ST--··-• 

l! ~ !! 

66 78, HS 9.10 

66 ·80.8tl IO.JS 

...... CftlTERlON-·-

~ !!! ! 

.~15 .~o·, .JfJ 

,515 .504 .41 

------------------··-BLACK-••······------··---

............ TEST··-···· -·...cllTUIOH-··-

l! ~ ~ ~ !!! ! 

38 72.19 8.49 .316 .471 ·" 

38 76.14 11.90 ,316 .471 ,61 

----~--r£st or-~-----

SICNtFICANCE 

STIJ INTU• 

!!!':2! ~ £!!:!. 

N!r NS NS 

NS us "' 

--------tEST OF-------~ 

SIQHFICANCE 

INTtR• 

~LOP£ CtPT 

StD 

~ 

NS NS NS 

NS NS NS 

I 

./ 

/ 

~- .l 
I 

I 

., / 

the combination of transmuted 'test battery scores for MCAT and OPM 157 plus 

extX"a credit point.s for air traffic control relate4 knqwle:dg:e based ori. OKr 

scores replicates the. operational, use of the selection tests. The analysis 

of fairness provided in Table lcl shows that this test battery including 

OKT, predicts the ATC laboratory pass or fail/withdraw crHerton equally for 

men and women and· fo·r Black and White groups. 

the final step in the fairness ana·lysis was to examine the- entire test batte_ry 

co10prised of MCAT, OPM 157 and OPM 24 with and without excra credit for air 

traffic conttol rela·ted knowledge based on·. OKT s_cores. The results are provided- in Table 1:!3. In order to add 010' extra credit points, it- was again 

necessary to transmute the tot::.l raw score for MCAT, OPM 157 and OPM 24 and 

to establish a passing raw score which was equated- to -a tninsmutl;!d score of 

70 as shoWn below: 

1978 1978 

ATC Applicant$ ATC Trainee~· 

~ MEAN X WT • TOTAL ~ X WT • TOTAL 

MCAT 1 30.6 2 61.2 36.9 2 73.8 

*(MCAT 2 35.6 2 71.2) 42.6 2 85.2 

OPM 157 28.8 1 28.8 38.8 1 38.8 

OPM 24 40.6 1 ~ 46.6 l ~ 

Est. ~Iean Raw ScUre 201.8 244.4 

*See page 143 for derivation of estimated mean score for MCAT 2. 

Again, an estimated mean raw score for the applicant group w-as derived using 

a ratio method as follows: 

(1) Actual Applicant Mean (N • 6000) 

(4 X MCAT 1 + 2 X 157 + l X 24) 

(2) Actual Mean ATC Trainee· Group (N • 592) 

(4 X MCAT 1 + 2 X 157 + 1 X 24) 

(3) Actual Mean ATC Trainee Group (N • 592) 

(2 X MCAT 1 + 2 X MCAT 2~ + l X 157 + l X 24) 

• 222.3 (Table 62) 

• 272.6 (Table 83) 

• 245.6 (Table 83) 

(4} 245.6 

272.6 

X . ---. 222.3 

200.3 Est. Mean for Applicant Group on AIC Test 

Weighted 

(2 X MCAT 1 + 2 x MCAT 2 + 1 X 15 7 + 1 x 24) 

Given these two estinlated mean scores for an applicant group (201.8 and 200.3) 

on the IIXC test l!attery c010prtsed of MCAT, OPM 157 and OPM 24, a mean value 

of 200 was usecl as a npassin~" scqre and equat~d to a transmuted score_ of 70. 

150 

' 
/ 

The total raw weighted scores for each ATC trainee on HCAT, 01'!'1 157 aA<I 

Ol'H 24 were then transmuted by: 

- ( \~S - 200) X 30 ) + 70 -IW\ 'P.S -200 

R5 • Total weighted raw score 

MAX RS • 306 

MEAN RS • 200 

' ' 

Table 122 shows the rt'csults of the fairness analysis on this AJ:C test battery 

(for both raw anti transmuted scores) and for the ATC test battery transmuted 

acores in combination with extTa c.redit points based on OK! for those- trainees: 

who scored 70 or m.ore on the test.. battecy .. 

As shown in Table 122, there were no-significant differences on the ATC test 

battery bet~eeri men and women. However, the addition of OPM ~4 test scores 

(Arithmetic Reasoning) to the scores of HCAT and OPM 157 resulted in the esti-

.. ted standard error (population variances) between the Black and White groups 

being statistically significant. The statistical tests dev.eloped by Gullik.sen 

aad Wilkes (34) t·equlre that if a significant difference is found at any of 

the thre-e •teps ~ the ··analysis, no· further· te-sts are made since the regtession lines used fot predictions-·are. by definition, uneq:ua.l. 

151 

•·· . 
,_ 

/ 

_·"" 

'\: 

' 

-... 

' 

;/\" 

' 

' 

. 

I 

·, 

--

I 

t _I 

... ' 

_;.. .... · .. 

/,.~ 

I 

' J 

II 

F 

'I~ 

/ 

... -------~---~-- -~--- ~ --

.... 

nsr 

HeAT TOTAL} om t 57 (RS) 

Ol'M 24 

HCAT TOTAj Of'M 157 . (TS) 

..... 24 

OPK H7 (TS). 

-~-----

-tat-

! !!!!!! !!1. 

509 2~7.29 26.95 

509 8).)3 ., .(13 

•-caiTWOII--

~ §! .! 

.629 .484 ~31 

.629 .484 • )7 

HCAT TOTAL} 1 

OPM 24 509 ••••• 9.72 .629 .loP.~ .44 

- J<PT& . 

_,..__:.._ 

-~-~ -c&ItwoiM- ntt .! Ill!!! §! !!!41 §! L "' N MCAT.TOTJ 

OPH 157 (lS)- S39 248.32 25.97 ,640 ,480 .)) OPH 2/. 

HCAT TOTA'] .OPM 157 (TS) 519 OPM 24 13.6_7 7.35 .MO .480 .ll 

OPl'f- ~AT 157 TOTAL] . (fS) J 

OPM 24 - S"l9 IB8.0J 9.42 .640 .480 ,40 

OKT ] (PTJ) 

TABLE 122 

AKAL.Ysu or t£St r.u~us 

--------------· --- -----TEST-

!! ...... !!! 

66 235.27 30.08 

•• 79.9$ 8.SI 

•• 82.01 10,09 

--

--T!St-

.!! !!!41 

)8 21-l. 62 

)8 n. 29 

,. 77..16 

/ 

~ •• 

28.61 

8.10 

t I, 45 

,. 

•, 

II.ACJ 

-CI.lTU.J;Q¥o.-... 

!!!!!! §! L 

.su .SOl! . • 39 

.sn .SC4 .)9 

.515 .,504 .42 

-42l'IQW»--

~ !!· L 

.]16 .4-11 .68 

,)16 .471 ••• 

• 316 .47) ,64 

~--rut o•---

ll<*lPl CAIICt 

STO urr~-

!!!!! SLOP! Ji&! 

.. NS NS 

NS NS •• 

•• NS KS 

-nsr 01'-----:-

STO 

llCIItflC4JtCI 

INTU.• 

!!!2! §LOtc u.n; 

S!G 

stc 

NS NS NS 

I 
The fact that ::he regression lines fot the White and B·!ack grqups are statistically W>equal, by itself, doP:> not identifY' the practical 1tt>plications for 

a selection procedure. Und·er the Cleary model of test fairness, if tests are 

"biased" (unequal). an alternat.ive solution is to use the .separate reg~~ssian 

lines for selection decisions. This solution, howevec, explicitly introduc·es 

race, sex, or ethnic group (iri this case race) as a prediCtor in the_ selection 

process. This is incomp.ltible with the ethical position· of "Qualified 

Ind.J, vidua lism" and la1o-· ~ 1:.~) • 

In these circumstances, a sin&le regression line derived by combining the 

groups typically has been used as· the basis for .selection procedures. The 

effect of this is tO "ov.erpredict" the criterio!l (e.g., performance) for 

that group which scored low~r on the test and had a lower mean on the criterion ~easute (J2). In the_nnalysis of this ATC test battery, use of a 

single regression line obtained by combining the White and Black groups (or 

the total sample} would "bias" the selection procedure in favor of the Black 

group. This result con{ot'li>S with the general findings in the published 

literature· referred to by the General Accounting Office in theit review of 

Federal selection tests DJ). 

table 122 also shows that <~hen extra credit points based on Oltt are added to 

the transmuted scores for this ATC test battery, there were no. signif-ic-ant 

differenceS between men and women or the Black and White groups. While 

this- indicates that the combination of the test battery a:ud OKT points predicts the ATC laborato.Jry pass or- fail/withdr_aw criterion equally for all 

groups, the test battery i"tself would be used to establish initial appointment eligibility for applicants. The fact that addition .of OKT points results 

in equal treatment for those who pass the· test doe'S not adequately address 

the unequal treatment resulting from the use of OPM 24 under the U~iform 

Guidelines. Given that OPH 24 results in unequal prediction of the pass or 

fail/withdraw criter-ion between the White and Black g-roups, it shoUld not 

be used as pa:rt of the ATC selection battery. From a practical viewpoin.t, 

the exclusion of OPM 24 has essentially no tmpact on the multiple correla-

.- ·"tiftft or the .predictive value for the total _group or for men and women Since 

its contribution to the prediction of the performance criteriOn is very 

..,..u (see Tables 95 and 9n). 

Alternative Selection Procedures. During the course of these research studies. 

a numbe·r ·of alternative instruments of a 'cognitive and non-cognitive structure 

were examined. The 1972, 1977, an~ 1978 studies of full-performance, develop'-

mental, and tra-inee air traffic controllers examined the relationshiP between 

non-cognitive instruments (including biographical dau and personality tests) 

and performance criterion. Use of biographical data as an alternative 

selection procedure was examined in some depth. 

In May 1979, th~ "•.A proposed ts the Office of Personnel Management (OPM) a 

demonstratioA pr, ~c.t Under the Civil Service Reform Act that address-ed 

recruitment and s-election of wo_men and minorities in the air traffic control 

occupation (35). This proposal involved a 5-year period during which. approximately half of the. new ATC hires (750} ill, the FM Souehern region would be 

153 

----- ______ ,_ ------------------------- .. ... ..r· ----,-,/ :.··~- ·,~· 

:.~- ' I • - •.-" 

' 
selected in nonregistor order as a control g;oup and the other half (750) 

se"iected in tue regular man~er from the OPM register to form a nqncontrol 

group. Candidates in the Southern region wou.id be givef\ a detailed multiple 

choice biographical q·uestionnaire· ancj "profiled'' on relevant life expe.rience 

dimensions and grouped into life prOfile clus.ters. The life profi--les would 

then be us.ed as an altert,.::t:e means for ranking and sele-ction 'of candidates 

fot the control group. Succe.:o.~ in A!C training_woulC. then be eValuated for 

both the control and noncontrol &f.:'lups as a m-eans validating the altel.native 

selectiQn proc'edure ( 16). 

After review of the propos_al and. the biographical questionnaire, OPM pointed 

out a number of concerns on "Job Relatedness", Priv...1cy, "Subjectivity'', and 

"Public Relations" for most of t-he· questions included in the biogr~phical 

form (36). Op December 4, 1979, the DeputyeDirector, OPM, suggested the use 

of numerical scores based on "life profill;!s'' derived from empirical data 

established from validity studies. It was a!so ;>ointed out that the "job 

relatedness" issue presented a difficult problem because of practical and· 

legal requirements (37). 

The biographical· questionne!.ire ?reposed for this p.roject has been administered 

to 545 of the ATC trainees attending the FAA Academy during the period 1976-

1977. Information on sex, race, OPM selection scores, ATC training labOratory 

composite scor.es and training pas.s/fail status. ·was also available fo·r thi.s 

sample of 545 studeli·ts. These data were provided: ·to the Institute for 

Behavioral Research, University of Georgia, for analysis. Eight factors were 

identifie~ through factor analysis: 

o Academic factor 

o Sot:.ial factor 

o Child Relationship factor 

o Initiative factor 

o IFR/VFR Experience fac·tor 

o Parent~! Permissiveness factor 

o Physical/Sports facto!· 

o Socio/Econ~ic factor 

Based on these, the 545 trainees wer ..... clustered into six subgroups with simila":" 

life experien,ces. Because of the weak relationship between Sl:Jecess or failure 

in the laboratory· training and the subgroup memberships, the. eight li"fe expe:tietice factors were used to predict the laboratory score usirtg regression 

and stepwise regression analysis. The results shoWed that the factors w~re 

not good predictors of the laboratory compOsite score which determined the / 

pass/fail status of ATC trainees. Generally the results showed that AT.C 

experience and tests involving mathematics and phys-ical scie-nces would yield 

better predictions of success in AT training. The report (39) suggested 

that using a sample of working controllers to establish similar life experience and group membership might prove more meanin!'ful. llhile this had been 

planned. as part of the project metho.dology ( 36), the resul.ts of the University 

of Georgia study, the OPM concerns regarding validity of biographic.al data as 

a selection basis and cos-t consideration resulted in ·terminating this .approach 

toward alternative sele-ction procedures. 

154 

/. .-..:, )-'k " ' ~-,, ., !-;~~ ""':"""'-· , ·. .• j, ~ ~., 

r 'j >·--~- ' -·- / ~ 

,. fi;i-~ .. · X j I 
As pointed out tn conjumetion with the •!lalysis of the salilple of 953 ATC 

trainees attendtng the FAA Academy !n 1978 (page 101), the FAA recognized· 

the difficulty ·of selecting women and minorities from competitive OP~l 

registers in 1968.. As. a conseQ.lJencl!!. the P.ted·evelopmental-ATC prog:ra..r,:1 ;.ra$ 

established. ln 1974, Executive "rder 11813 provided. for npncempeti.Uve 

coovef.$ion of Cooperative Education students and this· authority was incorporil-ted io the recruitm:ent and selection p·roaram for w0111en -and minorities 

in air traffic'and other FAA occupation•. 

Thes·e alternative selection. programs_ have bee]J. an.d will continue t;o be a 

maJor vehicle· to addres$ the adverse impact on women and minorities as 

well as the federal Equal Opportunity Recruitment P•ograJO (FroRP) requirements established by the 1978 Civil Service Reform Act. 

Table 121 shows the total nUIIIber of Al'C traineeo hired from January 1976 

through October 1980; those hired from OPM register (competitive) and those 

hired through the· ~re&evelopmental and Co-op program {non-competitive) by 

rae e and _ sel!t--

TAJ!LE 123 

A1'C Tl.lt.tHtES Hlk£5 

J,_uarr 1976-.- -Gc:t<~.,n 1980 

Croup 

Totd 

Hi 

1 of CocJpetitive 1 of Moo-~titive- 1 of rea Total Hiru crOup Hi.r'U Croup 

... 66)3 ( 84. 3%) 62UI ( 93. 51) ltlS 6. 5%) 

-· !ill ! 15. 71) ..ill .!.~7~4!)_ !!!!; .!. !2~6!)_ 

Tot-•~ 7894· Ct·oo.ou 70H ( U.4%) an 10.6%) 

._,riua lDdi.aa 21 .3!) II ( 15. 71) 3 14.31) 

Miaa 66 .911 " I U.411 5 ( 1.~u 

Bhpuic no ( 2.81) 134 ( 60. 91) •• ( U.U) 

I lack 618 ( 7.91) 300 ( 4&. S%) )II ( Sl.51J 

Vbite !!.!!! ( U.ll) ~ .! !l~8!l_ ill .!.""6~2.!1_ 

Tot.• I nts (100.0%) .. ,s ( 19.2ll 140 ( 10.1%1 

155 

,. "f'l.· •' '\, ·, I ' ·~,. \ 

.. : .. ~--4.~_... . ' .1 ,, ~ ·-\ " ., . . - -.....; \ ~ 

L. k./.'. ,.,_ .. f...:..'~ .. · .. -~-........- . _ . .-;· """' / -~ 'I . • ... · .. 
.. CO!ict,lJ,SIONS A.\1> R.ECO!«E:<.'DAT!ONS 

CONct,lJSIJ)I'S: The various resea~ch .studi<~s previously discussed support the 

following conclusions. 

• The experimenta-l ATC test battery COI!lprised of MCAT, OPM !57, 

and OPM 24 is a valid an!i statistically scgnificant tool for 

use in screening applicants for the ATC occupation. The need 

for screening applicantS is pa.r-ticularly important fo-r this 

occup.ation because of: (1) the high co~t of training to the 

taxpayer; (2)- the large number of applicants in relation to 

the. relatively few vacancies Whir.l:l are filled each yea_r; a!id 

(J) the fact that there is essentially no "self -screening" on 

the part of applicants si~e there are no educational or specialized experience requirements for appointment eligibility 

at the ent.ry grade levels (GS-5 or GS-7). 

• The use of the Occupational Knowledge Test (OKT) in conjunction 

with the experimental ATG test battery increas-es the predictive 

validity of th" selection proc·ed1,1re. It provides a l;>ette.r 

lllethod of establishing the applicant's knowledge relevant to 

the ATC occupation as compared to the Rating Guide presently 

used by the Off:lce of Pe:rsonnel Management~ Further. the 

use of oKT allows applicantS who have acquired this ~owledge 

outside of the specific work experiences given credit t.mder 

the present_ Rating· Gu,id:e to eanl.- extra credit -for competiti-ve 

selection consideration. 

• The use of the ex:pitricental ATC -test battery and OK1' s~ores for 

ranking competitive ATC applicants for appointmertt consideration 

and_ seleCting those with the highest: scores first is a val.ici' use 

of the applicanti:t test scores sin~e it si€;nifican·tly increases 

_the probability of success in the ATC occupation as measured by 

pass;ng the Initial ATC Qualification Training Program. The 

addition of veterans preference points to an applicant's score 

on the ATC test battery is not a valid predictor of success in 

the ATC oc~upation. 

• The statistical analyses encompassed·- a large number of different 

experimen-tal test_s including the five OPM tests presently u!jied 

for scoring ATC applicants. The resulting experimental ATC 

test battery (MCAT, OPM 157, and OPM 24) was derived from 

multiple_ regre$sion analysis. the tests were examined with 

respect to both unweighted and weighted test score values. 

Only the OPM 157 test of the pre~ent five OPM tests contributed 

to the multiple correlation (R) predicting the ATC training 

(performance) ariteria. Weighting the tests increased the 

validity of the ATC test bac-tery. 

156 

¥ .

...................... z· ..,., , ~.1' ....,. - • , . . r .···-\··.·' • .. ~-_; 

• /" .. J.· i/', ·\~~~ ' 

y. r? ·--:.t .... ~ ... ~ .... !~ .... 

... ,.. ·.'~.. ; . ·."!.·; · .... -~~·.·?..· t •. -~=--~~~~~ 
• Analysis of the utility of the ATC ten battery in41cates a 

significant potential for redi.ICing overall cost of ATC 

training. The now tes~ is effective in identifying those 

applican~s w!)o P~"" the present OP!t ATC test but· have high 

faH/withdraw rates, ~., the Initial ATC Qualification training 

program. o:pel"ational USt. of the new ATC test battery \oi'Ol;~d. 

requir~ m.ore_ f~equen.t admin1.:tration (perhaps chrl!e times a year) 

in order to obtain more applican.ts. with Earned R..r.ti.ng scores 

above 85 as. comra.red to the present "rM- t_est. . 

• Establishing the passj.ng score on the new ATC test battery at 

the approximate. mco:1n score for an ATC applicant group is 

supported by the analysis of fail/withdraw rates by score range 

groups. Setting a. higher passing score (for eXample a transmuted 

score of 80) would sigoi.fican~ly inc't'e-:.se the adv~rsa impact oa 

women and minorities. as well as make it extremely· difficult to 

recruit women and minorities through either·competitive or noncompetitive select1on procedures. 

• The present OPM ATC test battery and the experimental ATC tes.t 

battery both have .an adve-rse. impact .on some min~rity groups, 

particula-rly Blacks. When extra credit for veterans preference and 

ATC related knowledge· qr exPerience i~ gran-ted· to those applicants 

who pass t:he test battery, there is also an adverse impact on 

wc.men. T_he experim~rital ATC test battery lta-~ a somewhat. greater 

adverse impact on sele€tion_ of Women and Blacks than does the present 

OPM"ATC test batterx. The veterans preference credit is a significant factor in .the ·adverse· im.pact on Women. While· the stati-stical evidence doe.s not support the validity of veter-ap.s preference 

in relation to perfonnanLe criteria, it is required by law. 

• Given the evidence of adverse impact· on women and some minority 

groupa, especially Blacks, analyses of test fai.rness were completed on the yarious codlponents of the experimental test ba..ttery 

and the OKI. These analyses show ~hat the requinments of the 

Uniform Guidelines on Employee Selection are met for the I>TC test 

battery comprised of MCAT and OPM 157 and for MCAT, OPM 151 and 

OICT e_xt_ra credit· p·pints.. This _was true for both men and wom.en 

~d the White and Black groups. 

• The Multiplex Controller Apti·tude test (MCAT) is the major component in the validities obtained with the new ATC test. The 

analysis 6f fairness for MCAT, by itself, include~ a sample of 

men and women a.s well as White and Black trninL"'es which w~1s proportional to the total populati<m of ATC trainees attending the 

FAA Academy during the period oo January 1976 tbro14gh October 1980. 

157 

\ . ' ~ ~....; 

.·\i.f~. · .... 
• the addition of OPM 24 test scores to the MCA:f and OPM I S7 test 

battaty reveal.ed a significant difference in the population 

variance between the White and Black groups indic.a-ting: that an 

AIC test b"ttery c0111prisec;l of MCAT, OPM 157 and OPM 24 did not 

meet the fairness requirements of the CniJorm Guidelines !or the 

Whit'e and Blac~ groups. The effect of !:h~se Jifferences would be 

to bias the -test battery Somew-hat in f_avor bf the Black .g{oup if 

a c_om.mon regression line for the two groups were used. However, 

there-·were no Significant differences betwtet:t men and women on 

this ATC test battery. 

RECOMMENDATIONS: Base'd on the conclusion derived from the resear~;h studies, 

the fQll6win,g rec.QIIlmenda.~ions for the use of selection ·tests and procedures 

to qualify, rate- and rank applicants for the Air Traffic Control occ:upation 

are: supported. 

• The ATC Test BatterY used for qualirying applicants for pl~cement 

on competitive OPM registers or for qualifying applicants for 

uon-competit.ive appointlllent consideration should consist of: 

(1) The twQ fonns of the Multiplex Controlle~ Aptitude Test 

(MCAt) with the. total correct scores weighted by a 

value Of ~. 

(2) The present OPM Test, Abs'trac t Reasoning and Lette_r. 

Sequence sco-red (R-l/4W) weighted by a value of 1. 

(3) Each form of the MCAT hss 55 questior>s segmented into 

two parts; 2.0 minutes~ is allowed for Part A {27 or 28 

questions); and, 15 minl\te~ allowed for ·Part B (27 or 

28 questions). Eight minu.tes is also allowed for 

reading test_ directions and practice pro-blems. Theref.ore, a total of 43 minutes is required for administr-a+ 

tion cf e.ach HCAT form and the total test time require<! 

for both MCAT fotDIS :is 1 hour ·and 26 minutes. The 

Abstract Reasoning and Letter Sequence test is a twopart teSt; each part has 25 ques·tiori.!J. T·est time for 

Part A is 15 minutes; Part B is 20 minutes. There are 

9 sample item~ with 5 minutes allowed for these praccice 

ques-tions. 

(4) Total testing time for the recommended ATC Test Battery, 

including practice a,nd test familiarization time, is 

2 hours· .and 6 mitiutes. 

• The colllpoaite weighted raw acore on the test battecy should be transmuted by a linear· transformation of the distribution so that the 

. 

aean raw score of the applicant group is .. eq_uated to a passing score 

of 70. 

158 

; 

' 

\ 

., 
• All a-pplicantS~ either C:ompetit!Ye Qr no.n-competitive, who $core 

70 or gre·aeer should be considered as eligible for appoint:Uent 

consideration in the Air Tra£fic Control occupation. 

• The OccupaUonal Knowledge Test (OKI) should be administered to 

all competitive and non-co;npeti.ti'Ve applic.ints toðer with the 

recommended ATC Test Batte~y~ The results of the OKT test should 

be used for granting additiona!. earned- cr2dit in place of the 

prese~t OPM Rating Guide. 

(1} The OKI is an 80-item test and is scored for correct 

answers only. Test time is 50 minutes. There a~e no 

practic~ items. 'i"..,ro minutes are allowed for reading_ 

test directions. 

{2) The total test administration time, including OKT, is 

2 hours and 58 minutes. 

(3) Additional earned credit should be granted only to 

thOse applicants wbp achieved_ a score of 70 or ·greater 

on the qualifying ATC Test Batta.ry (MCAT and OPM 157). 

(4) Additional earned credit points should be granted to 

qualifying applicants based OR OK! scores as follows: 

OKI Scates 

Right Transmuted 

010" Scores Scale 

()..: 5ll ()..: l 00 

52-55 

56-59 

60-63 

64+ 

• 

-

65-69 

70-74 

7)-79 

80+ 

• 

= 

Additional 

Points Earned 

3 

5 

10 

15 

(5) For competitive applicants, additional points based on OKf 

scores should be added to the scureS of appli~ants who 

achiev-e- 70 or more on the AIC Test Battery. The total sc.ore,. 

together with any veterans- preference points_, should be used 

to rank candidates for selection c~nsideration on the OPM 

r-egister. 

(6) For non-competitive applicants, the OK! scores together with 

the ATC Test Battery should be used as a basis for offering 

GS-7 appointments in the ATC occupation rather than GS-5 

predevelopmental appointments t~ those individuals whose high 

seo.res indicate they have the aptitude and lcnowledge to 

enter the GS-7 In:Ltial ATC Qualification training. 

• Qua~ified applicants should be ranked for competitive appointment 

consid-eration based oo the BUill of their scores on the ATC Test 

Battery, the OKI and Veterans Preference credit with thos·e applicants haying the· highest total score (Earned Raeing) given first 

consideration for seleetion. 

159 

... r· . . -~· ., ·.·~-· ·. ·~ ~ 

' : \!/ . . I , .~ •. 

' fll .. !. :.7-Ji . . ' . •k I / 1 

.·}· !;/. 

; 

i 
• The el<isting OPM regist'>r of qudified applicanc, (appr~"imately 

4,500) should be retained. Applicat.ts who qual! :y on tl,. basis 

of the new ATC Test lfatt~ry should be ,nterspnsed in their 

appropri~te rank order with those aPplica-nts who a.i:'e presently 

o~ the register. 

• T~he six parallel fonn.; of ~CAT and the ei;~ht parallel for;:ns of 

OKT which have bee-n deveh::'ed and used in conjunc_tiotl with the 

validation studies should be ~laced into operational use for ATC 

app lie ant screening and se lee tion. 

• The FAA, Office of Aviation ~edicine, should be delegated the 

responsibility fran OP~ for continued develop~ent of additional 

parallel forms. of MCAT and Otcr .to insure against compromise of 

operatioQal test fcrms. 

160 

... -'--·-- .,.., . ..._ ---~--- _______ .. 

---·--· ... __ :;... _, 
\ 

B l BLIOCR.\Pl!Y 

• 

' 

(1) 81"\'"~k..l'tt.'~ L. llist~)ric Ovt>rvh~ .of Rcs.2~rc_t~ ~n.d Oe·Jcl0p~!£_~Air TrafJic 

Cl•ntn,llcr SCTCCti:m Selection Procccd.in~, ~lilitary Testing Ass(.H.:iatic.m., 

oc·t:;be"r 1~19, 19f9: 

(2) C"'-bb, B. B. Air Traffic Aptitl!,.~~.Test_}~.l~·a.~ures of Military nnd FAA Controll-er 

Cl'~ntn.,ll~r Trainees. Fedt.•ral Aviation ,~!ministration, Office of Aviation 

;:t~Jrc-i-1\c~-Repc:r'tNO. A.'t-71-.'40-1971. 

(3) Cobb~ B. B. A Comparativt:> Study of Air Traffic Trainee Aptitude !_est Nea.sures 

ltwt'~}\'ins Navy, NUrine Corps., and FM C,.)ntrollers. Federal Avi"'tiun 

Administration, Office of Aviation Hedicine,' Report r>o. AM-68-14, 1968. 

(4) Cobb. B.B~-, C. L. Young,. .and B.· L. Rizr.uti. Education as a Factor 10 the ~electi~n of Air Traffic Controller Trainees. Federal- Aviation Administration, Otl-~~­

of Aviuien H~dicine, Report l\o. AM-76-6, 1976. 

(5) C.ollT'l~n, J. G. R.ev:iew and Evaluation :lf Present System for Selection of Air 

Tra£(lc Controllers" ''

1

ashington, D.C. Report for Phase 1, Task I. 

Federal Aviation J\_dminis.tratioo Cqntract to DOT-FA-70\-.'A-2371,. Education and 

Public Affairs Inc., July 1970. 

(6) Ni"·~, J •• J.G. Cfllmf'n and 0. Dorncne-ck. Predictin~: Sltccess of APP-lic;;.mts 

for P\."'Sitions as Air Traffic Control_S[!~t: ialists in the A1r lraf tic Serv1ce. 

Education and ·Public Aff.<.lil·s Inc., Contract DOT FA-7"5~A.,;.,3646, May 8, 1977. 

(7} \•.'l,rk"''" Attri-butes for Air Traffic Con-t n1l Jobs qnd Theic t-teasurernent. 

(d-;l ..... ation a.nd Public Affail:sTnc.; 1970. 

(8) Air Tr~1ffic Controller Selection and R~tcntion Task Force Report. 

Aviath,n Administration~ December 1974. 

Federal 

(9) Air 1'rnffic Controller Selection and Retention Cost Analysis. fed~:ral 

Aviation Administration, t-hlrch 1975. 

(10) y_,,ining of U.S. Air Traffic Controllers. Institute for Defense Analysis. 

(IDA Report R206), Januar)• 1975. 

(11) Air Route Traffic Control_ Center, Descriptive andFl~· Diagrams of Con~rol 

Functions. TII-5l29/000/00, System Development Corporation, 30 June 1974. 

(12) Hilne, A. ~1. and J, ·G. Colmcn, Selecti~n of Air Traffic controllers for 

Federal Aviation Administration. Education and Public Affairs Inc., 

. (13) 1!ummnry of Education and Public Affairs Inc. Report on: Predicting Success 

of Applicants for Positions as Air Traffic Control_ Specialists in the Air 

l!.affic Control -Spedalists in the Air Traff-ic Service. Federal Aviation 

Administration. August 1977. 

(14) Hies, J. and J. G. Colmen. 

da.tions for. E!CS Seluction .. 

\ 

Final Repo~t of Task 1, l)eve lopment of Rec_ommenJanua-ry 1976. 

' .. '· I I '· I ' 

I 
:j 

(15) Colmt;!:n,. J. Valiclity_,2f the Cattell 16 Perspn<~l_~~L!~-~~J~.t~n·aJ~~~~J 

Other "r\on ... Cogniti~•.'stS for Selection and i'la5emen~t~...!~._i_~_Tra.fi!_c_ 

f_o_!!E!_~l sp~ci.ilists.. Ed:u.:ation and Public Affairs Cnc., Federal Aviation 

AdminiStration ~:ontract DOT FA-75\~A-3'646, ~tay 8, 1977. 

(16_) Nies, .L. J. G. Colmen ~:"'ld 0. Domeneck. ~_r~r~~is._~~:_-~Pr~.~-i_r._ti~_S!J_~£_C~·f 

~cants for Positions aS A.ir Traffic Control Specialists in t_!l_c Air 

Traffic Service. I:ducntion and Public Affairs lnc.,. ~ontract DOT FA-7%.'~\-lfJ, 

Hay 8, 1977. 

(17) Unifcl'm Ct.~ldelines on Em..E!_~c Selectio\ "roct>durcs, fPH Letter 335-12, 

Ap~·~nd.ix A, Offi-ce of Pcrsoni:1el Nanagemeht. 

U8} Statistical Analyses of ATC Applicants, Office of Personnel ~lan:l~eme.ntt, 

. (Unpubli&hed, availaFle from OPM and FAA). 

(19) Daiiey, J. C., and E •• ~. Pickrel. 

for Air Tr'affic ContrOllers~ FAA 

No. AH-77-25; 1977. 

Development of New -Selection TestS· 

Office of Aviation Medicine Report 

(~0) Boone, J. 0~ Toward the DevelopmEnt of a New Selection Batterv for Air 

Traffic Control Sr~~cialists, FAA OffiCe of Aviation Ncdicin·e Report 

No. AM-79-21, Sep:"~mber 1979. 

(21) Thorndike, R. L. Personnel Se.l.ection.- Wiley and Sons; New York, 1945. 

(:?:2)' B-oone, J. ~· and. '!1. A. Lewis. The Developm-ent of a r\e"' Procedure to !ltakC' 

Naximum_ Use "?.!~~ila._ble lnfc;'ll:·mation When Correction Corr':ktlons_~E_~ 

Res·triction Ra.nge Due 'tO Selection. FAA Office of Aviation ~tedicine 

Repoit~N1~78-36; September, 1978. 

(23) National En Route and Terminal Air Traffic Training Progr_~· FAA Order 

3220.18, Jahuaq• 15, 1976. 

(24) Instructional PrograM Guj.de (IPG) :rerminal. TP-12-G-1, Hay 1976. 

(25) Instruct;f..pnal Program. Guide (IPG) En Route. EP-12-f·-lA, September 1977. 

(26) Le-wis, M.A. Use of the Occupational Knowledge Test to Ass~E~"<}-ia ~red_!_~ 

in Selection of Air Traffic Controllers. FM, ()ffic~ of Aviation Hedicine-, 

(27) Lewis~ N.A. Imp.lementation of an Occupational Knowled~e Test to Assip 

Extra Credit in Selection of Air Traffic_Controllers. FAA, Offic~ o{ 

Aviation Nedicine, Unpubiished Report. 

(28) Buckley, t. P., W. F. O'Connor and Beebe, T •. A Comparative Analysis ~f 

Indi-vidual and System Performance Indices fqr· the Air Trnffic Control ~st~~!.' .. 

FAA, NAFEC, Atlantic City, New Jersey,. 1970 .. 

(29) Bucl.<ley, E. P., and Beebe, T. The Development of a Hot ion Pictur-e 1-reasut·,•ntt•nt 

Instrument for Aptitude for Air Traffic Contr:oL FM. N·,:'FEG, Nov"embe-r ___ f?-itf .. - -

. . 

···-.... 

·- .• 
; 

' 

(:~0) ·D.,iley, J. T. Developntent of an oecup01t ional !,:now ledge Test for ATCS. 

FAA Office of Aviation Nedidne Report (Unpublished), April 1971. 

(31) Cohen., J. Statistical Power Analysis for Beh~vioral Science, New York: 

(32) 

Academy Pr-ess-.. 1969. 

Hunter"" .J. 

Analysis. 

C"mmiasion 

E .. and Schmidt, F .. L., Fairne-ss of Selection Tests: A Cr~tical 

Personnel Research and ~D:"e"v'-'c~l"o"e"m'ce"n'-'t'-:Ce2t",t:"eC.r~, "u;;-. sf?'.~C';i":v-;1;1'-',;S"e"=r~v:;i':c-"e= 

(PS 76-5), September, 1976 •. 

OJ) Federal Emplm-ment Examinations: .. ~ Th£l~-~ve_t::9!J!1l Opportunity and 

Iter it Principle Goals?, General Accounting Office, FPCD-79-1,6, Nay 15, 1979. 

(34) 

' (35) 

(36) 

(37) 

(38) 

Gullik.s-en,. H., and Wilks, S. S .. , Regressiort 'fests for Sever-al Samples .. 

Psychometrika, 1950, p. 91-114. 

Letter to AS'sistant Director·. Policy Analysis, Office- Of Per-sonnel Managem.en.t, 

from. Associate Administrator for Administr·ation, Fede"ral Aviation 

Administration, Nay 1, 1'}79. 

Resel!rch and Development Project on the Selection of Air Traffic .. Control 

Candidate•, Federal Aviation Administration Southern Region, June 1979·. 

Letter to Associate Administrator for Administration, Federal Aviation 

Administration, from Director, Personnel Rese.arch and Pevelopment. Center, 

June 28, 19-79. 

Letter from Deputy Dir'ector, Office of Persannel 1>1anagement to Associate 

Adm-inistrator for Administration, FM, December 4, 1979." 

(39) Gauger, K., Analysis of the Federal AViilljion Administration's Life 

Uperience Que~tio·nn.aire, lnstit.~ f~.;,.r . .U¢liavioral.. Research., Urtiversity Of 

Georgia, Harch 198;1. 

... ·,:(,:4',j~~)·¥0;"i~f,~····'··~ '. 

~ ' ,•• ·'"-~·'"""'"'";(~'-···: •. _...._;~~··.$ -:~.--: -~ ... "';;;;o'Oc-,.,. 

·· ...... \ \ 

I