Recovery of the FAA Air Traffic Control Workforce, 1981-1992

AIRCRAFT ALTITUDE SPEED ROUTE

10 7000 480 AGKHC
20 7000 480 BQUJE
ce] 7000 240 AGJE
4 6500 240 CHKJF
50 6500 240 DIKGB
60 8000 480 IKJE
70 8000 480 FJKID
‘SAMPLE QUESTION

WHICH AIRCRAFT WILL CONFLICT?

A. 60 AND 70
B. 40 AND 70
C. 20 AND 30
©, NONE OF THESE

Figure 2-1: Example Multiplex Controller Aptitude Test (MCAT) item

Symbols
+ pea pllVO 7] OAGHa

A BC DE

LyINvaA?T ery? \
A BC ODE

Letters
1) XCXDXEX A)FX B)FG c) XF D)EF £)XG

2) ARCSETG A) HIB) HU c) UJ pd) UI &)IV

Figure 2-2: Example Abstract Reasoning (ABSR) test item

12


CHAPTER 5:

ORGANIZATIONAL CLIMATE PERCEPTIONS OF THE AIR TRAFFIC CONTROL
SPECIALIST (ATCS) WORKFORCE, 1981-1992

Richard C. Thompson
FAA Civil Aeromedical Institute, Oklahoma City, Oklahoma

Five organizational climate perceptions of Federal Aviation Administration (FAA) Air Traffic Control Specialists

(ATCS) are examined from 1984 through 1990. These perceptions of job satisfaction, management concern for

employees, management direction, pay and benefits equity, and organization of work were found in previous research
to be related to the 1981 Professional Air Traffic Controllers Organization (PATCO) strike. Because of this

relationship, FAA developed several human resources initiatives to enhance employee perceptions of these areas. Trend

data for the five organizational climate dimensions are examined during the course of rebuilding the ATCS workforce.

The results suggest the FAA made modest improvements between 1984 and 1988, but lost some ground in 1990.

On August 3, 1981, the Professional Air Traffic
Controllers Organization (PATCO) engaged in a
strike against the Federal Aviation Administration
(FAA). The strike led to the termination of a large
portion of the Air Traffic Control Specialist (ATCS)
workforce. The strike also led to an examination of
the organizational climate perceptions of the FAA’s
ATCS workforce (Jones, Bowers, & Fuller, 1984).
Climate perceptions of the agency’s employees have
been obtained biennially since that initial assessment
as part of the agency’s human resource initiatives
developed in the wake of the strike. The purpose of
this paper is to describe controller perceptions of
organizational climate during the rebuilding of the
ATCS workforce.

The FAA and PATCO had a difficult relationship
for anumber of years. The strike was the culmination
of this conflict. On the day of the strike, more than
10,000 controllers walked off the job. President
Reagan gave the strikers 48 hours to return to work,
which served as a cooling off period and a deadline.
The FAA implemented a strike contingency plan
with non-striking employees, military controllers,
and supervisors filling in for the strikers. In the end,
those controllers who did not return to work were
terminated and the union was decertified. The FAA
had to quickly replace a substantial portion of its
workforce, find out why the strike occurred, and if
possible, correct the problems.

33

One of the investigations conducted by the FAA
focused on employee-management conflict, and in-
cluded an assessment of employee perceptions. The
assessment resulted in the “Jones Report” (Jones,
Bowers, & Fuller, 1982). Jones et al. identified sev-
eral correlates of the strike, such as increasing utiliza-
tion of the National Airspace System (NAS), episodic
controller stress, equipment failures, poor manage-
ment/employee relations and communication, and,
in general, low morale. The Jones Report found that
morale in the FAA was very low, even at the upper
levels of the agency’s hierarchy. Morale was indicated
by the summary pattern of employee perceptions
about the agency. Areas of particular concern in-
cluded satisfaction, management concern for em-
ployees, rewards (pay and benefits), organization of
work, and managerial and supervisory direction.

For the FAA as a whole, the Jones Report results
indicated a low level of job satisfaction. Job satisfac-
tion is typically treated as an outcome of the manage-
ment of human resources, so low job satisfaction
scores are often interpreted as problems with manage-
ment (Cranny, Smith, & Stone, 1992). Similarly, the
employees perceived that management did not con-
sider the effects of decisions on employees, suggest-
ing that management did not obtain adequate
employee input. Employees also perceived the job to
be inadequate in terms of formal and informal re-
wards. Instead, the perception was that rewards were


Chapter 4: Air Traffic Control Specialist Field Training Programs, 1981-1992

Table 4-4: Terminal field training status by FAA Academy program

FAA Academy Program*

Terminal Screen

Non-radar Screen

1981-85 1986-92 Group Total

Field Training Status? N % N % N %
FPL 2,538 79.0 2,840 88.6 5,378 83.8
Developmental

Facility switch 305 9.5 215 6.7 520 8.1
FSS transfer 68 2.1 14 0.4 82 1.3
Fail 301 9.4 136 4.2 437 6.8
Group Total 3,212 100.0 3,205 100.0 6,417 100.0

Note: *All percents are column percents.
‘Terminal field training status: FPL, reached Full Performance Level. Facility switch, transferred to
another Terminal facility before reaching FPL status. FSS transfer, transferred to FSS facility before
reaching FPL status. Fail, failed training and separated from ATC occupation.

29


Recovery of the FAA Air Traffic Control Workforce, 1981-1995

70%

60%

50%

40%

30%

20%

10%

0%

Disagree
EiNeither
Agree

1986

Figure 5-1: Job satisfaction (based on a single item) perceptions of air traffic control specialists

80%

70%

50%

40%

30%

20%

10%

Disagree
Neither
Agree

1984

Figure 5-2: Satisfaction perceptions of air traffic control specialists

1988

38



Recovery of the FAA Air Traffic Control Workforce, 1981-1992

level controllers. They then attended the FAA Academy
screening program. In addition, the FAA moved people
internally from other occupations into controller jobs.
Programs were established that allowed lower-graded
employees to take the OPM written ATCS aptitude test
battery, and bid for entry-level positions in air traffic
control. Thus, for example, a secretary or clerical worker
could compete for an entry level training position as a
controller. The people hired under this program usually
started as controllers below the normal GS-7 entry
grade. They received additional training before they
reached GS-7. The minimum passing scores on the
OPM test still applied to them, and they were required
to meet the same medical and background investigation
requirements as other applicants. Of course, they also
had to pass the FAA Academy screening program upon
reaching the GS-7 grade.

Other alternative hiring methods centered around
former military controllers. The FAA hired many former
military controllers through the basic civil service hiring
process, described at length in the first part of this paper.
It was also not uncommon for the FAA to hire military
controllers through alternative hiring procedures. For
example, Congress established special, expedited hiring
mechanisms for Vietnam-era veterans, and for people
who had recently left military service. The FAA still
required the test, medical, and background investiga-
tions when using this approach, but it was, nevertheless,
somewhat easier to hire people by this method. Not
uncommonly, former military controllers might be
hired at one step above the normal entry grade, based on
their military controller experience. These individuals
could bypass the FAA Academy screening program and
go directly into training at an FAA air traffic control
facility. In 1988, the military services reduced the size of
their forces, The FAA took advantage of the opportu-
nity to hire military controllers (Manning & Aul, 1992).
This program was conducted under very different pro-
cedures as authorized by OPM. The controllers were
hired at the GS-9 grade, one grade above the normal
entry grade. Above the GS-7 grade, the OPM test was
optional. In this case, the agency chose not to use it.
Instead, subject matter experts (personnel experts and
controllers) rated the candidates’ experience, education,
and training. This was a daunting task, as there were
over 1,400 eligible applicants. A standard rating guide,
containing six factors, was used. All selectees had to
meet the medical requirements and be cleared through
a background investigation. In addition, FAA air traffic
control staff contacted the selectees’ former military

facilities to verify their work experience and obtain
references or recommendations. About 600 military
controllers were hired in this way. They were placed
directly into facility training, bypassing the FAA Acad-
emy screening program (Manning & Aul).

Current Trends in Controller Selection

On February 21, 1992, the FAA closed the process of
hiring from the general population. No new applica-
tions to take the OPM test were processed, except for
certain veterans who had a statutory right to take the
examination. On August 12, 1993, President Clinton
again permitted the former controllers who had left the
agency in 1981 to be considered for employment with
the FAA. The FAA accepted applications from the
former controllers from September 1, 1993 through
October 15, 1993. Over 5,000 applications were received
in that short period. From 1992 until now, it has not been
necessary to hire significant numbers of people. Between.
1992and 1997, the FAA hired approximately 100 control-
lers peryear. This has been due to stable workforce with very
low attrition and retirement rates (General Accounting
Office, 1997), use of contractors to operate less busy non-
radar towers, and a slower than anticipated growth in
aviation in the early part of the decade.

However, the FAA may increase its hiring in the near
future but under very different conditions. The reason
for the hiring is primarily thar attrition is anticipated to
increase in the next five years, as the post-strike genera-
tion of controllers ages. The FAA is now starting to hire,
to ensure that replacement staff are trained and ready
when needed. The FAA estimates that about 500 new
controllers per year will be needed over the next several
years. The General Accounting Office (GAO) has re-
cently produced slightly lower estimates (GAO, 1997);
but both organizations point to the need for future
hiring (Schroeder, Broach, & Farmer, this volume).
Approximately 50 cooperative education students also
began work study programs with the agency in 1997.
These individuals will come to a different FAA. In April
of 1996, the FAA was legally exempted from most of the
civil service laws. The agency now has the authority to
develop new and innovative personnel systems and, for
many months, has been incorporating new procedures
based on studies of the best practices in other organiza-
tions. Based on those best practices and lessons learned
in the recovery of the controller workforce between
1981 and 1992, the FAA is developing and implement-
ing innovative processes to select and train the next
generation of air traffic controllers.


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

2,500
1,750 Joc

2,

48

14,569) as of April 1996

Figure 6-1: Age distribution of controller workforce (


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

the job performance of older workers, particularly in
safety critical occupations. However, Cobb and asso-
ciates (Cobb, 1967; Cobb, Nelson, & Matthews,
1973) have demonstrated that combined ratings (su-
pervisors, crew chiefs, and coworkers) of controller
performance are lower for older than for younger
controllers. This is consistent with data gathered
earlier at the en route centers. There are several
factors (motivation, differences in aptitudes and abili-
ties, and rating bias) that may have contributed to
these differences, other than physiological aging.
While we do not understand the full implications of
the age-related changes on the performance of air
traffic controllers, careful attention should be paid to
what appears to be age-related performance changes
as the overall age of the controller workforce increases
during the next 15 years.

Training Implications

Based on the projected increase in retirement eli-
gible personnel and the lead-time of around two to
three years for individuals to complete training, the
FAA is in the process of developing plans to hire
controllers over the next two decades. Facilities will
thus have expanding training requirements over the
next 20+ years to equip new controllers to handle
traffic in the future, as older controllers retire. How-
ever, this is not the only training burden faced by
facility managers and the Air Traffic Service. Within
the next 20 years, enhanced hardware/software ATC
work stations, decision aids, and new policies and
procedures will be introduced into the NAS to im-
prove capacity and to increase safety and efficiency.
By the turn of the century we will observe the effects
of the introduction of data link, the ARTCC Display
System Replacement (DSR), Standard Tower Auto-
mation Replacement System (STARS), Global Navi-
gation Satellite System (GNSS) technologies, and
airborne warning technologies. During the first de-
cade of the 21st century, new decision support sys-
tems will be introduced. Emphasis on integrating
“free flight” concepts into the NAS may result in a
number of significant changes in how air traffic is
handled. Under some proposals there may be a shift
in responsibility for separation of aircraft between
flight crew members and controllers, as a part of “free
flight.” No matter which scenario is finally adopted,
the introduction of these new systems and procedures
into the NAS over the next two decades will generate
continuing requirements to train ATCSs to operate

46

these new systems. Since these changes will largely
coincide with the influx of new controllers, it will
present a significant logistical effort with respect to
retraining the existing cadre of full performance level
controllers, while training new controllers at the
same time. Concerns about the availability of suffi-
cient staff within the facility to maintain normal
operations and still provide on-the-job training are
likely to be of special concern at facilities that have
been identified as “hard-to-staff,” where staffing lev-
els have remained below projected levels. Thus, there
will be a general need for facility-specific plans to
address the training requirements associated with the
hiring of new controllers to replace retirees and the
retraining of the existing workforce to function on
the more automated systems of the future.

While the cognitive changes in performance capa-
bilities may not posea significant problem for the safe
and expeditious flow of traffic under typical condi-
tions, it may pose additional burdens as older ATCSs
are required to transition to new technologies and
new procedures. Evidence suggests that older em-
ployees experience greater difficulty acquiring new
skills than do younger employees (Charness &
Bosman, 1990). In the use of computer technology,
there is research that suggests that older adults re-
quire longer training time and, in general, do not
obtain the same level of skill as younger individuals
(Czaja, 1996; Sharit & Czaja, 1994). The difficulty
of this transition is, of course, dependent on the
extent to which the job changes and the nature of
those changes. At this point, we do not know enough
about the proposed future systems to determine the
extent to which age-related factors should be consid-
ered in the development of the new procedures and
training programs. Human factors assessments will
be needed to determine the appropriateness of training
approaches for transitioning to the new technologies.

Selecting Controllers for Future Systems

Presentations made earlier in this session have
illustrated efforts associated with the recruitment,
selection, and training of new ATCSs. Research is
currently ongoing with respect to developing and
validating a 3" generation computer-administered
test battery to select controllers for the near term
(1998 - 2005). But what are the cognitive and per-
sonal characteristics that may be needed by control-
lers operating future systems? The new technologies
will involve greater automation and the availability


REFERENCES

Aerospace Sciences, Inc. (1991). Air traffic control spe-
cialist pre-training screen preliminary validation:
Final report. (Final report delivered under con-
tract DTFA01-90-Y-01034). Washington, DC:
Federal Aviation Administration Office of the
Deputy Administrator.

Air Traffic Services. (1996). ATS FY96 Business Plan.
Washington, DC: Federal Aviation Office of the
Associate Administrator for Air Traffic Services.

American Educational Research Association, American
Psychological Association, & National Council
on Measurement in Education. (1985). Standards
for educational and psychological testing. (5th ed.).
Washington, DC: American Psychological Asso-
ciation.

Associate Administrator for Human Resources Man-
agement. (1990, August). Report on the FAA em-
ployment manager's conference, June 25-29, 1990,
Arlington, VA. Washington, DC: Federal Avia-
tion Administration Staffing Policy Division.

Aul, J.C, (1991). Employing air traffic controllers. In
H. Wing & C.A. Manning (Eds.), Selection of air
traffic controllers: Complexity, requirements, and
public interest (pp. 7-12). (DOT/FAA/AM-91/
9). Washington, DC: Federal Aviation Adminis-
tration Office of Aviation Medicine. (NTIS No.
ADA238267)

Billings, C.E. (1997). Aviation automation: The search
for a human-centered approach. Mahwah NJ:
Lawrence Erlbaum Associates.

Boone, J. (1984). The FAA air traffic controller train-
ing program, with emphasis on assessment of
student performance. In S.B. Sells, J.T. Dailey, &
E.W. Pickrel, (Eds.), Selection of air traffic control-
ers (pp. 155-88). (DOT/FAA/AM-84/2). Wash-
ington, DC: Federal Aviation Administration,
Office of Aviation Medicine. (NTIS No.
ADA147765).

Bowers, D.G. (1983). What would make 11,500 people
quit their jobs. Organizational Dynamics, 11, 5-19.
Broach, D. (1997). Designing selection tests for the future
National Airspace System architecture. (DOT/FAA/

AM-97/17). Washington DC: Federal Aviation
Administration, Office of Aviation Medicine.

53

References

Broach, D., & Brecht-Clark, J. (1993). Validation of
the Federal Aviation Administration air traffic
control specialist Pre-Training Screen. Air Traffic
Control Quarterly, 1, 115-33.

Broach, D., Farmer, W.L., & Young, W.C. (1997).
Differential prediction of FAA Academy performance
on the basis of race and written Air Traffic Control
Specialist aptitude test scores. Manuscript submit-
ted for publication.

Broach, D. & Manning, C.A. (1994). Validity of the air
traffic control specialist non-radar screen as a predic-
tor of performance in radar-based air traffic control
training. (DOT/FAA/AM-94/9). Washington,
DC: Federal Aviation Administration, Office of
Aviation Medicine. (NTIS No. ADA279745).

Broach, D., & Manning, C.A. (1997). Review of air
traffic controller selection: An international perspec-
tive. (DOT/FAA/AM-97/15). Washington, DC:
Federal Aviation Administration, Office of Avia-
tion Medicine.

Broach, D., Young, W.C., & Farmer, W.L. (1994).
Effects of practice and coaching on ATCS selection
test validity. Paper presented at the 40" Annual
Meeting of the Southwestern Psychological Asso-
ciation.

Cattell, R.B., Eber, H.W., & Tatsuoka, M.M. (1970).
Handbook for the Sixteen Personality Factor Ques-
tionnaire (16PF). Champaign, IL: Institute for
Personality and Ability Testing.

Collins, W., Boone, J., & VanDeventer, A. (1984). The
selection of air traffic control specialists: Contri-
butions by the Civil Aeromedical Institute. In
S.B. Sells, J.T. Dailey, & E.W. Pickrel, (Eds.),
Selection of air traffic controllers (pp. 79-112).
(DOT/FAA/AM-84/2). Washington, DC: Fed-
eral Aviation Administration, Office of Aviation
Medicine. (NTIS No. ADA147765).

Collins, W., Manning, C., & Taylor, D. (1984). A
comparison of pre-strike and post-strike ATCS
Trainees: Biographic factors associated with Acad-
emy training success. In A. VanDeventer,
W.Collins, C.Manning, D.Taylor, & N.Baxter
(Eds.), Studies of poststrike air traffic control spe-
cialist trainees: I. Age, biographical factors, and
selection test performance related to Academy train-
ing success (pp. 7-14). (DOT/FAA/AM-84-6).
Washington, DC: Federal Aviation Administra-
tion, Office of Aviation Medicine. (NTIS No.
ADA147892).


Trites, D.K. & Cobb, B.B. (1964b). Problems in air
traffic management: IV. Comparison of pre-em-
ployment, job-related experience with aptitude
tests as predictors of training and job performance
of air traffic control specialists. Aerospace Medi-
cine, 35, 428-36,

Tsang, P.S. & Voss, D.T. (1996). Boundaries of cogni-
tive performance as a function of age and flight
experience. International Journal of Aviation Psy-

chology, 6, 359-77.

U.S. House of Representatives Committee on Govern-
ment Operations. (January, 1976). Twelfth Re-
port: Selection and training of FAA air traffic con-
trollers. Washington, DC: U.S. Government Print-
ing Office.

Office of Personnel Management (Friday, March

10, 1989). Personnel management demonstra-
tion project; Department of Transportation/Fed-
eral Aviation Administration demonstration
project. Federal Register, 54, 10199-207.

VanDeventer, A.D. (1981, May). Field training perfor-
mance of FAA Academy air traffic control graduates.
Paper presented at the annual Scientific Meeting
of the Aerospace Medical Association, San Anto-
nio, TX.

VanDeventer, A.D. & Baxter, N.E. (1984). Age and
performance in air traffic control specialist train-
ing. In A. D, VanDeventer, W.E. Collins, C.A.
Manning, D.K. Taylor, & N.E. Baxter (Eds.),
Studies of poststrike air traffic control specialist train-
ees: I. Age, biographic factors, and selection test
performance related to Academy training success
(pp. 1-6). (DOT/FAA/AM-84/6). Washington
DC: Federal Aviation Administration, Office of
Aviation Medicine. (NTIS No. ADA147892)

US.

57

References

VanDeventer, A.D., Collins, W.E., Manning, C.A.,
Taylor, D.K. & Baxter, N.E. (1984). Studies of
poststrike air traffic control specialist trainees: I. Age,
biographic factors, and selection test performance
related to Academy training success. (DOT/FAA/
AM-84/6). Washington, DC: Federal Aviation
Administration, Office of Aviation Medicine.
(NTIS No. ADA147892)

Weithoner, C.E. (1985). 1984 employee attitude survey
follow-up action. (DOT/FAA Memorandum).
Washington, DC: Federal Aviation Administra-
tion, Office of Human Resources Management.

Wickens, C.D., Mavor, A.S., & McGee, J.P. (Eds.).
(1997). Flight to the future: Human factors in air
traffic control. Washington DC: National Acad-
emy Press.

Wigdor, A.K., & Sackett, P.R. (1993). Employment
testing and public policy: The case of General
Aptitude Test Battery. In H. Schuler, J.L. Farr, &
M. Smith (Eds.), Personnel selection and assess-
ment: Individual and organizational perspectives
(pp. 183-204). Hillsdale, NJ: Lawrence Erlbaum
Associates.

YTU.S. GOVERMENT PRINTING OFFICE: 1998 - 660-088/80007


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

Projections of the staffing requirements and asso-
ciated air traffic levels are presented in Figure 6-2. A
gradual increase in ATCS staffing over the next two
decades is designed, in part, to keep pace with the
expected increase in aviation activity (Air Traffic
Services, 1996). To determine the effects of the
continued aging of the current group of controllers
on the need to hire and train new personnel, we
developed a research model that allows us to forecast
the number of ATCSs eligible for retirement over the
next decade. Recognizing that there are many factors
that can be incorporated into a model for projecting
the hiring and training flow of the future, ours is
sensitive both to attrition associated with deaths due
to accidents and other health conditions (this is not
age corrected so it represents a conservative estimate
of losses attributed to these events) and the availabil-
ity of retirement eligibility at age 56. While different
models may produce slightly different predictions,
the general shape of the distributions remain rela-
tively the same. Using this research model, we pro-
jected the number of retirement-eligible controllers
from 2001 through 2020, as illustrated in Figure 6-
3. The greatest potential loss due to retirements of
ATCSs will occur around 2015 to 2017. By 2016,
over 50% of today’s ATC workforce will have reached
the age at which they are eligible for retirement.

The General Accounting Office (GAO) was criti-
cal in a recent report (GAO, 1997) of the FAA’s
retirement projections that were based on the as-
sumption that, in any year, around 20% of those
eligible would retire. The GAO developed projec-
tions that were based on recent data that indicated
that the average controller retires at 56 years of age
with 31 years of service. When compared with the
FAA projections, GAO predicted smaller numbers of
retirees in 1997 through 2001. GAO based their
calculations on the probabilities of retirement of
controllers in various age cohorts during fiscal years
1992-1996. The average yearly retirements would
increase from 198 in 1997 to 299 in 2003. In 2009,
more than 600 retirements would be expected. That
average number of yearly retirements is likely to
remain at that level or higher for several years, given
the large numbers of ATCSs who will reach age 56
from year 2009 through 2017.

44

Controller Aging and Performance

Another consideration associated with the overall
aging of the workforce during the first two decades of
the 21st century concerns the potential effects of
aging on cognitive functioning and the consequences
of those changes for on-the-job performance. There
are indications in the scientific literature that during
the middle decades of life most individuals experi-
ence some decline in the speed of their cognitive
functioning. In fact, Fozard and Nutall (1971), in
their study of age-related changes in scores on the
General Aptitude Test Battery (GATB), indicated
that the average 60 year-old individual would be able
to meet the occupational aptitude patterns for only
17 of 36 occupations. Changes are more frequently
observed in the fluid or process aspects of cognition
(speed of processing information, memory, and mak-
ing decisions). Some scientists feel that overall per-
formance on those tasks may start to decline at ages
as early as in the 20s (Salthouse, 1990). However,
what is often referred to as the crystallized measures
of intelligence (vocabulary and general information)
may improve with increasing age until the late 60s.
When viewed in the context of most jobs, Schaie
(1988) has argued that the reported age differences
are generally too small to have meaningful conse-
quences. However, little has been said about the
potential implications of these changes for personnel
in safety-related positions where speed of informa-
tion processing, memory, and decision-making are of
critical importance.

Nearly all of the scientific information available
refers to the effects of age-related declines in perfor-
mance on somewhat novel laboratory tasks, com-
pared with more familiar tasks or job performance. A
series of studies by Trites, Cobb, and their associates
(e.g., Cobb, Lay, & Bourdet, 1971; Trites, 1963;
Trites & Cobb, 1964a; 1964b) consistently demon-
strated an inverse relationship between age of entry
and performance of ATCS trainees at the FAA Acad-
emy and in field training. Those studies led to Con-
gressional action to restrict the age of entry into the
FAA Academy training to age 30.

A subsequent Civil Aeromedical Institute (CAMI)
study (VanDeventer & Baxter, 1984) of 8,573 FAA
Academy trainees where the age of entry was more


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

Table 2-6: Descriptive statistics and correlations for 1986-92 Non-radar Screen graduates

assigned to En Route OJT
Sample’ Population”* Correlations’
Variable N M SD N M SD (1) (2) (3)
(1) RATING 3,969 93.79 5.06 106,201 76.33 14.67 493 -.129
(2) SCREEN 4,449 80.26 5.89 12,756 72.26 11.80 192*** -.247
(3) STATUS 4,732 77.3% FPL -.014 -.126*"*
22.7% NOT FPL
Notes: STATUS outcomes for first assigned field facility only as of March 1997 ** p< .01, ***p <.001

“RATING based on CAMI records for examinees tested 1982-85
“SCREEN based on Non-radar Screen entrants who did not withdraw, 1982-1985
“Uncorrected correlations in lower left half, corrected in upper right half.

Table 2-7: Civil service RATING validity in predicting FIELD TRAINING outcome by sample,
based on corrected correlations

Sample
Period FAA Academy Field OJT R B-RATING B-SCREEN
1982-85 Terminal Screen Terminal 304*** -.025 -.291***
1982-85 En Route Screen En route 363*** -.107*** -.297***
1986-92 Non-radar Screen Terminal .142*** -.041 -.118***
1986-92 Non-radar Screen En route .247*** -.010 -.242***

*p <.05, *p< .01, ***p < .00

Table 2-8: Adverse impact analysis by gender based on OPM eligibility codes

Sex
OPM Eligibility Males Females Row totals
Eligible 69,056 14,564 84,070
(58.1%) (42.0%)
Failed test 49,902 20,077 69,979
(41.9%) (58.0%)
Column totals 118,985 34,641 53,599

Notes: Percents are column percentages

16


Recovery of the FAA Air Traffic Control Workforce, 1981-1995

based on annual performance reviews that may or
may not have been related to actual day-to-day per-
formance. The organization of work, in terms of the
procedures and processes involved in completing
assigned duties, was perceived to be problematic by
controllers. Finally, managerial and supervisory di-
rection was rated poorly by the ATCS workforce.
Problems with supervisors were believed to be prima-
rily due to the autocratic style of supervisors selected
from the ranks of controllers.

The perceptions of striking and non-striking con-
trollers were compared in the Jones Report. The
results indicated that controllers who struck, and
were subsequently fired, and controllers who re-
mained had a similar pattern of organizational per-
ceptions, although the striking controllers generally
indicated lower levels of most perceptions. In addi-
tion, the data indicated that the strike occurred, in
part, due to “...an organization climate that they
[ATCSs] experienced as uncaring, unconcerned for
its people, uncommunicative, and unreceptive” (Bow-
ers, 1983, p. 16). In response to these conclusions the
FAA developed several national human resource ini-
tiatives that targeted the morale and supervision
problems identified in the Jones Report. These initia-
tives included survey-feedback programs to respond
to employee concerns, improvement of the selection
and human resources training of supervisors, and
changing pay systems.

One of the primary FAA responses to the strike was
the implementation of an agency-wide biennial em-
ployee survey. The biennial survey was created to
provide management with a picture of employee
perceptions, and to provide a means of tracking those
perceptions over time. In addition, feedback from the
survey provided management with information re-
garding problem areas to be addressed. Following the
first administration of the survey in 1984, interviews
were conducted in three regions to further clarify why
people responded they way they did. The interviews
indicated the areas of primary concern were related to
human resource management practices. In addition,
regions were expected to formulate action plans to
address human resource management problems within
the region. The regional plans were then combined
into a national plan (Weithoner, 1985). Human
resources specialists were hired in each region to
further support efforts to initiate plans and programs
to address these issues.

34

After the 1986 administration of the agency-wide
employee survey, the feedback and action planning
process that had emerged from an earlier administra-
tion was augmented and formalized in the Survey
Feedback Action (SFA) program. The SFA relied on
two surveys: (1) The biennial national survey de-
scribed above that focused on national, agency wide
issues, and (2) regional surveys focused on more
specific regional issues. The regional surveys assessed
employee perceptions of supervisors and manage-
ment. Based on the results of the regional surveys,
regional plans were developed to improve training,
communication, and supervision. The effectiveness
or impact of these regional action plans were to be
evaluated with the next round of surveys.

Another problem for the agency that resulted from
the strike was under-staffing of ATCSs. Understaffing
was an immediate problem following the strike be-
cause it takes as long as 3 years after hiring to train a
controller to Full Performance Level (FPL;
VanDeventer, Collins, Manning, Taylor, & Baxter,
1984). Understaffing of ATCSs was a continuing
problem for the FAA in some facilities (General
Accounting Office, 1986, 1989). To enhance re-
cruitment and retention of employees, the DOT/
FAA Pay Demonstration Project (U. S. Office of
Personnel Management, 1989) was implemented.
This project provided retention bonuses to employ-
ees in safety-related occupations such as ATCS, elec-
tronics maintenance technicians, and aviation safety
inspectors at difficult to staff facilities.

Another change that occurred due to the strike was
the influx of a large number of new employees. After
the termination of the striking controllers, replace-
ments had to be hired, trained, and placed on the
boards. These new hires differed from the replaced
controllers in terms of previous experience and apti-
tude (VanDeventer, Collins, Manning, Taylor, &
Baxter, 1984). In addition, there were some differ-
ences in the training of pre-and post-strike control-
lers. In order to rapidly replace the controllers, the
agency hired a substantial number of replacements
“off the streets” who were required to complete the
FAA Academy screen, followed by continued train-
ing at the FAA Academy and in the field. A smaller
number of former military controllers were hired at
higher grade levels and placed directly into field
facilities for on-the-job training (Manning, Kegg, 8
Collins, 1989; Manning & Aul, 1992). The new


TABLE OF CONTENTS

Introduction...
Dana Broach, Ph. D. FAA Civil ‘Aeromedical Institute

Chapter 1: Employing Air Traffic Controllers, 1981-1992 ......sccssosssssssscssssseressessssssesssesesuneesssssssnsneeeeeesene 3
Jay C. Aul, FAA Office of Personnel
Chapter 2: Air Traffic Control Specialist Aptitude Testing, 1981-1992 ...........ccssssssesssssesrsecseessneeseeeeesene 7

Dana Broach, Ph.D., FAA Civil Aeromedical Institute

Chapter 3: FAA Academy Air Traffic Control Specialist Screening Programs and Strike Recovery........ 17
Pamela Della Rocco, Ph.D., FAA Civil Aeromedical Institute

Chapter 4: Air Traffic Control Specialist Field Training Programs, 1981-1992 ........ccsccssssesseeeessnteseeene 23
Carol A. Manning, Ph.D., FAA Civil Aeromedical Institute

Chapter 5: Organizational Climate Perceptions of the Air Traffic Control Specialist (ATCS)
Workforce, 1981-1992 ..
Richard C. Thompson, Ph. D. FAA Ci

Chapter 6: Current FAA Controller Workforce Demographics, Future Requirements,

| Aeromedical Institute

and Research Questions ..........cccssssssssessssssssessscssesssccaressesueeaesuesssensesueencenecuesenesuessessecseeseesesaueesusseessetseeene 43

David J. Schroeder, Ph.D., Dana Broach, Ph.D., and William L. Farmer, M.

FAA Civil Aeromedical Institute

LIST OF FIGURES

Figure Page
2-1: Example Multiplex Controller Aptitude Test (MCAT) item .......ccscsssssesecscsssssssescesesssnnnesssecssnnneeeeeeeraes 12
2-2: Example Abstract Reasoning (ABSR) test item ....s..ssssssseccssesssesssresssessessueessnessneessuecnnscsaseesaresseeesnecesnee 12
2-3: Distribution of RATING for examinees and selectees.......1....ssssssssssssssssssessssssssereeeeeeessnnesseesenneeeeees 13
2-4: Regression of test score (TMC) on FAA Academy score (SCREEN) by gender ........sssssssesssssssseeeeees 13
2-5: Regression of test score (TMC) on FAA Academy score (SCREEN) by race ..s.ssssccssssesssseceesssneeeseeseee 14

3-1: Percent achieving en route FPL field status by FAA Academy score

3-2: Percent achieving terminal FPL field status by FAA Academy score
5-1: Job satisfaction (based on a single item) perceptions of air traffic control specialists ......::..:ssseeee 38

5-2: Satisfaction perceptions of air traffic control specialists

5-3: Management concern for employee’s perceptions of air traffic control specialists .........s.ssseceseeeees 39
5-4: Pay and benefits perceptions of air traffic control specialists .......ssssssssssssssssesscssssssneeescessenneteeseessnnneeses 39
5-5: Organization of work perceptions of air traffic control specialists .....1..ssssccsssssssssssesssseescssseescssseeseesses 40
5-6: Management direction perceptions of air traffic control specialists .........cecssseesssescessseseesseeseesssesssseee 40

6-1: Age distribution of controller workforce (N=14,569) as of April 1996 w....cccecssstecssseessssnesesnneesssecsase 48

iii


Chapter 4: Air Traffic Control Specialist Field Training Programs, 1981-1992

Table 4-1: Demographics of FAA Academy program graduates

FAA Academy Program

En Route Screen Terminal Screen Non-radar Screen Group Total
1981-85 1981-85 1986-92
N % N % N % N %
Gender
Male 3,928 88.0 2,814 83.0 6,683 82.0 13,425 83.9
Female 535 12.0 576 17.0 1,465 18.0 2,576 16.1
Total 4,463 100.0 3,390 100.0 8,148 100.0 16,001 100.0
Minority Status?
Amer. Indian 19 A 17 a) 47 6 83 5
Asian-Pac. 28 6 32 1.0 98 1.2 158 1.0
African-Amer. 81 1.8 119 35 339 43 539 3.4
Hispanic 86 2.0 73 2.2 254 3.2 413 2.6
Non-Minority 4,194 95.1 3,117 92.8 7,182 90.7 14,493 92.4
Total 4,408 100.0 3,358 100.0 7,920 100.0 15,686 100.0
Entry Type®
Competitive 4,046 92.7 2,772 82.4 6,635 82.7 13,453 85.4
Airway Science 8 2 4 a 75 9 87 6
Pre-Devel. 57 1.3 113 3.4 155 1.9 325 27
Coop. Ed. 34 8 49 1.5 422 5.3 505 3.2
ATA 43 1.0 94 2.8 564 7.0 701 AS
VRA 34 8 41 1.2 21 3 96 6
Noncompetitive 134 3.1 237 7.0 86 1.1 457 2.9
Former ATC 10 2 54 1.6 61 8 125 8
Former GS-9 2 0 2 0
Total 4,366 100.0 3,364 100.0 8,021 100.0 15,751 100.0
Education"
No coll. degree 2,346 53.3 1,648 51.6 5,230 64.9 9,224 59.0
College degree 2,053 46.7 1,544 48.4 2,824 35.1 6,421 41.0
Total 4,399 100.0 3,192 100.0 8,054 100.0 15,645 100.0

Notes: *All percents are for columns within grouping.
“Minority status: Amer. Indian = American Indian/Alaskan.; Asian-Pac. = Asian-Pacific Islander.
“Entry Type: Airway Science = graduate of accredited Airway Science program. Pre-Devel. =
participated in FAA Pre-Developmental Program; Coop. Ed. = Cooperative Education Program; ATA
= former Air Traffic Assistant (GS-2154 or GS-2102); VRA = hired under Veterans Readjustment
Act; Former ATC = previously graduated from FAA Academy and failed field training; Former GS-9
= previously hired under GS-9 military hire program and failed field training.
“Education: No coll. degree includes high school graduates and candidates with some college hours
but no degree. College degree includes those with a bachelor’s or higher degree.

27


N Persons

Chapter 2: Air Traffic Control Specialist Aptitude Testing, 1981-1992

35,000

WE Applicants
30,000 4 j—W-Selectees foc oot t ttc tect rcrcc rrr Bp errr ccc

25,000 4
20,000 4
15,000 4
10,000 4

5,000 +4

04
20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110
RATING

Figure 2-3: Distribution of RATING for examinees and selectees

70 15 80 85 90 95 100
Test score (TMC)

Figure 2-4: Regression of test score (TMC) on FAA Academy score (SCREEN) by gender


Chapter 3: FAA Academy Air Traffic Control Specialist Screening Programs & Strike Recovery

4,997. The structure of the screening programs was
not changed to accommodate the strike recovery
(Boone, 1984).

Following the strike, the pass rates in both the En
Route and Terminal Screen programs declined (Man-
ning, Kegg, & Collins, 1989). Between August 1981
and September 1985, entrants to the En Route Screen
program had an average pass rate of 52%, compared
with the pre-strike pass rate of 67%. En Route Screen
failures represented 36% and withdrawals represented
12% of the total 8,536 entrants. In the Terminal Screen
program, the post-strike pass rate was 68%, compared
with the pre-strike rate of 74%. Terminal Screen pro-
gram entrant failures represented 26%. The Terminal
Screen program withdrawal rate was 6% of total entrants.

Examination of the demographic data from pre- and
post-strike FAA Academy entrants (Collins, Manning
& Taylor, 1984) revealed a shift in the proportion of
entrants reporting prior ATC experience. Prior to the
strike, 70% of FAA Academy entrants reported either
aviation or air traffic control experience prior to entry.
After the strike, nearly two-thirds of the entrants had no
prior aviation-related experience. FAA Academy pass
rates were higher for the group of entrants with prior
ATC experience than entrants with no prior experience
(Collins, Manning, & Taylor, 1984). The decline in the
percentage of entrants with previous ATC experience,
particularly prior military controllers, may have affected
the pass rates. Because the initial qualification courses
were designed for individuals with no prior experience,
this shift in demographics was manageable. Table 3-1
presents demographics for each of the time periods
examined in this paper.

The “Non-radar Screen,” 1986-1992

Changes to the program during the first decade
involved such modifications as alterations in scoring
and the number of graded problems (Manning, Kegg, &
Collins, 1989). In 1985, however, a major change was
made to consolidate the En Route and Terminal Screen
programs into a single FAA Academy program, the
“Non-radar Screen.” Analyses of the longitudinal data-
bases had begun to point to areas in which the screening
effectiveness and cost benefits could be enhanced. Be-
cause the En Route Screen program was found to better
correlate with measures of field success such as
supervisor's ratings (VanDeventer, 1981), than the Ter-
minal program, the En Route Screen program was
selected as the basis for the Non-radar Screen (Manning,
Kegg, & Collins, 1989). Some changes were made to the

weights assigned to the scores on various components to
achieve a projected 60% pass rate. The change to using
a single screening course meant that students were not
assigned to an option or facility until they completed the
Non-radar Screen and their FAA Academy scores could
be used to place graduates. It was anticipated that
assigning students with higher aptitude to more com-
plex facilities would result in further reductions in field
training attrition. In addition, the Non-radar Screen
was re-sequenced in Fiscal Year 1989 to be the first
course completed by FAA Academy students. It was
anticipated that by postponing the three-week, aca-
demic phase of training that originally preceded the
pass/fail Non-radar Screen, the training costs associated
with people who would fail or withdraw from the screen
would be avoided through the re-sequencing. The nine-
week Non-radar Screen was implemented in October
1985 (FY 1986) and continued through the January
1992 entering class. Non-radar Screen pass rates were
56.6% for 14,392 entrants. Failures represented 32.2%,
and 11.2% withdrew.

Validity Evidence

Over the years, several studies examined various
measures to assess the predictive validity of the FAA
Academy screening programs. These included attrition
rates, supervisor ratings, and field training progress.
VanDeventer (1981) reported a correlation of .56 (ad-
justed for explicit restriction in range on FAA Academy
score) between the FAA Academy final composite score
and the field supervisor’s rating for the en route option.
Manning, Della Rocco, and Bryant (1989) used field-
training status as well as field instructor rating as crite-
rion measures. They reported adjusted correlations
between the FAA Academy composite score and in-
structor ratings of .46 and .30 for the En Route and
Terminal Screen programs, respectively. In the first
report of data on Non-radar Screen graduates, Della
Rocco, Manning, and Wing (1990) reported an ad-
justed correlation of .44 between FAA Academy com-
posite scoreand field training status of graduatesassigned
to the En Route option. In 1994, Broach and Manning
reported evidence that the Non-radar Screen was a valid
predictor of FAA Academy radar training performance.

To evaluate the final status of the Non-radar Screen
program, correlations between the FAA Academy com-
posite score and field training status from the CAMI
tracking database for FAA Academy graduates who
entered as competitive hires were examined. Field status
was a variable that identified ATCSs in their first field


Chapter 2: Air Traffic Control Specialist Aptitude Testing, 1981-1992

Table 2-3: Descriptive statistics and correlations for 1982-85 Terminal Screen graduates
assigned to Terminal OJT

Sample* PopulationTM*® Correlations*
Variable N M SD N M SD (1) (2) (3)
(1) RATING 2,814 92.31 5.72 99,887 74,93 14.24 493 -.168
(2) SCREEN 3,174 79.45 5.48 4,686 73.82 11.32 222*** -.303
(3) STATUS 3,174 88.0% FPL -.037 -.146***
12.0% | NOT FPL
Notes: ‘STATUS outcomes for first assigned field facility only as of March 1997 ** p< .01, **p <.001

“RATING based on CAMI records for examinees tested 1982-85
“SCREEN based on Terminal Screen entrants who did not withdraw, 1982-1985
“Uncorrected correlations in lower left half, corrected in upper right half.

Table 2-4: Descriptive statistics and correlations for 1982-85 En Route Screen graduates
assigned to En Route OJT

Sample* Population”* Correlations*
Variable N M SD N M sD (1) (2) (3)
(1) RATING 4,021 92.56 5.37 99,887 74.93 14,24 .502 -.256
(2) SCREEN 4,276 82.02 7.17 4,686 73.82 11.32 .214%** -.351
(3) STATUS 4,276 66.1% FPL ~.047** — -.231**

33.9% NOT FPL

Notes: “STATUS outcomes for first assigned field facility only as of March 1997 ** p< ..01, ***p <.001
*RATING based on CAMI records for examinees tested 1982-85
“SCREEN based on En route Screen entrants who did not withdraw, 1982-
1985
“Uncorrected correlations in lower left half, corrected in upper right half.

Table 2-5: Descriptive statistics and correlations for 1986-92 Non-radar Screen graduates
assigned to Terminal OJT

Sample* Population*® Correlations*
Variable N M SD N M SD (1) (2) (3)
(1) RATING 2,765 93.31 5.15 106,201 76.33 14.67 458 -.095
(2) SCREEN 3,145 78.28 6.07 12,756 72.26 11.80 .178*** -.137
(3) STATUS 3,298 95.0% FPL -.027 -.071***

5.0% NOT FPL

Notes: “STATUS outcomes for first assigned field facility only as of March 1997 ** p< .01, ***p < .001
*RATING based on CAMI records for examinees tested 1982-85
“SCREEN based on Non-radar Screen entrants who did not withdraw,
1982-1985
“Uncorrected correlations in lower left half, corrected in upper right half.



Recovery of the FAA Air Traffic Control Workforce, 1981-1992

Table 4-2: Coding of field training status at first facility by option

Option
Status En Route Terminal
1 FPL FPL
2 Developmental Developmental
3 Terminal transfer Facility switch
4 FSS transfer FSS transfer
5 Fail Fail

Note: FPL, reached Full Performance Level. Developmental, still in training
at first facility. Terminal transfer, transferred to Terminal facility before
reaching FPL status. Facility switch, transferred to another Terminal facility
before reaching FPL status. FSS transfer, transferred to FSS facility before
reaching FPL status. Fail, failed training and separated from ATC
occupation.

Table 4-3: En Route field training status by FAA Academy program

FAA Academy Program*
En route Screen Non-radar Screen
1981-85 1986-92 Group Total
Field Training Status” N % N % N %
FPL 2,758 65.7 3,405 77.2 6,163 71.6
Developmental 48 11 48 6
Terminal transfer 289 6.9 393 8.9 682 7.9
FSS transfer 424 10.1 115 2.6 539 6.3
Fail 728 17.3 450 10.2 1,178 13.7
Group Total 4,199 100.0 4,A11 100.0 8,610 100.0

Note: *All percents are column percents.

“En Route field training status: FPL, reached Full Performance Level. Devel., Developmental. Term.
transfer, transferred to Terminal facility before reaching FPL status. FSS transfer, transferred to FSS
facility before reaching FPL status. Fail, failed training and separated from ATC occupation.

28


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

cal fairness of the written ATCS aptitude test battery
was recently examined in a series of papers as the first
step toward assessing to what degree, if any, the
battery may have served as an “engine of exclusion”
(Seymour, 1988) of women and minorities from the
ATCS occupation (Young, Broach, & Farmer, 1996;
Broach, Farmer, & Young, 1997). Technical fairness
refers to the regression model of test bias for which
there is a reasonable professional consensus, as em-
bodied in the 1985 Standards for Educational and
Psychological Testing (American Educational Research
Association, American Psychological Association, &
National Council on Measurement in Education),
rather than a socially constructed standard regarding
test use (Sackett & Wilk, 1994; Gottfredson, 1994).
Technical fairness in the regression model, and under
the Uniform Guidelines on Employee Selection Proce-
dures (Equal Employment Opportunity Commis-
sion, 1978), encompasses two issues. First, the effect
on protected groups arising from use of a particular
cut score on the predictor must be evaluated. A
selection rate for any protected group that is less than
four-fifths (4/5 or 80%) of that of the majority group
will “... generally be regarded by the Federal enforce-
ment agencies as evidence of adverse impact” (29
CFR 1607.4.D). Second, where the use of a selection
procedure results in adverse impact, the Uniform
Guidelines on Employee Selection Procedures require
that the test user evaluate the degree to which differ-
ential predictions of future job performance are made
from selection test scores by subgroup (29 CFR
1607.14.B.(8).(b)).

Gender

Previous research on written ATCS selection tests
suggested that mean score differences by gender were
insignificant (Rock, Dailey, Ozur, Boone, & Pickrel,
1984a, pp. 476) and that, overall, “the evidence for
adverse impact against women based, on this sample,
was marginal, at best” (Rock, Dailey, Ozur, Boone,
& Pickrel, 1984b, pp. 507). This conclusion was
based on results of a 1984 study in which 57% of men
(n= 3,835) passed the written test, compared to 45%
of women (7 = 1,473). The adverse impact ratio in
this case was 0.78, rather than the 0.80 required
under the “four-fifths rule of thumb.” Young, Broach
and Farmer (1996) analyzed archival records for
170,578 applicants. Based on test scores, OPM de-
termined the eligibility of applicants for employment
by the FAA. OPM codes indicating that an applicant

10

had either failed the test (‘IA’) or scored too low for
consideration (‘IS’) were categorized as test failures.
All other ineligibility codes were categorized as “other
ineligible,” and excluded from the analysis. OPM
codes indicating eligibility were categorized as “eli-
gible” for employment. The adverse impact analysis
compared the proportion of women considered eli-
gible with that of men. The analysis is presented in
Table 2-8. A significantly lower percentage of women
(42.0%) than men (58.1%) were coded by OPM as
eligible for employment as controllers, based on test
scores (Z = -3.58, p < .001). The ratio of female to
male selection rates was .72. Using the 4/Sths rule of
thumb of the Uniform Guidelines on Employee Selec-
tion Procedures, it appeared that the use of scores on
the written ATCS aptitude test battery to determine
eligibility for employment in the ATCS occupation
between 1981 and 1992 resulted in statistically signifi-
cant adverse impact on female applicants.

Given the finding that there appeared to be adverse
impact against women, the Uniform Guidelines on Em-
ployee Selection Procedures (29 CFR 1607.14.B.(8).(b))
and Standards for Educational and Psychological Test-
ing (Standard 1.20, p. 17) next required an investiga-
tion of the relationship between test scores and job
performance for evidence of differential prediction
by subgroup. The classical, regression-based model
of test bias was used as the analytic framework by
Young, Broach, and Farmer to evaluate the degree to
which the written ATCS test battery differentially
predicted performance in initial ATCS qualification
training at the FAA Academy. After correcting corre-
lations between test score, FAA Academy composite
score, and gender for explicit and implicit restric-
tions in range due to prior selection of the sample on
aptitude test score, the null hypothesis of a common
regression line for the genders was rejected, suggest-
ing the presence of some degree of test bias. Statisti-
cally significant differences by gender in regression
slopes and intercepts were found in the step-down
regression analysis, indicating the need for separate
regression equations for men and women. The regres-
sion line for men slightly over-predicted the perfor-
mance of women in the FAA Academy, as shown in
Figure 2-4. The practical consequence of the appar-
ent differential prediction was that women effectively
needed a higher test score than men to have an equal
likelihood of passing the initial qualification training
at the FAA Academy.


Federal Aviation Administration. (1993). FAA diversity
plan. Washington, DC: Office of the Administra-
tor for Human Resources Management Career
Systems Division.

Federal Aviation Administration (1996). People systems
in the FAA: A new look [Brochure]. Washington,
DC: Author.

Federal Aviation Administration. (1996). National air-
space system architecture version 2.0. Washington,
DC: Federal Aviation Administration, Office of
Systems Architecture and Program Evaluation.

Fozard, J.L. & Nutall, R.L. (1971). General aptitude
test battery scores for men differing in age and
socioeconomic status. Journal of Applied Psychol-
ogy, 55, 372-79.

General Accounting Office (1986). Aviation safety: Se-
rious problems concerning the air traffic control
work force. (GAO/RCED-86-121). Washington,
DC: Author.

General Accounting Office (1989). Aviation safety: Se-
rious problems continue to trouble the air traffic
control work force. (GAO/RCED-89-44). Wash-
ington, DC: Author.

General Accounting Office (1996). Aviation acquisi-
tion: A comprehensive strategy is needed for cultural
change at FAA. (GAO/RCED-96-159). Wash-
ington, DC: Author.

General Accounting Office. (1997). Aviation safety:
Opportunities exist for FAA to refine the controller
staffing process. (GAO/RCED-97-84). Washing-
ton, DC: Author.

Ghiselli, E., Campbell, J., 8¢ Zedeck, S. (1981). Mea-
surement theory for the behavioral sciences. San
Francisco: Freeman.

Gottfredson, L.S. (1994). The science and politics of
race-norming. American Psychologist, 49, 955-63.

Hartigan, J.A., & Wigdor, A.K. (1989). Fairness in
employment testing: Validity generalization, minor-
ity issues, and the General Aptitude Test Battery.
Washington, DC: National Academy Press.

Henry, J.H., Ramrass, M.E., Orlansky, J., Rowan,
T.C., String, J., & Reichenbach, R.E. (1975).
Training of U.S. air traffic controllers. (IDA Re-
port R-206). Arlington, VA: Institute for Defense
Analysis. (NTIS No. ADA006603)

55

References

Jones, L.J., Bowers, D.G., 8 Fuller, $.H. (1982). Man-
agement and employee relationships within the fed-
eral aviation administration (Vol. 1 & II). Wash-
ington, DC: Federal Aviation Administration.

Kuder, G.F., & Richardson, M.W. (1937). The theory
of the estimation of test reliability. Psychometrika,
2, 151-60.

Lewis, M.A. (1978). Objective assessment of prior air
traffic control related experience through the use
of the Occupational Knowledge Test. Aviation,
Space and Environmental Medicine, 49, 1155-59.

Lilienthal, M.G., & Pettyjohn, F.S. (1981). Multiplex
Controller Aptitude Test and Occupational Knowl-
edge Test: Selection tools for air traffic controllers.
(NAMRL Special Report 82-1). Pensacola, FL:
Naval Aerospace Medical Research Laboratory.
(NTIS No. ADA118803)

Manning, C.A. (1991). Procedures for selection of air
traffic control specialists. In H. Wing & C.A.
Manning (Eds.), Selection of air traffic controllers:
Complexity, requirements, and public interest (pp.
13-22). (DOT/FAA/AM-91/9). Washington,
DC: Federal Aviation Administration, Office of
Aviation Medicine. (NTIS No. ADA238267).

Manning, C.A. & Aul, J.C. (1992). Evaluation of an
Alternative Method for Hiring Air Traffic Control
Specialists with Prior Military Experience. (DOT!
FAA/AM-92/5). Washington, DC: Federal Avia-
tion Administration, Office of Aviation Medi-
cine. (NTIS No. ADA246587)

Manning, C.A., Della Rocco, P.S., & Bryant, K.D.
(1989). Prediction of success in FAA air traffic
control field training as a function of selection and
screening test performance. (DOT/FAA/AM-89/
6). Washington, DC: Federal Aviation Adminis-
tration, Office of Aviation Medicine. (NTIS No.
ADA209327).

Manning, C.A., Kegg, P.S., & Collins, W.E. (1989).
Selection and screening programs for air traffic
control. In R.S. Jensen (Ed.), Aviation Psychology
(pp. 321-41). Brookfield, MA: Gower Technical.

Milne, A.M., & Colmen, J.G. (1977). Selection of air
traffic controllers for Federal Aviation Administra-
tion. (Final report under FAA contract DOT-
FA70WA-2371). Washington, DC: Education
and Public Affairs, Inc.

Morrow, D., Leirer, V., Altieri, P., & Fitzsimmons, C.
(1994). When expertise reduces age differences in
performance. Psychology and Aging, 9, 134-48.




Chapter 4: Air Traffic Control Specialist Field Training Programs, 1981-1992

resulted in reassignments across options and not to
facilities within the same option. Thus, it was consid-
ered appropriate to categorize field training status sepa-
rately for each option.

Changes in Field Training Status Over Time

Tables 4-3 and 4-4 show status in en route and
terminal field training by the FAA Academy program
attended. About 10% more graduates of the Non-radar
Screen program reached FPL status than did graduates
of either the En Route or Terminal Screens. Table 4-3
also shows that a slightly higher percentage of Non-
radar Screen graduates (about 9%) than En Route
Screen program graduates (about 7%) transferred to
terminal facilities. However, the percentage of transfers
to FSS as well as failures was lower for Non-radar Screen
gtaduates than for graduates of the En Route Screen.
Also, a small percentage of Non-radar Screen program
graduates had not completed their training when these
analyses were conducted in April of 1997.

Graduates of the Non-radar Screen program who
entered terminal facilities had a lower percentage of
facility switches, FSS transfers, and failures than did
graduates of the Terminal program. Additional analyses
of field training status by year of graduation showed a
systematic increase in rates of reaching FPL over time.
For FAA Academy graduates entering en route field
training, rates of reaching FPL status were 63-66% for
1982-84 classes, 70-75% for 1985-88 classes, and 80—
83% for 1989-91 classes. Only about 72% of 1992 FAA
Academy Non-radar Screen program graduates in en
route field training reached FPL status as of April 1997,
butan additional 13% of that group are still in training.
(Although previous studies found that the average time
to reach FPL status in the En Route option is about 3
years [e.g., Manning, Della Rocco, & Bryant, 1989],
some individuals were identified who remained in train-
ing for 6 years or more.) For those entering terminal
field training, rates of reaching FPL status were 76-82%
for 1982-84 classes, 85-89% for 1985-89 classes, and
89-92% for 1990-92 classes.

The causes of observed changes in rates of attrition
from training over time are difficult to determine be-
cause modifications occurred in the content of both the
FAA Academy Non-radar Screen and in field training.
The Non-radar Screen program, introduced in 1986,
contained less information about weather and aircraft
performance characteristics than did the former Termi-
nal and En Route Screens. Subsequent training (both at
the FAA Academy and in the field) was eventually

25

modified to incorporate those lessons. It might be
expected that removing lessons from the Non-radar
Screen program would result in an increased training
attrition rate. However, training attrition rates actually
decreased for Non-radar Screen graduates. Thus, changes
in the content of the Non-radar Screen program were
probably not responsible for the changes in field attri-
tion rates for Non-radar Screen program graduates.

Another factor that may have reduced attrition from
en route field training is a change in the content of the
field training program. en route field training was
changed at some facilities to decrease the emphasis on
non-radar air traffic control duties and increase the
emphasis on radar associate duties. Although a certain
amount of non-radar training had been required for all
trainees, it was not directly applicable to the ATC duties
that some controllers actually performed. It was deter-
mined that the amount of non-radar training provided
should be proportional to the type of control that would
be conducted by the developmental controllers. This
change, which made the training more relevant, may
also have resulted in a reduction in the attrition rate
because a high percentage of failures could be traced
back to non-radar training.

Some en route facilities also changed the order in
which training phases were administered so that train-
ing for all control activities (radar associate and radar
control) was completed for a pair of sectors before
moving on to any other sectors, Additionally, informal,
undocumented changes occurred in the administration
of field training as a result of a decrease in the number
of people entering the facilities after about mid-1983.
Prior to the strike, numbers of trainees entering the
system (701 in 1980 and 300 before August 1981) were
much lower than the first 2 years after the strike (3416
in 1982 and 1720 in 1983). Lower rates of failure and
transfers to other facilities might have occurred for
trainees who graduated from the FAA Academy after
1983 because their OJT instructors had more time to
spend with individual trainees.

Field Training Status and Demographics

Tables 4-5 and 4-6 show field training status for each
option by the demographic variables described above.
Table 4-5 shows field training status by demographic
groups for those entering en route facilities. Rates of
reaching FPL status were approximately equivalent for
male (71.4%) and female (72.5%) en route trainees.
African-American trainees had lower rates of reaching
FPL (about 63%) than did non-minorities (about 72%),


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

Table 3-1. Demographic profile of FAA Academy entrants

Pre-Strike Post-Strike
1976-1981 1981-1985 nvr 92
(N=6,059) (N=13,533) ’
Experience
No related experience 30% 65% 79%
Aviation Only 20% 17% 5%
ATC Operations 50% 18% 16%
Average Age 26.5 26.5 26.2
Education
High School Only 14% 9% 11%
Some College 48% 44% 54%
College Degree 38% AT%. 35%
Gender
Male 83% 85% 80%
Female 17% 15% 20%
Minority Status
Minority 13% 8% 12%
Non-Minority 87% 92% 88%

22


GLOSSARY OF ABBREVIATIONS AND STATISTICAL TERMS

ABSR

ANOVA
ARTCC
ATA
ATC
ATCS

CAMI

CSC
CST

FAA
FPL
FSS
GAO

MANOVA
MCAT

Abstract Reasoning Test
“Cronbach’s alpha;” A measure of
the internal consistency of a set of
items. a ranges from 0 to 1, and
indicates how much the items in a
scale are measuring the same thing.
Analysis of variance

Air Route Traffic Control Center
Air Traffic Assistant

Air traffic control

Air Traffic Control Specialist

“Beta weight,” or standardized re-
gression coefficient; used to weight
each X predictor ina multiple regres-
sion equation

Civil Aeromedical Institute, Okla-
homa City, Oklahoma

Civil Service Commission
Controller Skills Test
“Eta-squared;” A type of correlation
coefficient that can be used to ex-
press a curvilinear relationship be-
tween 2 variables

Statistic computed in analyses of
variance, used for testing the statisti-
cal significance of the amount of
variance accounted for in one (or
more) dependent variables by one
(or more) independent variables
Federal Aviation Administration
Full performance level controller
Flight Service Station

General Accounting Office
Statistical symbol for the mean (or
average)

Multiple analysis of variance

Multiplex Controller Aptitude Test

NAS
NTIS

OAM
OjT

OKT
OPM

PATCO

Statistical symbol for number of per-
sons in sample or analysis

National Airspace System

National Technical Information Ser-
vice, Springfield, VA

Office of Aviation Medicine
On-the-job training

Occupational Knowledge Test

U.S. Office of Personnel Management

“P-value.” A statistical symbol indi-
cating the probability of obtaining a
result by chance alone 5 times or less
in 100 (p <.05), 1 time or less in 100
(p <.01), or 1 time or less in 1000 (p
< -001) independent samples

Professional Air Traffic Controller
Organization

Statistical symbol for the multiple
correlation between a single crite-
tion (Y) and multiple predictors (X,,
X,, .. X). R may range between 0
and 1, with 1 indicating that Y is
perfectly predicted by some weighted
combination of Xs. In multiple re-
gression analysis, the values on one
or more predictors (X,, X,, ... X,) are
used to predict or estimate values on
the criterion (Y). Each predictor is
statistically weighted by the stan-
dardized regression coefficient (b)
such that overall errors in the predic-
tion of Y values are minimized. The
degree of overall linear relationship
between Y and the predictors X,, X,,
.. X,is indexed by the multiple cor-
relation (R). An R of 1.0 indicates
that Y is perfectly predicted from the
combination of predictors, and an R
of 0 indicates that Y cannot be pre-
dicted from any of the predictors.


Figure
6-2: Projected controller workforce (CWF) vs. aviation activity (1982-2002)

6-3: Controller retirement eligibilities after the year 2000 ........sssssssssssssssssstecceeessssneseseeessneeeseesssnnneeessessnens
6-4: Outcomes at FAA Academy by age category for 1986-1992 (N = 14,392)

6-5: Outcomes at first facility by age category for en route controllers (N = 9,063)
for 1986-1992 FAA Academy graduates

6-6: Outcomes at first facility by age category for level 4 and 5 terminal controllers (N = 839)
for 1986-1992 FAA Academy graduates

LIST OF TABLES
Table

2-1: ATCS Aptitude test battery scoring .....ccssssesssssscssssssessscccssssseeseecsesssavesssessssnsesessssunesseceensnnmeeesasnnmaesees

2-2: Correlation matrix structure for regression analyses

2-3: Descriptive statistics and correlations for 1982-85 Terminal Screen graduates assigned to
terminal OJT ....

2-4: Descriptive statistics and correlations for 1982-85 En Route Screen graduates assigned to
en route OJT ...

2-5: Descriptive statistics and correlations for 1986-92 Non-radar Screen graduates assigned to
terminal OJT ...

2-6: Descriptive statistics and correlations for 1986-92 Non-radar Screen graduates assigned to
en route OJT

2-7: Civil service RATING validity in predicting FIELD TRAINING outcome by sample,

based on corrected correlations

2-8: Adverse impact analysis by gender based on OPM eli;
3-1: Demographic profile of FAA Academy entrants .....
4-1: Demographics of FAA Academy program graduates..........0+
4-2: Coding of field training status at first facility by option
4-3: En Route field training status by FAA Academy program

4-4: Terminal field training status by FAA Academy program

4-5: En Route field training status by demographic variables ........cssssssssssssesssccessssseseeseensneeessesesstaeeeseanssse

4-6: Terminal field training status by demographic variables

4-7: Time to attrition (years) by field option, FAA Academy program, and type of attrition... 32

5-1: Sample size, means and standard deviations for the organization climate measures
by survey administration



Chapter 2: Air Traffic Control Specialist Aptitude Testing, 1981-1992

Race

A formal adverse impact analysis on the basis of race
was not technically feasible, as racial identifiers were not
collected for ATCS job candidates. However, previous
research (Rock, Dailey, Ozur, Boone, & Pickrel, 1984)
found that the OPM test battery had adverse impact
against African-Americans and Hispanics. In view of
the previous research, Broach, Farmer, and Young (1997)
investigated the relationship between test scores and
performance in the FAA Academy for evidence of
differential prediction on the basis of race. The classical,
regression-based model of test bias was again used as the
analytic framework to evaluate the degree to which the
written ATCS test battery differentially predicted per-
formance in initial ATCS qualification training at the
FAA Academy for African-American and white control-
lers. After correcting correlations between test score,
FAA Academy composite score, and race for explicit and
implicit restrictions in range, the null hypothesis of a
common regression line for African-American and white
controllers was rejected, suggesting the presence of some
degree of test bias. Statistically significant differences by
race in regression slopes and intercepts were found in the
step-down regression analysis, indicating the need for
separate regression equations for African-American and
white controllers. The regression line for whites over-
predicted the performance of African-Americans in the
FAA Academy. The practical consequence of the appar-
ent differential prediction was that African-American
controllers effectively needed a higher test score than
white controllers to have an equal likelihood of passing
the initial qualification training at the FAA Academy, as
shown in Figure 2-5.

Discussion

The written ATCS aptitude test battery in use from
1981 through 1992 was developed to enable the agency
to make reasonable predictions, for large numbers of
applicants, about their future job performance as the
basis for initial selection into a safety-related occupa-
tion. From a practical perspective, the primary function
of the written ATCS test battery was to winnow a huge
pool of applicants down to a smaller, manageable num-
ber suitable for further intensive, expensive evaluation.
Ononchand, the written ATCS aptitude test battery, as
used from 1981 to 1992, achieved that organizational
goal. The tests comprising the battery produced reliable
scores. A composite of those reliable scores, used as the
basis for selection decisions, was valid as a predictor of
near-term (relative to date-of-hire) training outcomes.

On the other hand, the composite score was not a
valid predictor of far-term (2 to 3 years from hire)
field training outcomes. Moreover, the fairness analy-
ses suggested that the battery had adverse impact
against women, and was likely to have had adverse
impact on African-American job candidates. The test
battery over-predicted performance in initial ATCS
qualification training at the FAA Academy on the
basis of gender and race. Adverse impact and over-
prediction of training and job performance is com-
monly reported for tests of cognitive ability similar to
the ATCS test battery, such as the General Aptitude
Test Battery (GATB) (Hartigan & Wigdor, 1989;
Schmidt, 1988; Wigdor & Sackett, 1993). This out-
come does not result because scores on the ATCS
aptitude test battery mean something different for
women and African Americans. The test is not biased
in that sense. Rather, the adverse impact and over-
prediction of subsequent performance in initial train-
ing for both groups results from the interplay of two
factors: the lower average scores for African Ameri-
cans and women relative to white males, and the less-
than-perfect validity of the composite test score. As a
consequence of these facts, the predicted perfor-
mance of minorities and women is higher than their
actual performance. The practical result is that more
women and minorities were hired on the basis of their
predicted performance than would have been hired
on the basis of their actual job performance.

In conclusion, the written test battery was a valid
mass testing tool, with which the FAA could reduce
the applicant pool toa smaller, affordable number for
more intensive evaluation of their aptitude for the
controller occupation. The relative costs of assess-
ment methods must be carefully considered, particu-
larly in view of the substantial numbers of job
applicants. For example, the cost of administering
the paper-and-pencil ATCS test battery was about
$20 per examinee, compared with a cost of about
$10,000 per person for initial training at the FAA
Academy (Broach & Brecht-Clark, 1993). There was
significant financial utility for the agency in using a
valid test as the first step in a multiple-hurdle, se-
quential personnel selection system. Moreover, the
test battery was fair, in that test scores over-predicted
subsequent training performance of African Ameri-
cans and women. Overall, the written ATCS aptitude
test battery was an inexpensive, practical, valid, fair,
and invaluable tool in rebuilding the controller
workforce between 1981 and 1992.


INTRODUCTION

Dana Broach
FAA Civil Aeromedical Institute, Oklahoma City, Oklahoma

On August 3, 1981, the majority of the nation’s air
traffic controllers went on strike. On August 5, 1981,
President Reagan summarily fired the controllers
who had not returned to work. The controller strike
left 4,669 operational controllers available in August
1981 to staff the nation’s Air Route Control Centers,
terminal radar approach control, and airport control
towers (National Transportation Safety Board, 1981).
The Federal Aviation Administration (FAA) con-
fronted a challenge unique in the annals of American
government: that of rebuilding a core, technical,
highly trained workforce in an occupation directly
responsible for the safety of millions of lives ona daily
basis. From late calendar year 1981 to mid-year
1995, the FAA rebuilt this critical workforce through
a large-scale testing, screening and training program.

The testing and screening programs operated from
1981 through March, 1992. Field training data col-
lection ended in the summer of 1995. By then, the
controller workforce was near its pre-strike level,
with about 14,400 controllers working in the nation’s
en route centers, terminal radar facilities, and airport
towers. Throughout this period, the Civil Aeromedi-
cal Institute conducted a longitudinal research pro-
gram focused on the validation of the air traffic
control specialist (ATCS) selection program.

The following papers represent the first major
retrospective analysis of the complete data set
describing the recovery of the FAA’s en route and
terminal ATCS workforce following the 1981 con-
troller strike.

In the first paper, Jay Aul describes the personnel
processes supporting the recovery of the controller
workforce, focusing on recruitment and hiring pro-
grams for the center and terminal options.

Second, a detailed description of the numbers of
persons tested and the instrument used to test for
aptitude are presented by Dana Broach. The second
paper also evaluates the validity of the test battery as
a predictor of performance at the FAA Academy and
in the field.

The third paper, by Pam Della Rocco, offers a
definitive statistical portrait of the 1981 to 1992 FAA
Academy screening programs for the en route and
terminal options. Della Rocco also evaluates the
validity of the FAA Academy screening programs as
predictors of field training outcomes.

Fourth, the on-the-job training (OJT) program in
en route and terminal field facilities through June,
1995, is described by Carol Manning. These papers,
taken together, provide a definitive description of the
processes used to test, screen, and train persons for
the ATCS occupation between 1981 and 1992.

The fifth paper, by Richard Thompson, chronicles
air traffic controller perceptions of organizational
climate in which this recovery occurred, drawing on
FAA survey data and documented organizational
initiatives.

Finally, the demographics of the current ATCS
workforce, projections about future trends and pos-
sible requirements, and potential areas of research in
relation to both demographic and technological
trends, are described by David Schroeder, Dana
Broach, and Bill Farmer.


CHAPTER 3:

FAA ACADEMY AIR TRAFFIC CONTROL SPECIALIST
SCREENING PROGRAMS AND STRIKE RECOVERY

Pamela Della Rocco
FAA Civil Aeromedical Institute, Oklahoma City, Oklahoma

In 1976, the Federal Aviation Administration (FAA) implemented screening programs for Air Traffic Control

Specialists (ATCSs) at the FAA Academy which, in combination with a paper-and-pencil selection test, provided a

robust two-stage selection system used in recovery from the 1981 Professional Air Traffic Controller Organization

(PATCO) strike. The purpose of the present paper is to review data from the FAA Academy screening programs with

regard to their role in the FAA strike recovery efforts, as well as the evolution of the programs in the post-strike years.

Data are presented for pass rates, demographics, and evidence of validity of the programs.

At the time of the 1981 Professional Air Traffic
Controller Organization (PATCO) strike, the Fed-
eral Aviation Administration (FAA) selected Air Traf-
fic Control Specialists (ATCSs) through a two-stage
process (Della Rocco, Manning, & Wing, 1990).
The first stage was the Office of Personnel Manage-
ment (OPM) written air traffic control (ATC) test
battery (Broach, this volume). The second stage was
a FAA Academy initial qualification course. This
two-stage process provided a robust foundation for
recovery of the controller workforce in the wake of
the firing of the striking ATCSs. The purpose of the
present paper is to review the FAA Academy screen-
ing programs with regard to their role in the FAA
recovery from the strike, as well as the evolution of
the programs in the post-strike years. Data are pre-
sented for pass rates, demographics, and validity of
the programs.

Background

The FAA has conducted research on the selection
of ATCSs since the 1950s (Collins, Boone, &
VanDeventer, 1984). Written aptitude tests for the
occupation were implemented as early as 1964 and
were administered nation-wide. Despite the increased
validity of the written Civil Service Commission
(CSC) selection tests over the previous system based
on prior experience alone, the high occupational
attrition rates were a concern through the late 1960s
and into the early 1970s. In 1974, for example, the

17

failure rate was about 43% for en route trainees and
38% for terminal trainees (Henry, Ramrass, Orlansky,
Rowan, String, & Reichenbach, 1975). This training
attrition typically occurred two to three years into an
individual’s career (Manning, 1991). During the
early 1970s, several groups examined the FAA’s air
traffic training and selection process (Corson, 1970;
Henry et al., 1975; Committee on Government
Operations, 1976). The reports recommended the
development of a validated, standardized, and cen-
tralized pass/fail training program, designed to screen
persons who lacked sufficient potential to become
fully certified ATCSs early in their career (Boone,
1984). The FAA committed to the development of
screening programs, which would follow successful
performance on the written CSC (renamed Office of
Personnel Management in 1978) air traffic test bat-
tery (Boone, 1984).

Description of FAA Academy Screening programs
Pre-strike Screening programs

In 1976, the pass/fail FAA Academy initial qualifica-
tion courses were implemented. They were based on
non-radar air traffic control procedures. Separate courses
existed for the two career options, En Route and Termi-
nal. Entrants were assigned to an option based upon
agency staffing needs. The screening programs evolved
from training previously administered by the FAA Acad-
emy. The courses were designed to assess the aptitude
of an individual with no prior knowledge of the


DOT/FAA/AM-98/23

Office of Aviation Medicine
Washington, D.C. 20591

Recovery of the FAA Air
Traffic Control Specialist
Workforce, 1981-1992

Dana Broach, Editor

Civil Aeromedical Institute
Oklahoma City, Oklahoma 73125

August 1998

Final Report

This document is available to the public
through the National Technical Information

Service, Springfield, Virginia 22161.

U.S. Department
of Transportation

Federal Aviation
Administration


Recovery of the FAA Air Traffic Control Workforce, 1981-1995

80%

70%

60%
[Disagree
Neither
Agree

Figure 5-5: Organization of work perceptions of air traffic control specialists

80%

70%

60%
Disagree

50% E\Neither
Agree

40%

30%

20%

0%

1986

Figure 5-6: Management direction perceptions of air traffic control specialists

40


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

printed with red ink. With the smoke or dust trailing
the aircraft, it looked to some people as if the airliner
was in trouble. That particular format for the an-
nouncement was dropped. At the same time, for
budget reasons, there were government-wide restric-
tions on printing in multiple colors. Eventually,
however, the FAA received permission to print its
announcement as a four-color glossy brochure. The
brochure was quite different from the standard gov-
ernment vacancy announcement. In addition to the
required information, it contained a brief story out-
lining how controllers handle a single airline flight.
There were sample items from the civil service exami-
nation for air traffic controllers and color pictures of
controllers at work in various settings. The FAA built
a staff of full time recruiters who sought out sources
of applicants. The FAA learned to use other recruit-
ment methods during this period as well. FAA field
personnel offices frequently used newspaper adver-
tisements to recruit. Magazine, television, and radio
advertisements were also used in some markets
(Schultz & Marshall-Mies, 1988).

Selection

Competitive processes

Basic qualifications and pre-hire testing. One
reason that extensive recruitment was necessary was
that potential controllers had to meet a number of
qualifications requirements. First, they had to be
United States citizens. Second, in most cases, they
could be no younger than 18 years of age. They could
not have reached their 31st birthday prior to initial
appointment into the terminal or en route options.
To qualify for consideration at the entry level civil
service grade 5 (“GS-5”), an applicant needed some
combination of undergraduate or graduate level edu-
cation, progressively responsible work experience,
and/or specialized aviation experience. Candidates
were also required to take (and pass) the OPM civil
service examination for air traffic control specialists.
This written test was designed to identify people with
the aptitude to be air traffic controllers (Broach, this
volume). Under Title 5 of the U.S. Code of Federal
Regulations, all such civil service entrance examina-
tions must be scored on a scale of 100. The bottom
point is not designated in the regulations. The passing
score for a civil service test must be set at 70. To avoid
adverse impact against minority groups and women,

OPM required that the score of 70 on this test be
calibrated to the 50th percentile of the population
applying for these jobs at the GS-5 grade level.

However, most recruitment was for grade 7 (“GS-
7”), which was the more common entry grade for the
controller occupation. The qualifying score on the
written ATCS aptitude test for the GS-7 grade level was
75.1 for applicants without prior specialized experience
or education. To qualify for GS-7 with a score of 70, an
individual had to have additional experience or educa-
tion. Typically, this was certain kinds of aviation expe-
rience, one year of graduate school, or superior grades in
college. Federal law also required that extra points be
given to certain military veterans. The matter of who
received these extra points is too complex to fully
explain here. For simplicity, it can be explained as
including veterans of the U.S. military, who served in
declared wars, or in military actions for which campaign
badges or expeditionary medals were awarded. Nor-
mally, five extra points were added onto the examina-
tion score for veteran’s meeting these criteria. If the
veteran hada military service-related disability, he or she
received ten extra points, instead of five. Veterans also
had priority in hiring over non-veterans. For example, a
candidate with veteran’s preference had priority in
hiring over a candidate with the same score, but without
the veteran’s preference.

Interview, medical, and security criteria.

Candidates were also interviewed. The interview had
several purposes. One key purpose was to provide infor-
mation to the candidates about the ATCS occupation.
Most candidates did not have a clear understanding of
the work of air traffic controllers. It was important to
explain the concept of rotating shift work, because many
did not understand it. It was also a chance for the
interviewer to evaluate the candidate for such factors as
ability to speak clearly and to speak in English. Inter-
viewers were asked to make a hiring recommendation
for each candidate, although the interview was rarely
used to screen out candidates.

Candidates who progressed through these hurdles
then had to meet the medical requirements for these
positions. The medical standard for entry into these
positions was (and is) quite strict. It included vision and
hearing standards, as well as a form of mental health
screen. Personality assessment has been used formally by
the FAA in conjunction with the ATCS programs since
1965 (Convey, 1984). From the outset, the 38-item


CHAPTER 1:

EMPLOYING AIR TRAFFIC CONTROLLERS, 1981-1992

Jay C. Aul
FAA Office of Personnel, Washington, District of Columbia

The civil service procedures and processes used to recruit and select air traffic control specialists (ATCS) following

the 1981 controller strike are described. Recruitment approaches and strategies, basic occupational qualifications

requirements for applicants applying under competitive rules, and additional selection criteria such as an interview,

medical examination, and security and suitability investigation are presented. Different noncompetitive employment

processes, such as cooperative education, are briefly surveyed, concluding with current trends in controller selection.

On August 3, 1981, 10,438 members of the Pro-
fessional Air Traffic Control Organization (PATCO)
staged a job action, or strike, against the Federal
government. On August 5, 1981, President Reagan
summarily fired the controllers who had not returned
to work. The PATCO strike left 4,669 operational
controllers available in August 1981 to staff the
nation’s airport control towers, terminal radar ap-
proach control facilities, and Air Route Traffic Con-
trol Centers (National Transportation Safety Board,
1981). The Federal Aviation Administration (FAA)
needed to rebuild as quickly and prudently as pos-
sible after the loss of more than half of its controller
workforce. As the FAA’s air traffic controllers, more
properly called air traffic control specialists (ATCSs),
are civil servants, the employment process was gov-
erned by United States federal civil service laws and
regulations. The regulatory agency overseeing these
laws and regulations was the U.S. Office of Personnel
Management (OPM). The rules and regulations re-
sulted in well-defined and rather strict employment
procedures to ensure that there was open and fair
competition for controller positions in the FAA. The
purpose of this paper is to provide an overview of the
employment procedures used by the FAA to recruit and
select ATCSs during the recovery from the 1981 strike.

Recruitment
The process logically began with finding and at-
tracting controller candidates. Under civil service
rules, job openings must be advertised in a way that
ensures open competition. The government calls
these advertisements “vacancy announcements.” In

concrete form, they are printed documents that con-
tain certain detailed information, disclaimers, and
statements. The announcements mustalso be distrib-
uted to organizations, such as state employment
services. Advertising was not only a legal require-
ment, it was a practical necessity during the recovery
period. The traditional sources of candidates were
former military controllers (Collins, Manning, &
Taylor, 1984), and people who knew about the job
from some kind of personal contact. That contact
often turned out to be relatives or neighbors who
were air traffic controllers. Those sources were insuf-
ficient to rebuild the work force after the 1981 strike.
To hire enough people, it was necessary for the FAA
to recruit from the general population. Advertising
also had to overcome the perception that the strike
circumstances did not make the job the of an ATCS
sound appealing. The FAA and controller jobs re-
ceived much negative publicity in the years following
the strike. Moreover, the FAA was competing for a
scarce resource consisting of people under age 31 who
could score high on the civil service examination for air
traffic control positions. The military and private sec-
tors also valued and tried to attract bright young people.

The first post strike vacancy announcements were
little different from those issued by other government
agencies. They were several pages printed in black ink
on white paper, and the only graphic was the agency
logo. There was an attempt shortly after the strike to
create a more attractive, exciting announcement.
That turned out to be a picture of an airliner taking
off, with a banner above it saying something like,
“Control the Skies.” Unfortunately, the picture was


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

Collins, W.E., Nye, L.G., & Manning, C.A. (1990).
Studies of poststrike Air Traffic Control Specialist
Trainees: III. Changes in demographic characteris-
tics of Academy entrants and biodemographic pre-
dictors of success in air traffic controller selection and
Academy screening. (DOT/FAA/AM-90/4). Wash-
ington, DC: Federal Aviation Administration,
Office of Aviation Medicine. (NTIS No.
ADA223480)

Charness, N., & Bosman, E.A. (1990). Human factors
and design for older adults. In J.E. Birren & K.W.
Schaie (Eds.), Handbook of the psychology of aging
(3"ed.) (pp. 446-60). New York: Academic Press.

Cobb, B.B. (1967). Relationships among chronologi-
cal age, length of experience, and job performance
ratings of air route traffic control specialists. Aero-
space Medicine, 39, 119-24.

Cobb, B.B., Lay, C.D., & Bourdet, N.M. (1971).
Relationship between chronological age and apti-
tude test measures of advanced-level air traffic con-
trol trainees. (DOT/FAA/AM-71/36). Washing-
ton DC: Federal Aviation Administration, Office
of Aviation Medicine. (NTIS No. ADA733830)

Cobb, B.B., Nelson, P.L., & Matthews, J.J. (1973).
The relationships of age and ATC experience to job
performance ratings of Terminal area controllers.
(DOT/FAA/AM-73/7). Washington DC: Fed-
eral Aviation Administration., Office of Aviation
Medicine. (NTIS No. ADA733449)

Cohen, J. (1988). Statistical power analysis for the behav-
ioral sciences (2nd ed.). Hillsdale, NJ: Lawrence
Erlbaum Associates.

Cohen, J. (1992). A power primer. Psychological Bulle-
tin, 112, 155-59.

Convey, J.J. (1984). Personality assessment of ATC
applicants. In S.B. Sells, J.T. Dailey, & E.W.
Pickrel (Eds.), Selection of air traffic controllers
(pp. 323-52). (DOT/FAA/AM-84/2). Wash-
ington, DC: Federal Aviation Administration,
Office of Aviation Medicine. (NTIS No.
ADA147765)

Corson, J.J., Bernhard, P.W., Catterson, A.D., Fleming,
R.W., Lewis, A.D., Mitchell, J.M., & Ruttenberg,
S.H. (1970, January). The career of the air traffic
controller - a course of action. (Final report of the
Air Traffic Controller Career Committee). Wash-
ington, DC: Department of Transportation, OF
fice of the Secretary of Transportation.

54

Cranny, J.C., Smith, P.C. & Stone, E.F. (1992). Job
Satisfaction: How people feel about their jobs and
how it affects their performance. New York: Lex-
ington Books.

Caaja, S.J. (1996). Aging and the acquisition of com-
puter skills. In W.A. Rogers, A.D. Fisk, & N.
Walker (Eds.), Aging and skilled performance: Ad-
vances in theory and applications (pp. 201 - 20).
Mahwah NJ: Lawrence Erlbaum Associates.

Dailey, J.T., & Pickrel, E.W. (1984a). Development of
the Multiplex Controller Aptitude Test. In S.B.
Sells, J.T. Dailey, & E.W. Pickrel (Eds.), Selection
of air traffic controllers (pp. 281-98). (DOT/FAA/
AM-84/2). Washington, DC: Federal Aviation
Administration Office of Aviation Medicine.

(NTIS No. ADA147765)

Dailey, J.T., & Pickrel, E.W. (1984b). Development of
the air traffic controller Occupational Knowledge
Test. In S.B. Sells, J.T. Dailey, & E.W. Pickrel
(Eds.), Selection of air traffic controllers (pp. 299-
322). (DOT/FAA/AM-84/2). Washington, DC:
Federal Aviation Administration Office of Avia-
tion Medicine. (NTIS No. ADA147765)

Della Rocco, P.S., Manning, C.A., & Wing, H. (1990).
Selection of air traffic controllers for automated
systems: Applications from current research. (DOT/
FAA/AM-90/13). Washington, DC: Federal Avia-
tion Administration, Office of Aviation Medi-
cine. (NTIS No. ADA230058).

Della Rocco, P.S. & Sawyer, G.D. (1990). Declining
pass rates in the FAA Academy Screen and trends in
selection of women and minority candidates. (Memo-
randum Report CAMI/AAM-523-90-1). Okla-
homa City, OK: Federal Aviation Administra-
tion, Civil Aeromedical Institute.

Equal Employment Opportunity Commission. (1978).
Uniform Guidelines on Employee Selection Proce-
dures, 29 CER Part 1607.

Federal Aviation Administration. (1984, September).
Employment program for developmental Air Traffic
Control Specialists. (FAA Order 3330.30C). Wash-
ington, D.C. Department of Transportation, Fed-
eral Aviation Administration.

Federal Aviation Administration. (1985, July). Na-
tional air traffic training tracking system. (FAA
Order 3120.22A). Washington, D.C. Depart-
ment of Transportation, Federal Aviation Ad-
ministration,


Recovery of the FAA Air Traffic Control Workforce, 1981-1995

assessed by 4 items (average 0, = .87). The amount of
direction provided by management was assessed by 3
items (average & = .87). Management direction in-
cludes things such as setting of goals, and providing the
employees with appropriate guidance. For all of the
multiple-item measures, the items assessing the dimen-
sion were averaged to obtain a single dimension score.

Results

A one-way ANOVA demonstrated there was a
significant trend in the reporting of the job satisfac-
tion item, F(3, 31164) = 68.70, p< .001. The means
and standard deviations for each survey administra-
tion are presented in Table 5-1. As can be seen, there
was a gradual increase in job satisfaction between
1984 and 1988, and a slight decrement in 1990.
However, levels of job satisfaction in 1990 were still
above those noted in 1984 and 1986. Post hoc
analyses indicate all means are significantly different.
Because of the very large number of respondents,
however, any difference between means is likely to be
statistically significant. Therefore, the effect size, or
the amount of variance accounted for was examined.
(Cohen, 1988, 1992). For this analysis, the amount
of variance accounted for was very small (1? < .004).
The small effect size is due, in part, to the single item
measure of job satisfaction, which has a larger stan-
dard deviation compared to measures based on mul-
tiple items, as can be seen in Table 5-1.

To determine if there were any differences in the
reported perceptions of the five multiple item mea-
sures of organization climate across administrations
a one-way MANOVA was conducted. This test indi-
cated the presence of at least one significant differ-
ence, F(15,93024)=151.02, p < .001. Follow-up
univariate analyses for each of the five dependent
variable measures were also statistically significant.
The descriptive statistics for this analysis are also
reported in Table 5-1. Again, however, because of the
number of respondents, effect sizes were examined
for each analysis (Cohen, 1988, 1992). With a single
exception, the effect size of each analysis was trivial
(e.g., 1? < .01). The exception was for pay and
benefits satisfaction (1? < .042). While this is still a
small effect, an examination of the means indicates
that the level of perceived equity of pay and benefits
decreased most after the first assessment.

36

The lack of discernible upward or downward trends
in the above analysis led to an examination of re-
sponse rates. For this analysis, the individual re-
sponses to each item that comprise a measure were
assessed to determine what proportion of responses
can be categorized into disagree, neither agree nor
disagree, or agree for each administration of the
survey. Figure 5-1 shows the trend for the single item
measure of job satisfaction. As can be seen, there is a
noticeable upward trend from 1984 to 1988, with a
drop in 1990. Figure 5-2 shows there is a bimodal
distribution of responses to the job satisfaction index
with just over 50% of the respondents indicating
satisfaction, and about one third indicating dissatis-
faction. In 1986, 1988, and 1990 the discrepancy
between the job satisfaction item and the satisfaction
index is primarily due to lower ratings of benefits
satisfaction, and toa lesser extent, to items dealing with
working conditions and organization satisfaction.

The percent of respondents indicating manage-
ment has a concern for employees also increases
slightly across the first three survey administrations
(see Figure 5-3). At the same time, the number of
respondents disagreeing decreases. A different pat-
tern of responses emerges for pay and benefits, and is
summarized in Figure 5-4. Here, a majority of re-
spondents indicate that pay and benefits are equi-
table, while 20% disagree. There is no upward trend
for the first three administrations. Again, it appears
that perceptions regarding benefits reduce the num-
ber of respondents perceiving equity. Perceptions of
benefits on the 1986 survey may be related to the
process of conversion from the Civil Service Retire-
ment System (CSRS) to the Federal Employee Retire-
ment System (FERS). Figure 5-5 shows the response
rates for the degree to which ATCSs indicate work is
well organized. There is an upward trend in the agree
category from 1984 to 1988, followed by a drop.
Finally, Figure 5-6 also shows an upward trend for
the respondents who report adequate management
direction, and a corresponding downward trend for
respondents who disagree. Once again, in 1990, the
percent of agree respondents drops. Overall, there
appears to be an improvement in perceptions of
ATCSs from 1984 to 1988, as reflected by increased
agreement and decreased disagreement for most of
the climate dimensions. Itis likely that the 1990 drop
in perceptions for many of the climate dimensions is


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

terminal training included classroom, laboratory, and
OJT components. Individuals who completed their
training at one terminal facility, became Full Perfor-
mance Level (FPL) controllers, then transferred to an-
other facility, were required to complete all phases of
training relevant to their new facility, even if they had
already completed the same phases at their previous
facility. This process was required to accommodate
learning how to perform ATC duties in a new airspace
at a new facility which may be different than the way
those duties were performed in the airspace at the
previous facility.

Sample Demographics

As described by Della Rocco (this volume), the
demographics of candidates entering the FAA Academy
screening programs changed between 1981 and 1992,
so it can be expected that the demographics of those
entering field training would reflect those changes.
Table 4-1 compares demographics for graduates of the
1981-1985 En Route Screen program (hereafter re-
ferred to as ‘the En Route Screen’), the 1981-1985
Terminal Screen program (hereafter referred to as ‘the
Terminal Screen’) and the 1986-1992 Non-radar Screen
program (hereafter referred tos ‘the Non-radar Screen’).

Differences in demographics can be seen between
groups of graduates of the two FAA Academy programs
conducted at the same time as well as groups that
graduated at different times. The percentage of males
graduating from the En Route Screen was slightly
higher (88%) than was the percentage graduating from
either the Terminal Screen (83%) or from the Non-
radar Screen program (82%). The percentage of non-
minorities was highest among En Route Screen graduates
(95%), lower among Terminal Screen graduates (about
92%), and lowest among graduates of the Non-radar
Screen (about 91%). Most trainees graduating from the
FAA Academy entered through the competitive hiring
process. Over 90% of En Route Screen graduates were
competitive hires, while about the same percentage of
Terminal and Non-radar Screen graduates (82.4% and
82.7%, respectively) were hired competitively. Of those
for whom the amount of education was known, about
the same percentage of En Route and Terminal Screen
graduates had a college degree (46.7% and 48.4%,
respectively) while fewer Non-radar Screen program
graduates had a college degree (about 35%).

24

Field Training Outcomes

Developmental controllers who entered field train-
ing after the 1981 strike were required to complete each
phase of training successfully in a specified period of
time to avoid removal (FAA Order 3330.30C, 1984). If
removed from training, their disposition depended on
the type of facility to which they were assigned and the
amount of training they completed prior to removal.
Generally, en route trainees who completed part of their
radar training prior to removal were eligible for reassign-
ment to a terminal facility of level 3 or below. En route
trainees completing initial radar associate training prior
to removal were eligible for reassignment to a non-radar
facility or a Flight Service Station (FSS). Those who
completed the classroom portion of radar associate
training were eligible for reassignment to an FSS. A
somewhat similar reassignment structure existed for
terminal trainees. Those assigned to a level 4 or 5 facility
who failed to complete training on local control were
eligible for reassignment to a level 1 or 2 non-radar or
limited radar facility. Those who failed to complete
radar training were eligible for reassignment to a lower
level non-radar facility or level 2 limited radar facility.
Those who failed to complete radar training after certi-
fying on at least 1 radar position were eligible for
reassignment to a lower level terminal facility.

A variable was developed to describe the status in
field training for trainees at their first facility assignment
after FAA Academy graduation. Use of this variable
assumed that a developmental who transferred to an-
other facility before reaching FPL status did so because
of failure to complete the training program. Exceptions
to this assumption were made for those that we knew
had transferred or separated from the occupation for
reasons unrelated to failure and for en route trainees
who switched to another center before reaching FPL
status because this type of transfer was typically unre-
lated to training performance. However other transfers,
for example, from en route to a lower level terminal
facility or to a FSS, or from a terminal radar facility to
a lower level terminal facility or a FSS, were assumed to
be related to training failure.

The values of the variable indicating field training
status are shown in Table 4-2. Facility switches within
the terminal option were typically from a higher activity
facility to a lower activity facility and typically resulted
from failure to complete training. On the other hand,
facility switches within the en route option were not
considered failures because en route failures typically


Chapter 5: Organizational Climate Perceptions of the ATCS Workforce, 1981-1992

Table 5-1: Sample Size, Means and Standard Deviations for the Organization
Climate Measures by Survey Administration

1984 1986 1988 1990
(N =12,377) (N =14,027) (N =2,086 ) (N =2,456 )

Organization Climate Measures = M SD M sD M SD M SD

Job Satisfaction’ 3.32 1.25 3.37 1.23 3.64 1.15 3.49 1.18
Satisfaction 3.32. 1.25 3.26 0.80 3.35 0.78 3.22 0.78
Management Concern for 2.31 0.99 2.49 1.04 2.57 0.98 2.42 0.96
Employees

Pay and Benefits 3.49 0.89 3.12 0.76 3.32 0.71 3.20 0.75
Organization of Work 2.53 0.85 2.69 0.91 2.75 0.85 2.65 0.85
Management Direction 2.92 0.83 2.98 0.87 3.04 0.84 2.92 0.85

Notes: *Based on a single item

4l


CHAPTER 4:
AIR TRAFFIC CONTROL SPECIALIST FIELD TRAINING PROGRAMS, 1981-1992

Carol A. Manning
FAA Civil Aeromedical Institute, Oklahoma City, Oklahoma

This paper presents statistics describing the status of graduates from the FAA Academy ATCS screening programs

(Della Rocco, this volume) in subsequent field training programs. Of the 8,160 FAA Academy graduates assigned to
En Route facilities between 1981 and 1992, 71.6% were certified as full performance level (FPL) controllers by June
1995. Of the 6,417 FAA Academy graduates assigned to Terminal facilities between 1981 and 1992, 83.8% were
certified as FPL by June 1995. The data indicate that the FAA successfully implemented and executed a large-scale
field training program to rebuild the controller workforce following the 1981 PATCO strike.

When approximately 10,400 of 17,000 air traffic
controllers were fired asa result of the 1981 Professional
Air Traffic Controller Organization (PATCO) strike,
the FAA was required to hire and train their replace-
ments. Other papers in this volume describe the pro-
cesses for recruiting and selecting Air Traffic Control
Specialist (ATCS) trainees. This paper presents statistics
describing their status in ATCS field training programs.
The statistics were extracted from a data base developed
at the FAA Civil Aeromedical Institute (CAMI) after
the 1981 strike. The data were used to track develop-
mental progress in field training (per FAA Order
3120.22A, 1985) and provide feedback about the effec-
tiveness of the selection procedures. The complete field
training data base included information about those
who graduated from the FAA Academy and certain
other types of trainees (e.g., military hires) regarding
overall success rates in field training as well as pass rates
and times to complete specific phases of field training.
Only field training status for FAA Academy graduates
will be described here.

Background

Successful graduates of the FAA Academy in Okla-
homa City entered a multi-phase training program
administered at their assigned field facility. The content
of this training depended on their option (En Route or
Terminal) and facility assignment. For several years
after the strike, trainees assigned to the en route option
(that is, Air Route Traffic Control Centers or ARTCCs)
took equivalent types of training that differed only
according to their area of specialization. Each ARTCC

23

partitions their airspace into between 4 and 8 areas of
specialization. A developmental who began training at
an ARTCC wasassigned to only 1 of these areas and was
required to certify that he or she could perform air traffic
control (ATC) duties on all sectors (or sub-partitions of
airspace) within that area before being allowed to con-
trol traffic without supervision. After an initial phase of
training in which trainees demonstrated their knowl-
edge of the area of specialization by learning the maps
for all sectors, they took a series of training phases
designed to prepare them for certification on 3 types of
ATCS duties (those of the assistant controller, radar
associate/non-radar controller, and radar controller) in
their area. Training for one set of ATC duties was
typically completed on all sectors before beginning
training for the next set of duties. Training phases
conducted in classroom, laboratory, and OJT (on-the-
job training) environments were reported separately.
The content of training at terminal facilities de-
pended on the ATC positions used at each type of
facility. Non-approach or Visual Flight Rules (VFR)
facilities conducted training on Flight Data, Clearance
Delivery, Ground Control, and Local Control posi-
tions. Limited radar facilities conducted an additional
phase for the Non-radar position. Trainees at facilities
that had both non-approach and radar functions com-
pleted all the above phases along with an additional
phase of training for the Radar position. Some facilities
were split administratively so that controllers worked in
either the tower cab or the radar room but not in both.
In this case, they were required to take only phases of
training relevant to their specific duties. Each phase of


RATING

SCREEN

SD

STATUS

TMC

“R-squared;” Statistical symbol for
the squared multiple correlation.
R’estimates the proportion or
amount of variability in criterion (Y)
values or scores accounted for or
explained by the optimally weighted
combination of predictors.

Final civil service rating at hire, based

on OPM test scores and veteran’s
preference points

Variable representing controller per-
formance in the FAA Academy ATCS
initial qualifications training program
Statistical symbol for standard
deviation

Variable representing outcomes in
controller on-the-job field training
at the first assigned field air traffic
control facility

Transmuted Composite score, a
weighted sum of ABSR and MCAT
scores on the OPM test battery

vi

VFR

GLOSSARY OF ABBREVIATIONS AND STATISTICAL TERMS

Visual Flight Rules
Veterans Readjustment Act

Statistical symbol used to denote the
magnitude of difference between two
proportions in terms of the area un-
der a normal curve. Z is a statistic
used for testing if two proportions
are truly different from each other,
taking into account the base popula-
tion on which the proportions are
calculated. The likelihood of obtain-
ing a Z (or any other statistic) by
chance alone in 100 or 1000 analyses
based on different samples is the
significance (p) of the statistic. A p <
.05, < .01, or < .001 indicates that
less than 5 in 100, 1 in 100, or 1 in
1000 such analyses, respectively,
would yield a statistic of that magni-
tude by chance alone.


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

90

—— Whites

—~ 48 — African-Americans

70 15 80 85 90
Test score (TMC)

95 100

Figure 2-5: Regression of test score (TMC) on FAA Academy score (SCREEN) by race

Table 2-1: ATCS Aptitude test battery scoring

Test OPM # Scoring Weight N Items

MCAT 510 N Right 2 110

ABSR 157 N Right - (0.25*N Wrong) 1 50

OKT 512 N Right ° 80

Note: Extra points awarded as follows for OKT raw scores: 0-51 = 0 extra points; 52-55 = 3 extra points;

56-59 = 5 extra points; 60-63 = 10 extra points; 64-80 = 15 extra points for computation of the

civil service rating.

Table 2-2: Correlation matrix structure for regression analyses

RATING SCREEN STATUS
RATING e eaancy Ri screens
SCREEN ts Re sceuew
STATUS rs ts

Note: Sample correlation matrix structure shown below the

diagonal, corrected matrix structure above the diagonal.
rs = sample correlation; re = correlation corrected
(RATING!

for explicit selection on RATING; 1; = correlation
Viscreeny

corrected for incidental restriction in range due to
selection on SCREEN; and re = correlation
SCREEN)

corrected for explicit selection on SCREEN

14


Chapt. 6: Current FAA Controller Workforce Demographics, Future Requirements, and Research Questions

100
0 nes
BO 4---- +--+ - 22 e eee e eee e eee e cence eter eees
De nee
mms) GO + eB ug nec eet ee neces
ae? ee ee | jn

@ 50 4-R-------BRee BB B

(5)

=

o

a
LE 22 23-24 25-26 27-28 29-30 GE 31
(N=3,320) | (N=5,987) | (N=6,036) | (N=5,692) | (N=4,762) | (N=2,12
WPASSED 66.63 63.35 59.77 53.74 48.19 48.64
OFAILED 25.81 28.23 30.57 35.22 37.72 37.17
| WITHDREW 7.56 8.42 9.66 11.03 14.09 14.19

Figure 6-4: FAA Academy outcomes by age group for all entrants, 1981-1992

Percent

23-24

25-26 27-28 29-30 GE 31
(N=2,190) | (N=2,015) | (N=1,624) | (N=1,150) | (N=464)
mFPL 75.39 70.62 67.86 59.91 52.80
OSwitched 14.38 15.09 17.61 21.39 25.00
Failed 10.23 14.29 14.53 18.70 22.20

Figure 6-5: OJT outcomes at first facility by age category for FAA Academy graduates
initially assigned to en route centers (N = 8,748)

51


Chapter 4: Air Traffic Control Specialist Field Training Programs, 1981-1992

Table 4-6: Terminal field training status by demographic variables

Terminal field training status*

FPL Facility switch FSS transfer Fail
N % N % N % N %
Gender
Male 4434 84.7 390 7A 66 1.3 347 6.6
Female 944 80.0 130 11.0 16 14 90 7.6
Minority Status”
Amer. Indian 33 84.6 2 5.1 4 10.3
Asian-Pac. 63 87.5 3 42 2 2.8 4 5.6
African-Amer. 197 77.0 22 8.6 4 1.6 33 12.9
Hispanic 132 80.0 21 12.7 1 6 11 6.7
Non-Minority 4834 84.1 464 8.1 75 1.3 378 6.6
Entry Type
Competitive 4367 84.8 387 7.5 58 1.1 337 65
Airway Science 37 94.9 1 2.6 1 2.6
Pre-Devel. 119 72.1 19 11.5 6 3.6 21 12.7
Coop Ed 164 78.8 24 11.5 1 5 19 9.1
ATA 309 79.4 48 12.3 4 1.0 28 7.2
VRA 42 89.4 3 6.4 1 21 1 21
Noncompetitive 182 75.5 28 11.6 5 21 26 10.8
Former ATC 83 89.2 4 43 4 43 2 2.2
Former GS-9 1 100.0
Education®

No coll. degree 3157 85.9 282 7.7 25 0.7 210 5.7
College degree 2042 81.3 229 9.1 26 1.0 214 8.5

Note: *Percents are row percents.
“Minority status: Amer, Indian, American Indian/Alaskan. Asian-Pac., Asian-Pacific Islander. ‘Entry:
Airway Science, graduate of accredited Airway Science program. Pre-Devel., participated in FAA
Pre-Developmental Program. Coop. Ed., Cooperative Education Program. ATA, former Air Traffic
Assistant (GS-2154 or GS-2102.) VRA, hired under Veterans Readjustment Act. Former ATC,
previously graduated from FAA Academy and failed field training. Former GS-9, previously hired
under GS-9 military hire program and failed field training.
“Education: No coll. degree includes high school graduates and candidates with some college
hours but no degree. College degree includes those with a bachelor’s or higher degree.

31


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

Table 4-7: Time to attrition (years) by field option, FAA Academy program, and type of attrition

FAA Academy Program

En Route or Terminal Screen Non-radar Screen
1981-85 1986-92

Type of attrition N M sD N M SD
Terminal

Switch FSS 68 1.65 1.98 14 1.65 0.62

Fail 300 1.06 1.05 136 1.22 0.63

Other 169 1.26 1.42 140 0.94 0.77

All types attrition 537 1.20 1.33 291 1.11 0.72
En Route

Switch Terminal 289 2.20 1.41 392 2.60 1.33

Switch FSS 423 1.08 0.98 115 1.84 1.00

Fail 726 1.17 0.93 449 1.75 1.02

Other 180 1.18 1.16 281 1.58 1.26

All types attrition 1618 1.33 1.15 1237 1.99 1.25

Note: Option: Switch FSS, changed options from either En Route or Terminal to Flight Service
Station. Fail, failed training and separated from ATC occupation. Other, separated from
ATC occupation for reasons unrelated to failure. Switch Terminal, changed options from En
Route to Terminal.

32


Chapt. 6: Current FAA Controller Workforce Demographics, Future Requirements, and Research Questions

of increased decision aiding. The extent to which the
new system and procedures impact on the nature of
the job is highly dependent on the level of automa-
tion selected (Wickens, Mavor, & McGee, 1997). In
his recent book on automation in aviation, Billings
(1997) provides four scenarios regarding the nature
of the future air traffic control system: management
by delegation; management by consent; management
by exception; and free flight. There are marked differ-
ences in the roles and responsibilities of ATCSs
under these scenarios. Both the Automated En Route
air traffic control (AERA) and free flight concepts
appear to raise a number of human factors concerns
related to controller-machine interactions.

CAMI scientists have embarked on a program of
research designed to provide an answer to those
questions. The initial study was designed to assess
controller perceptions of AERA and determine how
perceptions of task activities under AERA would
impact the requisite knowledge, skills, and abilities
of ATCSs. Findings by Manning & Broach (1993)
indicate that many of the same skills and abilities
might be required. Since we are no longer moving
down an identical technological pathway, the ques-
tion remains regarding the demands and job tasks
associated with the future ATC systems. The infra-
structure technologies that are being readied for
introduction into the NAS during the remainder of
this decade are not likely to dramatically alter the job
duties and functions of ATCSs. However, the imple-
mentation of the initial and upgraded versions of
decision-support systems, and the revised AERA sys-
tem are likely to substantially modify the nature of
the job. Given the time required to recruit, select, and
train personnel, a new advanced selection system
needs to be developed coincident with the develop-
ment of the proposed future systems.

Traditional approaches to job analysis have fo-
cused on jobs that incumbents have already been
performing. In planning for the future, an approach
is needed that will allow us to identify the knowledge,
skills and abilities required as the systems are under
development. One strategy is to utilize the “strategic
job analysis” approach presented by Schneider and
Konz (1989). Efforts are underway at CAMI to
develop the methodology required to conduct a stra-

47

tegic job analysis for both air traffic control and
airway facilities positions. Once developed, the ap-
proach will be used to assess the changes in knowl-
edge, skills, and abilities as new systems are developed
for the NAS. With the anticipated increased empha-
sis on coordination and shared decision-making,
there may be additional requirements to identify
personality dimensions that play a critical role in
teamwork. Of course if the level of automation se-
lected is one where the human operator is primarily
asystem monitor and will only take over in the advent
ofan automation-related failure, this presents a much
different view of the nature of the job and the neces-
sary knowledge, skills and abilities. Outcomes from
the future job analysis assessments will be compared
with the recent baseline job analysis data (Nickels,
Bobko, Blair, Sands, & Tartak, 1995). The identifi-
cation of the requisite knowledge, skills, and abilities
that are sufficiently different from those required
with current systems will be used to modify existing
selection and training programs. Ideally, this will
allow the FAA to develop an integrated recruitment,
selection, training, and job performance data collec-
tion and analysis system. That system should be in
place in time to select the personnel who will be
operating those future systems. An outline for re-
search to support selection of personnel for the future
NAS architecture is described in Broach (1997).

Summary

While Billings (1997) and Wickens, Mavor, and
McGee (1997) have reviewed and identified a num-
ber of human factors that need to be addressed during
the transition to the more automated ATC system of
the future, in this paper we have sought to identify
personnel-related issues and concerns. The projected
future increase in air traffic, aging of the ATC
workforce, and increased requirements to train both
new personnel and members of the existing workforce
to operate the future more automated ATC system,
will need to be addressed. With the advent of the new
technologies, there is also a clear need to identify the
knowledge, skills and abilities required to perform
the duties of the future controller and to develop
instruments to support the recruitment and selection
of individuals who can best operate those systems.


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

facility as follows (Manning, 1991): 1 = Reached Full
Performance Level, 2 = Still in training (Developmental),
3 = Switched facilities, 4 = Switched options, or 5 = Failed.
Data for individuals who separated from the occupation
for reasons unrelated to performance were excluded.
The correlation between Non-radar Screen scores and
field status in the en route option was -.12 (N = 3,484,
ps<.-001). Adjusted for explicit restriction in range on
the FAA Academy score (Thorndike, 1949), this corre-
lation was -.25. The correlation between Non-radar
Screen scores and terminal field status was -.08 (N =
2,505, p < .001). The correlation, adjusted for range
restriction, was -.17. Figures 3-1 and 3-2 present the
relationship between FAA Academy Score and attain-
ment of Full Performance Level (FPL) field status at the
first facility assignment for en route and terminal op-
tions, respectively.

Evaluation of Potential Adverse Impact

The screening programs were monitored on an ongo-
ing basis for the success of minorities and women to
determine compliance with the Uniform Guidelines on
Employee Selection Procedures (“Uniform Guidelines,”
Equal Employment Opportunity Commission, 1978).
The Uniform Guidelines provided a “rule of thumb” for
assessing adverse impact in protected groups to deter-
mine if the procedure differentially selected one group
over another. An impact ratio is calculated for each
target group’s pass rate by the pass rate of the group with
the highest rate (typically the non-minorities). By the
rule of thumb, if the impact ratio is greater than or equal
to .80, the program is usually assumed to demonstrate
no adverse impact. The En Route and Terminal Screen
programs were found to have adverse impact against
African Americans in 1984-85 (Manning, Kegg, &
Collins, 1989). Part of the rationale for modifying the
FAA Academy program scoring was to reduce the ad-
verse impact. Initial assessments of the Non-radar Screen
program for the January 1986-January 1988 entrants
revealed an impact ratio of .832, an improvement over
the old En Route Screen program. However, for the
final two and a half years of the program the ratio for
African Americans fell below the .80 rule of thumb. A
review of impact ratios by fiscal year revealed the follow-
ing: .55 for FY 90, .60 for FY 91, and .52 for the final
classes during FY 92.

20

Discussion

The two-stage ATCS selection process involving the
Office of Personnel Management (formerly Civil Ser-
vice Commission) testing and FAA Academy screening,
was a very robust and predictable system that served the
agency well as a gateway to a safety-critical career. The
experience with the two-stage system, as well as a data-
base for assessing system performance, provided a foun-
dation for recovery from the PATCO strike in 1981.

The correlations in the present analyses between
Non-radar Screen scoresand field status were lower than
those previously reported. One possible reason was that
field training was modified during the late 1980s (Man-
ning, 1991). The influx of a large number of
developmentals at the lower-staffed facilities would
certainly have led to some adjustments in the processes
needed to provide instruction and supervision for the
on-the-job training provided to the developmentals. To
date, adequate criterion measures have been a problem
in establishing the validity of the ATCS screening
programs (Della Rocco, Manning, & Wing, 1990;
Broach & Manning, 1997). Thedecline in impact ratios
may have resulted from the direct hire authority granted
to the FAA by OPM during the late 1980s (Della Rocco
& Sawyer, 1990). A higher percentage of applicants
with lower OPM scores were hired with a predictable
outcome of lower FAA Academy pass rates.

In March 1992, the final classes graduated from the
FAA Academy Non-radar Screen program, marking the
end of an era for air traffic control specialist selection
procedures, which had lasted nearly two decades. The
system succumbed to various pressures, such as calls to
shorten and reduce the costs of the second stage screen
(Aerospace Sciences, Inc., 1991), as well as a compro-
mise of the OPM test battery (Manning, 1991). In the
Screen’s miniaturized training-testing program, indi-
viduals with no prior knowledge of the ATC occupation
demonstrated that they could separate airplanes in labo-
ratory simulations prior to continuing in the career
field. The FAA is currently examining the possibility of
using a computerized test battery to replace the former
selection systems. It remains to be seen whether or not
a cognitive test battery, which measures component
skills and abilities in combination with noncognitive
measures, can, in fact, effectively replace the work
sample model employed in the screening programs. The
history of ATCS selection certainly suggests that a
robust system can be developed to supply a steady flow
of qualified applicants.


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

0002 204 up aye Sant|iqi8i[9 ]UALLAINAL .O/JO.NUCD *¢-9 aanBiy

Fea, [OSH
0202 6102 8102 £102 9102 SL0z P10z E102 ZLOz 110z OL0z 6002 8002 LOO 9002 300 OD E002 ZOO 1002 000 6861 8661

ool

50




Chapter 5: Organizational Climate Perceptions of the ATCS Workforce, 1981-1992

70%

Disagree
| Neither
Agree

50%

40%

30%

20%

10%

0%

Figure 5-3: Management concern for employee's perceptions of air traffic control specialists
80%

70%
60%

[Disagree

EaNeither
50%

DAgree

Figure 5-4: Pay and benefits perceptions of air traffic control specialists

39


Chapter 5: Organizational Climate Perceptions of the ATCS Workforce, 1981-1992

related to the short-term budgetary furloughs the
agency underwent immediately prior to the distribu-
tion of the survey.

Discussion

This study briefly summarizes the perceptions of
five organizational climate dimensions identified as
problematic by Jones etal., and related to the PATCO
strike in 1981, and examines the FAA’s interventions
on controller perceptions over the course of the
ATCS workforce recovery. At a national level, the
FAA tried to improve the selection and training of
supervisors, to enhance communication and under-
standing through the use of survey feedback, and
tried to create more rewarding jobs by altering pay
systems. The results suggest that these interventions
have had a small positive impact, but have not mark-
edly altered the perceptions of ATCSs between 1984
and 1990. Relevant perceptions did improve slightly
from 1984 to 1988, then decreased slightly in 1990.

Of course, the apparent lack of change must be
viewed with some caution because of methodological
flaws with the present study. The methodological
problems include issues of measurement and design.
One methodological caveat is the use of self-report
measures. The exclusive reliance upon self-report
measures can create a susceptibility to monomethod
bias. Such a possibility is minimized, however, due to
the measures being obtained at different times. A
second methodological problem is an inability to
match respondents from one administration to the
next. Unmatched respondents create pseudo-inde-
pendent raters for each administration, and thus it is
only possible to examine agency trends, and not
change per se. Moreover, such analyses create more
conservative tests, because individual response biases
cannot be taken into account. In addition, the lack of
a control group is a design flaw, leaving open the
possibility that unknown external factors influenced

37

employee perceptions, either positively or negatively,
in addition to the human resource initiatives (e.g.,
changing retirement systems and furloughs).

Although methodological problems exist for the
present study, there are also several possible substan-
tive explanations for why ATCS perceptions have not
changed substantially. It is possible that the interven-
tions were not adequately developed, implemented,
communicated, or that the interventions targeted
inappropriate organizational dimensions or groups.
For example, the feedback from surveys was primarily
directed towards management, with management
then developing action plans based on the data.
Employees generally received only feedback summa-
ties, and the degree of employee involvement in
action planning varied regionally. Similarly, the pay
demonstration project was not generalized to the
entire workforce. Also, SIDP was eventually stopped
because ratings of peers were being conducted quid
pro quo, making the rating an ineffective predictor of
supervisory skills. Another possible explanation is
that the survey methodology used to assess changes was
insensitive to the perceptual changes that did occur.

Over the course of rebuilding the ATCS workforce,
the influx of new controllers and the human resources
initiatives lead to small improvements in the per-
ceived organizational climate. Small changes, as op-
posed to massive shifts, and some setbacks, are to be
expected because substantially changing an organiza-
tion, especially a large organization, is difficult and
time consuming. Because changes take place over
time continuous and repeated efforts are needed. Of
course, efforts to improve the FAA and the work
environment of ATCSs continue (e.g., FAA, 1996;
General Accounting Office, 1996). Specifically, re-
cent changes in personnel and acquisition reform are
expected to aid in the selection of new employees and
ease the purchase and placement of new technologies,
both of which should contribute to enhanced climate
perceptions of the ATCS workforce.


CHAPTER 2:
AIR TRAFFIC CONTROL SPECIALIST APTITUDE TESTING, 1981-1992

Dana Broach
FAA Civil Aeromedical Institute, Oklahoma City, Oklahoma

Between 1981 and 1992, entry into the demanding air traffic control specialist (ATCS) occupation in the Federal

Aviation Administration (FAA) was determined by applicant performance on a written aptitude test battery. That
ATCS aptitude test battery was administered over 400,000 times between 1981 and 1992. The reliability, validity,

and technical fairness of the test battery are evaluated in this paper. Available data indicated that the tests comprising

the battery had acceptable reliability. A composite score on the test battery predicted performance in initial ATCS

qualification training at the FAA Academy, but did not predict outcomes in field training. Evidence for differential

prediction of FAA Academy outcomes on the basis of race and gender was found in the technical fairness analyses. However,

the validity evidence supported using the battery as a mass testing tool by the FAA in rebuilding the controller workforce.

The Air Traffic Control Specialist (ATCS) occu-
pation is the single largest (about 17,000) and most
publicly visible occupational group in the Federal
Aviation Administration (FAA). Air traffic control-
lers are at the heart of a web of radars, computers, and
communication facilities that comprise an increas-
ingly complex and busy air transportation system.
Appropriate selection of personnel into these critical
positions is an important human factors problem.
From 1981 through 1992, the FAA used noncom-
pensatory, multiple hurdles to select people into the
ATCS occupation. The first hurdle was a written
aptitude test battery administered by the U.S. Office
of Personnel Management (OPM; Aul, 1991, this
volume). Between 1981 and 1992, the test was ad-
ministered over 400,000 times. Only 25,277 appli-
cants were selected on the basis of their test scores to
attend the FAA Academy initial controller training
program, the second major hurdle in the FAA’s
selection process. The purpose of this paper is to
describe the test battery and evaluate its psychometric
characteristics.

Description of the ATCS Written Test Battery
The aptitude test battery used from 1981 to 1992
was developed in response to the inadequacies of a
more traditional civil service battery that was in use
from 1964 to 1981. The effectiveness of the 1964
civil service battery was questioned as early as 1970,

in the final report of the Air Traffic Controller Career
Committee (Corson, Berhard, Catterson, Fleming,
Lewis, Mitchell, & Ruttenberg, 1970). A review of
controller training in the mid-1970s concluded that
improved selection methods could have substantial
monetary implications for the agency (Henry,
Ramrass, Orlansky, Rowan, String, & Reichenbach,
1975). A subsequent technical study of controller
selection tests found that the 1964 test battery was, at
best, marginal in predicting the job performance of
controllers (Milne & Colmen, 1977). Cooperative
research by scientists from the FAA Office of Avia-
tion Medicine (Dailey & Pickrel, 1984a, b), Civil
Aeromedical Institute (CAMI; Collins, Boone, &
VanDeventer, 1984), and OPM (Rock, Dailey, Ozur,
Boone, & Pickrel, 1982) resulted in the develop-
ment, validation, and implementation of a new writ-
ten ATCS aptitude test battery in 1981 following the
strike by members of the Professional Air Traffic
Controller Organization (PATCO).

Three tests comprised the new 1981 test battery:
the Multiplex Controller Aptitude Test (MCAT),
the Abstract Reasoning Test (ABSR), and the Occu-
pational Knowledge Test (OKT). The MCAT was a
timed, paper-and-pencil test simulating activities re-
quired for control of air traffic. Aircraft locations and
direction of flight were indicated with graphic sym-
bols on a simplified, simulated radar display (Figure
2-1); an accompanying table provided relevant


Chapt. 6: Current FAA Controller Workforce Demographics, Future Requirements, and Research Questions

--CWF >Av Activity

0 ATCS Staffing (Thousands) Aviation Ops (Millions) 200
or
: —o-—a | 180
10 100
5 50
0
1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002
Fiscal Year

Figure 6-2: Projected controller workforce (CWF) vs. aviation activity (1982-2002).

49


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

Myers, J. (1992). A longitudinal examination of appli-
cants to the air traffic control Supervisory Identifica-
tion and Development Program. (DOT/FAA/AM-
92/16). Washington, DC: Federal Aviation Ad-
ministration, Office of Aviation Medicine. (NTIS
No. ADA251879)

National Transportation Safety Board. (1981). Air traf-
fic control system. (NTSB-SIR-81-7). Washing-
ton, DC: Author.

Nickels, B.J., Bobko, P., Blair, M.D., Sands, W.A., &
Tartak, E.L. (1995). Separation and Control Hir-
ing Assessment (SACHA) final job analysis report.
(Deliverable Item 007A under FAA contract
DTFA01-91-C-00032). Washington DC: Federal
Aviation Administration, Office of Personnel.

Rhodes, S.R. (1983). Age-related differences in work
attitudes and behavior: A review and conceptual
analysis. Psychological Bulletin, 93, 328-67.

Rock, D.B., Dailey, J.T., Ozur, H., Boone, J.O., &
Pickrel, E.W. (1982). Selection of applicants for the
air traffic controller occupation. (DOT/FAA/AM-
82/11). Washington, DC: Federal Aviation Ad-
ministration Office of Aviation Medicine. (NTIS
No. ADA122795/8)

Rock, D.B., Dailey, J.T., Ozur, H., Boone, J.O., &
Pickrel, E.W. (1984a). Validity and utility of the
ATC experimental tests battery. Study of Acad-
emy trainees, 1982. In S.B. Sells, J.T. Dailey, 8
E.W. Pickrel (Eds.), Selection of air traffic control-
lers (pp. 459-502). (DOT/FAA/AM-84/2). Wash-
ington, DC: Federal Aviation Administration
Office of Aviation Medicine. (NTIS No.
ADA147765)

Rock, D.B., Dailey, J.T., Ozur, H., Boone, J.O., &
Pickrel, E.W. (1984b). Conformity of the new
experimental test battery to the Uniform Guide-
lines on Employee Selection Requirements. In
S.B. Sells, J.T. Dailey, & E.W. Pickrel (Eds.),
Selection of air traffic controllers (pp. 503-542).
(DOT/FAA/AM-84/2). Washington, DC: Fed-
eral Aviation Administration Office of Aviation

Medicine. (NTIS No. ADA147765)

Sackett, P.R., & Wilk, S.L. (1994). Within-group
norming and other forms of score adjustment in
pre employment testing. American Psychologist,
49, 929-54,

56

Salthouse, T.A. (1984). Effects of age and skill in
typing. Journal of Experimental Psychology: Gen-
eral, 23, 521-25.

Salthouse, T.A. (1990). Influence of experience on age
differences in cognitive functioning. Human Fac-

tors, 32, 551-69.

Salthouse, T.A., Babcock, R.L., Skovronek, E., Mitchell,
D.R.D., & Palmon, R. (1990). Age and experi-
ence effects in spatial visualization. Developmen-

tal Psychology, 36, 128-36.

Schaie, K.W. (1988). Ageism in psychological research.
American Psychologist, 43, 179-83.

Schmidt, F.L. (1988). The problem of group differ-
ences in ability test scores in employment selec-
tion. Journal of Vocational Behavior, 33, 272-92.

Schneider, B., & Konz, A.M. (1989). Strategic job
analysis. Human Resource Management, 28, 51-3.

Schultz, S.R., & Marshall-Mies, J. (1988). FAA ATCS
recruitment: Assessment of current procedures and
development of recommendations for an integrated
program. (Final report under OPM work order
342-053). Washington, DC: Federal Aviation
Administration, Office of Human Resources
Management.

Sells, S.B., Dailey, J.T., & Pickrel, E.W. (Eds.) (1984),
Selection of air traffic controllers. (DOT/FAA/AM-
84/2). Washington, DC: Federal Aviation Ad-
ministration Office of Aviation Medicine. (NTIS
No. ADA147 765)

Seymour, R.T. (1988). Why plaintiffs’ counsel chal-
lenge tests, and how they can successfully chal-

lenge the theory of “validity generalization.” Jour-
nal of Vocational Behavior, 33, 331-64.

Sharit, J. & Czaja, S.J. (1994). Aging, computer-based
task performance, and stress: Issues and chal-
lenges. Ergonomics, 37, 559 - 577.

Thorndike, R.L. (1949). Personnel Selection. New York:
Wiley.

Trites, D.K. (1963). Ground support personnel. Aero-
space Medicine, 34, 539-41.

Trites, D.K. & Cobb, B.B. (1964a). Problems in air
traffic management: III. Implications of training-
entry age for training and job performance of air
traffic control specialists. Aerospace Medicine, 35,
336-40.


Chapter 2: Air Traffic Control Specialist Aptitude Testing, 1981-1992

(Equal Employment Opportunity Commission,
1978), refers to the degree to which test scores used
for making employment decisions are predictive or
correlated with important and/or critical work out-
comes, elements, or behaviors. The ATCS aptitude
test battery has been validated against two classes of
work outcomes in previous studies: (a) performance
in the FAA Academy initial training programs (SCREEN;
Della Rocco, this volume); and (b) outcomes of on-
the-job field training (OJT) at the first assigned field
facility (srarus; Manning, this volume).

As the final civil service RATING was the basis of
operational personnel selection decisions, this retro-
spective analysis examined the validity of the final
RATING as a predictor of these two criteria for the
15,876 controllers who survived the FAA Academy
and were placed into field training between 1981 and
1992. The field training sTaTus criterion was coded
as follows: 1 = Reached Full Performance Level; 2 =
Still in Training (Developmental); 3 = Switched facili-
ties; 4 = Switched options; and 5 = Failed. As recom-
mended by Manning, persons who had attrited for
reasons other than performance were excluded from
the validity analysis. Because the samples had been
truncated by selection first, on RATING, and second,
by selection on FAA Academy score (SCREEN), correc-
tions for restriction in range were made as shown in
Table 2-2, using the formulae presented by Ghiselli,
Zedeck, and Campbell (1984). Correlations between
civil service RATING and FAA Academy SCREEN scores
were corrected for direct restriction in range due to
explicit selection on RATING. The corrections were
based on the population standard deviations pre-
sented in the Tables 2-3 through 2-6. Correlations
between FAA Academy (scREEN) and field training
outcomes (STATUS) were corrected for direct restric-
tion in range due to explicit selection on the FAA
Academy score. Finally, correlations between civil
service RATING and field training outcomes were cor-
rected for incidental restriction in range due to selec-
tion of the samples on FAA Academy SCREEN scores.
The uncorrected, zero-order correlations are pre-
sented in the lower left corner of each correlation
matrix and the corrected correlations in the upper
right corner. The corrected correlation matrices for
each sample were submitted to regression analysis to
estimate the validity of RATING as a predictor of FAA
Academy screen and field training sTaTus.

In view of the iterations of the FAA Academy
initial training programs, described by Della Rocco,
and differences in field training as described by
Manning, the sample was divided into four groups:
(a) 1982-85 FAA Academy Terminal program gradu-
ates assigned to training in the terminal option (Table
2-3); (b) 1982-85 FAA Academy En route program
graduates assigned to field training in the en route
option (Table 2-4); (c) 1986-92 FAA Academy Non-
radar Screen assigned to field training in the terminal
option (Table 2-5); and (d) 1986-92 FAA Academy
Non-radar Screen graduates assigned to field training
in the en route option (Table 2-6).

On one hand, this retrospective analysis found
that the final civil service RaTING had acceptable
validity as a predictor of performance in initial train-
ing at the FAA Academy for all four groups. The
uncorrected correlations between RATING and SCREEN
were statistically significant and ranged from .178 to
-222 across the four groups. The corrected correla-
tions between civil service RATING and FAA Academy
SCREEN ranged from .458 to .502. On the other hand,
regression analyses of the corrected correlation matri-
ces found that RATING was a relatively poor predictor
of field training status in three of the four FAA
Academy and field training variations (Table 2-7).
The standardized regression weight (B) for RATING
was not statistically significant for graduates of 1982-
85 Terminal Screen program assigned to field train-
ing in the terminal option (B = -.025, ns) or for
1986-92 Non-radar Screen program graduates as-
signed to either terminal (8 = -.041, ms) or en route (B
=-.010, ns) field training. However, RATING did enter
into the regression equation predicting sTaTus in en
route field training, with a standardized regression
weight of -.107 (p <.001), for graduates of the 1982-85
En Route Screen program, as shown in Table 2-7.

Fairness

The FAA, in its 1993 Diversity Plan, made a
commitment to attract, retain, develop, and manage
a diverse work force that visibly reflected the Ameri-
can population at large by the year 2000. Achieving
this goal will require substantial changes in the demo-
graphic profile of the ATCS occupation. Air traffic
control is a career field in which female and minority
workers have been historically under-represented rela-
tive to the American population at large. The techni-


Technical Documentation Page

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No.
DOT/FAA/AM-98/23

4, Title and Subtitle 5. Report Date

Recovery of the FAA Air Traffic Control Specialist Workforce, August 1998

1981-1992 6. Performing Organization Code

7. Author(s) 8. Performing Organization Report No.
Dana Broach, Editor

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS)

FAA Civil Aeromedical Institute

P.O. Box 25082 11, Contract or Grant No.
Oklahoma City, OK 73125

12. Sponsoring Agency Name and Address. 13. Type of Report and Period Covered
Office of Aviation Medicine

Federal Aviation Administration

800 Independence Avenue, S. W.

Washington, DC 20591

15. Supplementary Notes
This research was conducted under task AM-97-B-HRR-509

16. Abstract

The Federal Aviation Administration was confronted in 1981 with the challenge of rebuilding its core,
technical, and highly-trained air traffic control specialist (ATCS) workforce following the PATCO strike. From
late 1981 through mid-1992, the FAA rebuilt this critical workforce through a large-scale testing, screening and
training program. By mid-1992, recovery of the controller workforce was complete, and it was no longer necessary
for the FAA to conduct a large-scale hiring program. The six papers presented in this report represent the first
major retrospective analysis of the complete data set describing the recovery of the FAA's en route and terminal
ATCS workforce following the 1981 controller strike. The first paper describes the personnel processes, focusing
on recruitment and hiring programs for the en route and terminal options. The second paper presents a detailed
description of the aptitude test battery used to evaluate over 400,000 applicants between 1981 and 1992. The third
paper offers a definitive statistical portrait of the FAA Academy Screening programs as predictors of field training
outcomes. On-the-job training (OJT) programs in en route and terminal facilities are described in the fourth
paper. These four papers, taken together, provide a definitive description of the processes used to recruit, test,
screen, and train persons for the ATCS occupation between 1981 and 1992. The fifth paper draws on FAA
organizational survey data to describe controller perceptions of the organizational climate in which the workforce
recovery occurred. The sixth paper analyzes current controller workforce demographics and technological trends in
air traffic control to identify potential areas of future research.

17. Key Words 18. Distribution Statement
Air Traffic Control Specialist (ATCS) Selection; Document is available to the public through
Training; Personnel, FAA; Psychology; Organizational | the National Technical Information Service,
Climate; Demographics Springfield, VA 22161
79. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22, Price
UNCLASSIFIED UNCLASSIFIED 62

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized



Recovery of the FAA Air Traffic Control Workforce, 1981-1992

Table 4-5: En Route field training status by demographic variables

En Route field training status"

FPL Dev Term Transfer FSS Transfer Fail
N % N % N % N % N So
Gender
Male 5,292, 71.4 34 5 575 7.8 476 6.4 1,031 13.9
Female 871 72.5 14 1.2 107 8.9 63 5.2 147° 12.2
Minority Status’
Amer. Indian 28 77.8 2 5.6 3 8.3 3 8.3
Asian-Pac. 60 84.5 3 42 1 14 7 9.9
African-Amer. 162 63.3 7 2.7 24 9.4 29° 11.3 34 13.3
Hispanic 160 71.4 1 A 17 7.6 8 3.6 38 «617.0
Non-minority 5,630 71.6 40 5 622 7.9 492 6.3 1,078 13.7
Entry Type®
Competitive 5,472 72.9 38 5 557 7A 473 6.3 971 12.9
Airway Science 31 79.5 3 7.7 1 2.6 4 10.3
Pre-Devel. 76 54.7 1 7 22 15.8 11 7.9 29° 20.9
Coop. Ed. 191 70.0 4 15 25 9.2 10 3.7 43 15.8
ATA 167 60.7 4 1.5 46 16.7 9 3.3 49 17.8
VRA 29° 63.0 1 2.2 2 43 4 8.7 10 21.7
Noncompetitive 96 61.5 10, 6.4 12 7.7 38 24.4
Former ATC 17 68.0 1 4.0 3 12.0 4 16.0
Former GS-9 1 100.0
Education‘

No coll. degree 3,752 74.0 31 6 400 7.9 267 5.3 620 12.2
College degree 2,367 68.8 175 274 8,0 238 6,9 542 15.8

Note: *Percents are row percents.
“Minority status: Amer. Indian, American Indian/Alaskan. Asian-Pac., Asian-Pacific Islander. “Entry:
Airway Science, graduate of accredited Airway Science program. Pre-Devel., participated in FAA
Pre-Developmental Program. Coop. Ed., Cooperative Education Program. ATA, former Air Traffic
Assistant (GS-2154 or GS-2102.) VRA, hired under Veterans Readjustment Act. Former ATC,
previously graduated from FAA Academy and failed field training. Former GS-9, previously hired
under GS-9 military hire program and failed field training.
“Education: No coll. degree includes high school graduates and candidates with some college hours
but no degree. College degree includes those with a bachelor’s or higher degree.

30


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

occupation. These screening courses were the equiva-
lent of a miniaturized training-testing model of selec-
tion (Manning, Kegg, & Collins, 1989; Della Rocco,
Manning, & Wing, 1990).

The Terminal and En Route Screen programs con-
tained two pass/fail phases covering (1) an introduction
to aviation and air traffic control and (2) procedures for
non-radar control of air traffic in the airspace relevant to
each option (Boone, 1984). The first pass/fail segment
was strictly academic and covered topics such as prin-
ciples of flight, meteorology, air traffic service and the
National Airspace System. However, it was the non-
radar procedures phase of training that identified the
candidates with less chance of succeeding in the occupa-
tion. The non-radar phase for both options included
academics, laboratory simulation problems, and the
Controller Skills Test (CST). During the academic
classroom training, entrants were taught the rules and
principles of non-radar ATC. Students received a score
from this academic training which was included in the
final non-radar phase score. During the laboratory
training, entrants were required to apply the rules and
principles learned in the classroom to air traffic simula-
tion problems. Each entrant performed non-radar con-
troller job tasks in one or two thirty-minute scripted
scenarios daily. An individual instructor was assigned to
each student for each scenario. Following the scenario,
the instructor reviewed the student’s performances and
provided feedback on errors as well as correct applica-
tion of the rules and scenario scores. Instructors as-
signed a technical assessment (number of errors), as well
as an instructor assessment (subjective evaluation of
student performance) for each scenario. The laboratory
scenarios were structured to escalate in complexity. Five
of six of the final laboratory problems were graded and
counted in the final score. The CST was the final test
administered in the non-radar phase. It was a timed,
paper-and-pencil, multiple-choice assessment of the
student’s ability to apply non-radar ATC rules and
procedures. A final composite score of at least 70 (out of
a possible score of 100) was required to pass the non-
radar phase. Failure to achieve a composite score of 70
resulted in removal from the ATCS job. Although the
En Route and Terminal Screen programs were modified
over the years, the basic structure remained much the
same between 1976 and 1985.

At the time of the strike in 1981, the En route and
Terminal Screen programs had effectively achieved the
goal of reducing field attrition (Manning, 1991). Be-
tween 1971 and 1975, field attrition in both options

averaged 41%. Following the implementation of the
screening programs between 1976 and 1981, FAA Acad-
emy attrition rates were 29% and subsequent field
attrition was cut to 8%. Thus, within the first few
months of appointment to the occupation, the FAA
Academy programs screened out most of those indi-
viduals who lacked sufficient potential to become fully
certified ATCSs. The pass rate was 67% for En Route
Screen entrants and 74% for Terminal, with an overall
average of 70%. Of historical note, subsequent to the
strike, a new Office of Personnel Management (OPM)
test battery was implemented, replacing the previous
CSC battery (Broach, this volume; Collins, Boone, &
VanDeventer, 1984).

A key feature of the 1976 effort was to implement a
research database system to document and monitor the
effectiveness of the program. The database was estab-
lished at the Civil Aeromedical Institute (CAMI). Data
were collected on all FAA Academy entrants. Planning
for the second-stage pass/fail screening programs was
influenced by requirements identified by publication of
the original Uniform Guidelines on Employee Selection
Procedures in 1966 (Boone, 1984). The database pro-
vided a source for monitoring compliance with those
guidelines. In 1984, the CAMI tracking database was
established to track FAA Academy graduates through
Full Performance Level (FPL; Manning, this volume).
This database provided criterion measures for program
validation.

Strike recovery, 1981-1985

By the time of the strike in 1981, the FAA had five
and a half years of experience using the En Route and
Terminal initial qualification courses as screening pro-
grams. In addition, data on all entrants had been col-
lected in the research database. Thus, the system was in
place to select and monitor the progress of ATCS
candidates, including individuals who had no knowl-
edge of the occupation. The FAA Academy only needed
to gear up to manage the increased student load result-
ing from the strike recovery effort. Prior to the strike,
8,262 candidates (4,138 En Routeand 4,124 Terminal)
entered the programs in the approximately five and a
half year period between January 1976 and August 1981
(Manning, Kegg, & Collins, 1988). Following the
strike, 13,533 candidates entered during the four-year
period between August 1981 and September 1985. The
number of En Route Screen program entrants more
than doubled to 8,536, while the number of Terminal
Screen program entrants rose approximately 20% to


NOTICE

This document is disseminated under the sponsorship of
the U.S. Department of Transportation in the interest of
information exchange. The United States Government
assumes no liability for the contents thereof.




Recovery of the FAA Air Traffic Control Workforce, 1981-1992

but were slightly more likely to be transferred to an FSS
(about 11%) than were non-minority trainees (6%).
Other minority groups had rates of reaching FPL status
that were the same or better than were those for non-
minorities. FAA Academy graduates who were non-
competitive and Pre-Developmental entrants were less
likely to reach FPL status (about 62% and 55%, respec-
tively) than were competitive entrants (about 73%).
FAA Academy graduates from the Airway Science pro-
gram had a higher rate of reaching FPL (95%) than did
competitive entrants but their overall numbers were
low. Those with any type of college degree were some-
what less likely to become FPL (81%) than were those
without a college degree (86%).

Table 4-6 shows similar statistics for terminal train-
ees. Females were somewhat less likely to reach FPL
status (80%) in a terminal facility than were males
(about 85%) and were more likely than males to switch
facilities within the terminal option (11% as compared
with 7%). Hispanic (80%), and to a greater extent,
African-American trainees (77%) were less likely than
were non-minority trainees (84%) to reach FPL status.
Of those who failed to reach FPL status, Hispanics were
more likely to be reassigned to another terminal facility
(about 13%) than were non-minorities (about 8%) and
African Americans (13%) were more likely to fail and
separate from the occupation than were non-minorities
(about 7%). FAA Academy graduates who went through
Airway Science programs were more likely to reach FPL
(95%) than were competitive entrants (85%). Non-
competitive entrants (75.5%) and Pre-Developmental
entrants (72%) were less likely than were competitive
entrants to reach FPL status. Again, those with a college
degree were less likely to reach FPL status (81%) than
were those without a college degree (86%).

Time Until Occurrence of Training Attrition

Table 4-7 shows the average time for attrition from
training to occur by option and type of FAA Academy
program. Training attrition at the first facility occurred
between about 1 year and more than 2 ¥ years after FAA
Academy graduation. It appears that en route trainees
who failed to complete training were retained at their
facility longer than were terminal trainees. However, the
categories included in the Type of Attrition variable
were not the same for the two options so a direct
comparison is not appropriate. The amount of time
until attrition from training in the terminal option was

26

the same or shorter for Non-radar Screen graduates than
for graduates of the Terminal Screen for all types of
attrition except failure. On the other hand, the amount
of time to attrition from training in the en route option
was consistently longer for Non-radar Screen graduates
than for graduates of the En Route Screen program,
regardless of type of attrition.

Conclusions

The administration of ATCS field training programs
was affected by changes that occurred during the years
after the 1981 controller strike. In the early 1980s, there
was an influx of new trainees that tested the capabilities
of the training system. Not only did a large number of
trainees need to use the limited lab space at the same
time, there were also few instructors available to provide
OJT. The numbers of FAA Academy graduates entering
field training reduced after 1983, but it took several
years for the large group of 1981-83 trainees to make it
through the bottlenecks in the training system to com-
plete training. It is likely that FAA Academy graduates
who entered training later received more attention than
did those who entered training immediately after the
strike, Data regarding the length of time until attrition
occurred and rates of reaching FPL status support this
interpretation.

A more efficient second-stage selection procedure
was introduced at the FAA Academy in the fall of 1985.
Atthesame time, programs to recruit under-represented
groups resulted in increases in the percentages of women
and minorities entering field training. Women suc-
ceeded in training in about the same proportion as men,
while minorities had somewhat lower rates of reaching
FPL than did non-minorities. The overall result should
be an increase in the diversity of the ATC workforce.
Changes were made in the content of en route field
training programs after about 1985. These changes
resulted in a training program that better targeted
specific skills performed on the job.

The percentage of trainees reaching FPL status in-
creased for those entering field training during the late
1980s. Because so many changes occurred in both
selection procedures and training programs at about the
same time, we cannot determine why the success rates
increased. However, the data suggest that the FAA
successfully accomplished the requirement to train a
substantial number of new air traffic controllers to
replace the fired controllers.


Chapter 5: Organizational Climate Perceptions of the ATCS Workforce, 1981-1992

hires, however, were placed into the same organiza-
tional climate that had created the circumstances that
led to the original strike.

A final program, the Supervisor Identification and
Development Program (SIDP; Myers, 1992), was
created to address inadequate supervisor/subordinate
communication. The SIDP was created initially for
the air traffic, electronics technician, and aviation
safety inspector workforces. The purposes of SIDP
were to enhance the quality of supervisors selected for
internal advancement from the workforce, and to
provide improved training in human relations super-
visory skills, At the same time, the FAA’s Center for
Management Development changed its courses to
support this effort. Again, the Jones report had indi-
cated that one of the FAA’s problems was the way that
management, and to a lesser extent, first and second
line supervisors, treated subordinates. SIDP con-
sisted of a self-selection of candidates, who were then
rated by peers and supervisors on four performance
dimensions: interpersonal, communications, direc-
tion and motivation, and technical competence. Can-
didates were then given a skill-based interview. From
there, candidates were considered to be eligible for
promotion consideration, given further review, or
asked to enhance their leadership skills and reapply
(Myers, 1992).

Given the continuing efforts of the FAA to im-
prove the organizational climate, and the influx of
new organization members following the strike, it
seems prudent to assess any changes in the percep-
tions of agency employees, and in particular, the
perceptions of air traffic controller specialists (ATCS).
The four national initiatives, the biennial survey,
SFA, the Pay Demonstration Project, and SIDP,
coupled with the influx of new employees, may have
improved the organizational climate of the FAA.
Indeed, the four national human resource initiatives
were developed specifically to enhance the FAA’s
organizational climate. The human resource initia-
tives undertaken during the recovery of the ATCS
workforce have their roots in the conclusions drawn
by the Jones Report. The extent to which these
projects had an effect on employee perceptions will
be examined next.

35

Method.

Participants and procedures

Biennial surveys of varying content were adminis-
tered to FAA employees on four occasions (1984,
1986, 1988, and 1990). The first two administra-
tions consisted of a census of all agency employees.
The latter two administrations consisted of a strati-
fied random sample of approximately 15% of the
agency. This sample consisting of air traffic control-
lers (Tower, En route, or Flight Service Station per-
sonnel) and supervisors was drawn from the larger
samples of the FAA. The ATCS samples for each
administration were: 12,478; 14,222; 2,106; and
2,485, respectively.

Measures

Five organizational climate dimensions assessing
content areas similar to those identified as problem
areas by Jones et al. (1982) were examined. Each
dimension was assessed with the same items for each
survey administration, except for job satisfaction and
equity of pay and benefits. The initial survey in-
cluded a single item measuring job satisfaction. This
single item job satisfaction measure has been used as
a benchmark of job satisfaction since 1984. The three
subsequent surveys included additional satisfaction
measures (satisfaction with pay, benefits, physical
working conditions, work group, supervisor, and
organization) that were averaged to form a satisfac-
tion index (average 0 = .77). The single job satisfac-
tion item and the satisfaction index provide two
measures of overall morale in the agency. Such satis-
faction measures are commonly used as outcome
measures in studies of organization change (Cranny,
Smith, & Stone, 1992). The second measure focuses
on perceptions of management's concern for employ-
ees, which was assessed with 3 items (average 1 = .89)
from Jones et al. (1982). Management consideration
examines the extent to which management takes into
account the effects of decisions on employees. Em-
ployee perceptions of pay and benefit equity (formal
rewards) were initially assessed with two items (e.g.,
pay and job security equity) in 1984. Two additional
items (e.g., retirement and benefits equity, average &
= .74) were incorporated in subsequent surveys. Per-
ceptions regarding the way work was organized were


Chapter 3: FAA Academy Air Traffic Control Specialist Screening Programs & Strike Recovery

100

90 7

80 7

70 5

60 4

50 -
70-74 75-79 80-84 85-89 90-100
FAA Academy Score

Figure 3-1: Percent achieving en route FPL field status by FAA Academy score

100

70-74 75-79 80-84 85-89 90-100
FAA Academy Score

Figure 3-2: Percent achieving terminal FPL field status by FAA Academy score

21


Chapter 1: Employing Air Traffic Controllers, 1981-1992

subset from the two 187-item parallel forms of the
Sixteen Personality Factor Questionnaire (16PF; Cattell,
Eber, & Tatsuoka, 1970) was intended to operate as an
indicator of emotional instability. Candidates flagged
by the 38-item scale were referred for psychiatric and
psychological evaluation.

Finally, the candidates had to pass a background
investigation. This investigation was used for two pur-
poses. One was to determine if the person would receive
a security clearance. The other was to determine the
person’s general suitability for federal employment.
Individuals who had been found guilty of crimes, who
had been discharged from the military under less than
honorable conditions, who had been fired with cause
from previous employers, or who had been convicted of
any alcohol or drug-related offense had great difficulty
being cleared under this process. The FAA also looked
with caution on people who had repeated minor viola-
tions, such as multiple speeding tickets.

Large numbers of people were processed under
these procedures. At peak, nearly 50,000 people were
tested in a year. During the first years after the strike,
the FAA’s personnel offices were required to recruit
and hire specific numbers of people for each new class
date at the FAA Academy. These numbers were set
ambitiously high; initially, therefore, a few people
with test scores as low as 70 were sent to the FAA
Academy. Over time, the FAA was able to recruit and
hire people with increasingly higher test scores. Dur-
ing the late 1980s until 1992, the minimum hiring
scores varied from about 85 to about 95.

Post-hire screening. Once someone met the basic
eligibility criteria, passed the OPM test, was medi-
cally cleared, passed a background investigation, and
was interviewed, the person was usually hired. How-
ever, the selection process did not stop there. Select-
ees reported to the FAA Academy screening program
in Oklahoma City, Oklahoma (Della Rocco, this
volume). The individuals were officially appointed
into the civil service at the FAA Academy, becoming
paid employees of the FAA. They spent the first
several weeks of their employment in the FAA Acad-
emy screening program. The FAA Academy program
was considered to be initial training, but was con-
ducted with an emphasis on screening out those
unlikely to be successful in further FAA controller
training (Boone, 1984). The last version of the screen-
ing program was approximately nine weeks long.
Those who failed the FAA Academy program usually
had their employment terminated. The pass rate

varied, but was about 60 percent for most of the
period it operated. On average, about 30 percent
failed outright, and another 10 percent withdrew
from the program. This meant that nine weeks after
starting with the FAA, approximately 40 percent of
the new hires might no longer be employed with the
agency.

Re-engineering the selection process. The process
described up to this point was quite lengthy. In 1988, it
was estimated to take on average 18 months to hire an
entry level controller to send to the FAA Academy. In
particular, it might take several months to obtain a
background check on someone or to medically clear
someone who required a special review of his or her
medical situation. Asa result, in the late 1980s the FAA
embarked on what today would be called a “reengineering
project.” It successfully reengineered the hiring process
for entry level controllers and effectively reduced the
time to hire from 18 months to about six weeks. This
was, however, late in the post-strike recovery period and
many people still experienced delays due to existing
backlogs of work. There was another complicating
factor in the hiring process. A number of companies
discovered a market for teaching people to take the
OPM test. These companies aggressively marketed their
services and, as a result, may have brought many more
applicants to the FAA. Unfortunately, we also know
that the more timesa person takes the test, the lower the
relationship between test score and success in the FAA
Academy screening program (Broach, Young, & Farmer,
1994; Collins, Nye, and Manning, 1990). From evi-
dence like this, we concluded that the test was subject to
practice effects. The graduates of these training pro-
grams may have had inflated scores on the test, and this
could have contributed to some increases that were seen
in the failure rates at the FAA Academy in the late 1980s.

Noncompetitive processes

So far, only the conventional, competitive hiring
process has been described. Other legitimate, but less
well- known processes were also used. One such method
was cooperative education. The FAA formally recruited
students at colleges in certain majors, such as aviation,
mathematics, geography, and engineering. As employ-
ees in the FAA’s cooperative education program, the
students alternated periods of training to be controllers
at the FAA with periods of study in college. College
graduates who met all the requirements described pre-
viously, and completed the cooperative education train-
ing program, could be assigned to positions as entry




CHAPTER 6:

CURRENT FAA CONTROLLER WORKFORCE DEMOGRAPHICS, FUTURE
REQUIREMENTS, AND RESEARCH QUESTIONS

David J. Schroeder, Dana Broach, and William L. Farmer
FAA Civil Aeromedical Institute, Oklahoma City, Oklahoma

Between 1981 and 1992, the Federal Aviation Administration replaced approximately 11,500 air traffic
controllers in the wake of the 1981 Professional Air Traffic Controller Organization (PATCO) strike. Three issues
are considered in this paper: the aging controller workforce after the year 2000, training for advanced air traffic control

technologies with an aging workforce, and recruitment and selection criteria for future controllers. Ongoing research

to address these issues are described.

Following the Professional Air Traffic Controller
Organization (PATCO) strike of 1981, the Federal
Aviation Administration (FAA) was tasked with re-
placing approximately 10,400 air traffic controllers.
As the point of retirement eligibility for the first of
these replacement controllers approaches, a consider-
able amount of resources will need to be invested in
planning for and initiating recruitment, selection,
and training programs for the inevitable influx of new
controllers. As a result, the FAA faces a number of
personnel-related research and operational questions.
This paper focuses on three issues:

+ Aging of the Air Traffic Control Specialist (ATCS)
workforce;

¢Employee training for the advanced National
Airspace System (NAS) air traffic control (ATC)
system technologies; and

*The recruitment and selection of personnel to
operate the more automated ATC system of the
future.

Earlier papers in this volume focused on the pro-
cesses and tools used to rebuild the controller
workforce after the 1981 strike. But as the present
workforce ages, and inexorably moves toward retire-
ment eligibility, the issue of future selection to re-
plenish the ATC system becomes critical. Combined
with this are the soon to be changing demands im-
posed by the evolving NAS (FAA, 1996). A large
amount of research has gone into defining the knowl-
edge, skills, and abilities of the ideal air traffic

43

controller. As system dynamics and technologies
change, many of those essentials that define perfor-
mance in today’s system may no longer be as impor-
tant in tomorrow’s job, With this in mind, current
efforts are focused on defining the new system and
the skill requirements of the future controller
workforce.

Aging of the ATCS Workforce

The vast majority (87.9%) of the current ATCS
workforce entered the FAA following the PATCO
strike of 1981. To better understand the characteris-
tics of the ATC population, we extracted information
regarding its age and experience distributions from
the Consolidated Personnel Management Informa-
tion System (CPMIS) maintained by the FAA (Fig-
ure 6-1). The sample of 14,569 ATCSs (as of April
1996) does not include personnel who were listed in
ATCS office, staff, or supervisory/management posi-
tions. Since a significant portion of the workforce
was hired in the decade following the ATC strike of
1981, we found the expected bulge of controllers
(62.7%) in the 30-39 age range. The average age of
the current workforce is 37, with a standard deviation
of 6.38 years. Of this sample of 14,569 active con-
trollers, 11.3% of the controllers were 40 years of age
and older, and they had at least 15 years’ experience.
The percentage of the post-strike group that will fall
in that same category in 2001 increases to 40%. By
2011, nearly the entire population of the current
workforce will either be in that group, or will have
already retired.


Chapt. 6: Current FAA Controller Workforce Demographics, Future Requirements, and Research Questions

restrictive (18-30) than that studied by Cobb and
associates, demonstrated the typical inverse relation-
ship between age and pass rate. Figure 6-4 presents
the outcomes in the FAA Academy screening pro-
grams for 27,925 entrants from 1981 through 1992.
Overall success in the FAA Academy declined from
66.6% of those age 22 and younger to 53.7% of those
aged 27-28, and 48.6% of those in the 31 or older age
category. Age-related changes were evident in both
the failure rates as well as withdrawals. Lowered
success rates were also evident in field training. Out-
comes for the FAA Academy graduates initially as-
signed to an en route ATC facility for training are
presented in Figure 6-5. To be consistent with
Manning’s (this volume) analysis of OJT outcomes,
controllers still in training, or who left training for
reasons other than performance, were excluded from
this analysis. While 80.7% of those age 22 or younger
successfully reached the full performance level at
their first en route facility, only 52.8% of those in the
oldest age group (31+) did. From the younger through
older age groups, there was a systematic increase in
the percentages of trainees who switched to another
option (Terminal or Flight Service Station) or were
terminated from training.

The overall success rates of trainees entering the
terminal option were considerably higher, ranging
from 88% for those aged 22 or younger to 78.6% of
those aged 31 and older. One indication of the lower
overall difficulty of the terminal option on-the-job
training program is the time required to reach full
performance level (FPL) status. It ranges from slightly
over one year at the level 1-2 terminals to approxi-
mately 2 1/2 years ata level 4 or 5 facility. The latter
is still below the average of slightly more than three
years required at the en route centers. Much of the
observed difference in success rates across ages can be
attributed to outcomes from the larger terminal fa-
cilities (Level 4 and 5). Percentages of post-strike
trainees who completed, switched options, or washed
out of field training at the level 4 and 5 terminal
facilities are presented in Figure 6-6, again excluding
those still in training or who left for reasons unrelated
to job performance. The percentage who reached FPL
status ranged from 76.9% of controllers in the 22
years of age or younger group to 46.3% of those in the
31 and older age group. Thus, in the more demand-
ing terminal environments, there is evidence of the
expected age-related decline in on-the-job training

45

success. Most of the trainees at level 5 terminal
facilities are selected from the ranks of FPL control-
lers at other facilities. These outcomes are again
consistent with historical data gathered from respec-
tive field facilities (Trites, 1963; Trites & Cobb,
1964a, 1964b). Thus, older adults experience greater
difficulty completing the FAA Academy screening
and on-the-job training programs for Air Traffic
Control Specialists.

Some scientists have argued that we are not likely
to observe age-related differences at work because
many of the tasks are in continuous use and are thus
highly over-learned. It is difficult to assess the extent
to which older employees may be less proficient at
work due to the multitude of factors that influence
attrition as employees age (poorer workers often drop
out or move to other jobs, and the best are often
selected for promotions or are given opportunities to
switch to more challenging jobs). The effects of
seniority on job assignments in many occupational
settings are also often poorly understood. Salthouse
(1990) describes several mechanisms that may serve
to preserve overall competency and job performance
as adults experience age-related declines in the effi-
ciency of basic processes: compensation, accommo-
dation, elimination, and compilation. Rhodes (1983),
in a review of age-related differences in work atti-
tudes and behavior, concluded that the outcomes
from studies of the relationship between age and job
performance are mixed. Older employees did express
higher levels of job satisfaction, satisfaction with the
work itself, and job involvement.

Evidence for a possible moderating effect of expe-
rience on the typical effects of age on job-related tasks
is mixed. General support was found by Salthouse for
typing tasks (1990) and by Morrow, Leirer, Altieri,
and Fitzsimmons (1994) on readback of visual or
vocally-presented ATC communications. Evidence
was more limited for a set of time-sharing tasks
involving groups of pilots and non-pilots (Tsang &
Voss, 1996). Salthouse, Babcock, Skovronek,
Mitchell, & Palmon (1990) found little support
among groups of practicing architects on spatial
visualization tasks. Thus, while there is some labora-
tory-based evidence to support the critical role of
expertise or experience in the performance of older
workers on job-related tasks, there is insufficient job-
related performance research available to clearly docu-
ment the importance of various factors in determining


Recovery of the FAA Air Traffic Control Workforce, 1981-1992

information required to answer the item, including
aircraft altitudes, speeds, and planned routes of flight.
MCAT test items required examinees to identify
situations resulting in conflicts between aircraft, to
solve time, speed, and distance problems, and to
interpret the tabular and graphical information. The
ABSR wasa 50-item civil service examination (OPM-
157). To solve an item, examinees determined what
relationships existed within sets of symbols or letters.
The examinee then identified the next symbol or
letter in the progression or the element missing from
the set. A sample ABSR item is presented in Figure2-
2. The OKT was an 80-item job knowledge test that
contained items related to seven knowledge domains
generally relevant to aviation, and specifically rel-
evant to air traffic control phraseology and proce-
dures. The OKT was developed as an alternative to
self-reports of aviation and air traffic control experi-
ence. The OKT was found to be more predictive of
performance in ATCS training than self-reports
(Dailey & Pickrel, 1984b; Lewis, 1978).

The scoring of the MCAT, ABSR, and OKT is
presented in Table 2-1. The sum of the weighted
MCAT raw score and the ABSR score was then
transformed into a linear composite ranging from
19.5 to 100. This “Transmuted Composite” (TMC)
score was used to determine employment eligibility.
For applicants without specialized prior experience,
education, or superior academic achievement, a mini-
mum TMC of 75.1 was required to qualify at the
entry level GS-7 grade (Aul, 1991, this volume). Ifan
applicant achieved the minimum TMC, extra points
were awarded on the basis of scores on the OKT, as
noted in Table 2-1. The final civil service rating
(RATING) was the sum of TMC, extra points on the
basis of OKT scores, and any adjudicated veteran’s
preference points. Ranking, referral, and selection
were based on an applicant’s final RATING. In general,
only those competitive applicants with a RATING of at
least 90 were selected by the FAA for employment as
controllers, The distributions of applicant and selectee
RATING scores are presented in Figure 2-3, based on over
200,000 applicant records archived at CAMI for re-
search purposes.

Psychometric Characteristics
Reliability
Reliability in testing refers to the degree to which
scores on a test are free from errors of measurement
(American Educational Research Association, Ameri-

can Psychological Association, & National Council
on Measurement in Education, 1985). Test-retest
reliability is an estimate of the degree to which a
person will obtain approximately the same score
when re-tested after some time interval. The test-
retest correlation for the MCAT was estimated at .60
in a sample of 617 newly-hired controllers (Rock,
Dailey, Ozur, Boone, & Pickrel, 1981, p. 59). Paral-
lel form reliability estimates the degree to which an
applicant will obtain approximately the same score
on a different version of the same test. The parallel
form reliability, as computed on the same sample,
ranged from .42 to .89 for various combinations of
items (Rock et al., p. 103). Internal consistency
estimates the degree to which the items in a test are
homogenous (Ghiselli, Campbell, & Zedeck, 1981).
Lilienthal and Pettyjohn (1981) examined internal
consistency and item difficulties for ten versions of
the MCAT. Cronbach’s alpha for the ten versions
ranged from .63 to .93; the alphas for 7 of the 10
versions were greater than .80. The available data
suggest that the MCAT had acceptable reliability but
was vulnerable to practice effects. In contrast, no
item analyses, parallel form, test-retest, or internal
consistency estimates of the ABSR (OPM-157) test
have been reported by the FAA. Therefore, no con-
clusion can be drawn about the measurement proper-
ties of the ABSR. Published data are available for the
OKT. Parallel form reliability for the OKT ranged
from .88 to .91 (Rock et al., p. 65, 70). The internal
consistency estimate of reliability (Kuder-Richardson
Formula 20 (KR-20); Kuder & Richardson, 1937)
for a 100-item version of the OKT was .95 (Rock et
al., p. 51). Lilienthal and Pettyjohn reported Cronbach
alphas for ten versions of the OKT ranging from .85
to .94 on a sample of about 2,000 FAA Academy air
traffic control students. However, test-retest esti-
mates of reliability have not been published. The
available data suggest that the OKT had acceptable
reliability.

Validity

Validity generally refers to the appropriateness,
meaningfulness, and usefulness of specific inferences
made on the basis of test scores (American Educa-
tional Research Association, American Psychological
Association, & National Council on Measurement in
Education, 1985). Validity in the specific context of
employee selection, as discussed in the federal Uni-
form Guidelines on Employee Selection Procedures