CTI-ATCS Formative Evaluation

students. This objective requizes re-administration of
the personality inventories immediacely prior to gradu-
ation. A secondary goal is to compare the success and
failure of different kinds of personalities in the ATC
workplace. HU faculty plan to correlate scores on the
personality tests with teacher and student evaluations
starting in the summer term of 1993. The results
cannot be used ro validate the tests as selection insiru-
ments.

Tuition

HU waived tuition costs for the first wo TEAM/
ATC classes. Students must bear mition costs for
general education courses and for textbooks, but fi-
nancial aid is available to defray such costs. If and
when the program becomes an established course of
study offered by the university, students will be ex-
pected to assume all tuition costs. It is anticipated thar
financial aid will continue to be available on ademon-
strated need basis.

programs

E. L. Hamm has developed an extensive recruit-
ment program aimed at minorities and women. As
summarized below, six different media have been used
to recruit these target populations:

1, Radio advertisemeat. Radio advestisements were
aired on seven stations in three market areas, mostly
within the Hampton area. Stations were selected be-
cause of their high listener ratings among those from 18
to 34 years old and for their high number of minority
listeners. Two telephone numbers were listed in the ads:
one for HU and one for E. L. Hamm. Because of their
locations in the Virginia Beach metropolitan area, pro-
viding these cwo phone numbers allowed inverested
persons to call toll free from anywhere in the Hampton
to be the most effective medium used to recruit CTI-
ATCS candidates: 52% of the first class and 56% of the
ment.

2. Newspaper advertisement. A nationwide newspa-
per advertising campaign was undertaken. Advertise-
ments were placed in publications with large circulations
(c.g., Atlanta Journal, Boston Globe, Los Angeles Time,
as well as in smaller newspapers with high minority

14

readership (¢.g., Chicago Defender, Virginian Pilot, New
York Voice). Newspaper advertising was much less suc-
cessful than radio and was not used to recruit srudents
for the second class.

3. Campus visits. Campus recruitment visits focused
on communiry colleges with high minority enrollment.
One HU ATC instructor and one E. L. Hamm em-
ployce visited community colleges in Maryland, Vir-
Ginia, and North Carolina where they discussed the
to students. These visits were expensive and labor inten-
sive, and they resulted in no student interest in the CTI-
ATCS program. As 2 result, campus visits were not used
to recruit for the second class.

4. Trade journal publications and advertisement.
HU staff wrore articles for or advertised in trade jour-
nals, such as Virginia Aviation Newsletter, 99 News,
University Aviation Association, and ATCA. Although
used to recruit students for both of the first ewo classes,
these efforts were of limited success.

5. Direct mail advertisement. Direct mail advertis-
ing was targeted at 472 colleges and 36 U.S. Navy
educational counselors nationwide. These counselors
were sent informational letters that introduced the CTI-
ATCS program, provided a brief summary of the appli-
cation requirements, and asked counselors to make the
information available to students. The letter requested
that interested students contact HU's recruitment rep-
resentatives for additional information. Less than 2% of
these letters resulted in requests for information about
HU's CTI-ATCS program. A second direct mail cam-
paign was aimed at community college career placement
personnel. A poster and nine brochures accompanied
this lewer. Telephone response from this second cam-
peign was betres chan the response from the initial effort;
however, HU received no applications.

6, Miscellaneous publicity. Miscellaneous publicity
included announcement letters mailed to news depart-
ments of local radio and television stations. These an-
nouncements led to two television interviews on locai
news and talk shows. Also, a marketing brochure was
developed and distribuced on college campuses, at Army
installations, and ax athletic evencs. Finally, instructors
in HU’s Airway Science Program publicized the CTI-
ATCS program to students already enrolled at HU.
‘These misceliancous efforts, especially the outreach to


CTI-ATCS Fermative Zvaluation

CHAPTER 4
CONCLUSIONS AND RECOMMENDATIONS

The following discussion presents our conclusions
regarding the present status of the CTI-ATCS dem-
onstration program, and recommendations for im-
proving its management and operation. The comments
are based on the authors’ observations and some of the
responses to the subjective questions put to the pro-
gram administrators. Unless otherwise specified, the
comments are intended to apply zo all five programs.

Tut Demanp For CTI-ATCS Gaanuates Has
Bren Snarpry Cuatanep

In the interviews, the administrators of the five
collegiate programs recognized that the greatest threat
to and most serious problem for the CTI-ATCS
program is the reduction in demand for newly trained
controllers. The hiring rates characterizing the early
1980s were expected to continue for many more years
when the CTI-ATCS program was first conceived in
1988. For example, a draft proposal circulated by the
Mid-America Aviation Resource Consortium (MARC,
lazer renamed MrATCTC) in 1988 projected.a”...need

for 2,000 ATC specialists per year for the next ten
years” [italics added]. In other words, MARC pro-
jected that the FAA would add between 1988 and
1998 about 12,000 controllers (after losses at the FAA
Academy and in field training) to its staff of almost
10,000 full performance level (FPL) controllers.
The perception of a significant hiting requirement
appears to have been widely shared throughout the
aviation education community, the Congress, and the
FAA itself in the mid- and late-1980s. However, data
available as carly 2s 1989 from the CAMI ATCS
Training Tracking Data Base, maintained under FAA
Order 3120.22A, es well asa 1989 report published by
the General Accounting Office (GAO), suggested
otherwise. Data from these sources are summarized in
Figure 1. These trends suggested that ATCS hiring
requirements were likely to be reduced by about FY
1993 as the FPL controller workforce was rebuilt to its
pre-strike strength of about 13,000. In other words,
analysis of available data might have suggested that
the collegiate programs were planning to produce

Figure 1. Trends in the ATCS workforce based on GAO and CAMI data
available in 1989.



UAA's CTL-ATCS program a source of highly trained
ATCSs who have a extensive knowledge of Alaskan
airspace and are committed to staying in the region.

Seedents
Admission requirements and procedures

UAA maintains an “open enrollment” policy, re-
quiring only that students (a) be 18 years or older, (b)
have a high school or general equivalency degree, and
(c) participate in UAA’s assessment and advisement
process. Students interested in the ATC program are
advised of the FAA requirements for employment,
including the atrainment of a Class II medical certifi-
cate and the maximum age limitation (30 years) for
employment as an ATCS.

The OPM test is not a requirement for edmission
to the program. In fact, studeats have been advised to
delay taking the test until after completing the pro-
gtam to increase their chances of attaining a high
score. However, it has recently become clear that
students must take the OPM test well before gradua-
tion co avoid delays in hiring. Consequently, the UAA
advisory committee has recommended that the OPM
test be taken at the end of che first year or the

beginning of the second year of study.

Tuition

Tuition is calculated at $64 per credit hour, toral-
ling $3,907 as a maximum total cost for che 61 credit
hours required by the program. Additional fees in-
clude $280 for eight hours of simulator training and
$220 to cover aircraft operation costs in the AT 144
{ATC Airborne Lab).

Recruiting

Much of the recruiting at UAA is conducted uno
ficially by the director of the ATC program. He taiks
to many program candidates through his active in-
volvement in the Civil Air Patrol and other aviation
activities throughout the state. Official recruiting
activities are conducted through UAA's Enrollment
Services Office, which recruits students statewide for
all UAA programs, This office secs up booths at Alaska
College and Career Fairs held in the larger cities of
Anchorage, Fairbanks, Juneau, and Ketchiken. For

3

CT1T-ATCS Formative Evaluasion

smaller villages, representatives of Enrollment Ser-
vices speak on request at cegional fairs and a local
high schools.

To promote recruitment of underrepresented stu-
dents, the UAA offers one full scholarship per year in
the ATC program to be awarded to either a female or
a minoricy group member. In addition, Enrollment
Services maintains separate special offices for recruit-
ing and retaining both Native and African American
students, The Native American office administers a
program to help students make the transition from
village life to life in Anchorage and at the university.
This program is viewed as helping to keep Native
American students in school.

Stedent characteristics

With its urban mission and commuter-campus
environment, UAA attracts older students with an
average age of about 28. Because of FAA age require-
ments, students in the ATC program are a bit younger
(about 25-26). Detailed demographics of ATC classes
prior to the establishment of the CTI-ATCS program
are not available. However, the Alaska Region tracked
176 graduates of UAA who have some record in the
FAA's Consolidated Persanne} Management Infor-
mation System (CPMIS). Of those having a CPMIS
record, 9$ (54.0%) worked for the FAA for some
period of time or are still with the FAA, and 45
(25.6%) are women. Of the 45 women hired by the
FAA, 17 (37.89) are still with the agency.

The first class of 20 CTI-ATCS students graduated
this spring. In this class of 20, 6 already were em-
ployed by the FAA and 2 were not secking FAA
employment. Of the remaining 12 students who were
actively seeking FAA employment, there were 9 white
males, 2 white females, and 1 black male.

Program of Iastraction

Cariculum

The goal of che ATC program et UAA is to produce
graduates who aze able to function as a terminal
controlier at a Level I (WFR) facility or as a radar
associare in an en route facility. UAA program admnin-
isteacors and regional cfiicials favor an earlier model
for the CTI-ATCS program wherein the collegiate


CTlL-ATCS Formative Evaluation

lated that a problem in the CTI-ATCS program might
be the currency of instructor knowledge and experi-
ence, because they all were thought to be retired FAA
and military controllers. In fact, some of the five
programs do, in fact, employ retired controllers. How-
ever, the institutions all cited their respective FAA
regions as being helpful in keeping them current in
FAA rules and regulations. In addition, CCBC supple-
ments retited controllers with active, non-FAA con-
trollers who work full time at the Beaver County
Airport. UAA is the only program that does not use
retired controllers; instead, they employ a few active
controllers from the Alaska region to teach part-time
at the school. The relationship at UAA appears satis-
factory to both the school and the region, and effec-
tively eliminates the problem of instructor currency.

Some Procrams Have MaDe Processs in
Recauirinc Femares anp Mrvoarmes

One of the purposes for developing collegiate ATC
progtams was to attract females and minorities to che
profession. Some {not all) of the CTI-ATCS institu-
tions have made significant progress in that regard. In
particular, the programs at MnATCTC and HU have
actively recruited and trained female and African
Americans in their programs. UND and UAA have
actively sought out Native American students but
thus far have been less than successful in recruiting
them to their program. Although some limited progress
has been achieved, it should be noted thar the pro-
grams will have only limited impact on the diversity of
the concroller workforce, particularly in light of the
anticipated reductions in students being trained in
CTI-ATCS institutions.

Paocrams Do Not Mamrain Consistent STUDENT
Desocaamitics

Some of the programs (MnATCTC and HU} keep
detailed student demographics; however, they differ
in the type and format of student demographic infor-
mation they maintain. A couple of institutions (CCBC
and UAA) do not regularly maintain detailed records
at all. Without detailed and consistent demographic
information, it is difficult to assess the progress thar
the programs are making to recruit females and mi-
norities. This situation may improve when the FAA's

Office of Aviation Careers begins co track CTI-ATCS
students. The plan is for this office to track all CTI-
ATCS studenis, starting when they enroll in the
collegiate programs. The purpose is ro keep track of
the srudents’ progress in completing FAA require-
ments and to market the students to the regions when
they graduate. Using MnATCTC and HU asa model,
itis encouraged that all CTI-ATCS institutions main-
tain demographic data on students, consistent with a
common data base. In particular, it would be helpful
if the information on student ethnic background was
coded consistently across the schools.

Procrams Were Devevorep rao Dirrerent Jos
ANALYSES O& No ANALYSES AT ALL

Despite some commonalities in instructional de-
sign (e.g., the chunk-and-apply approach), the indi-
vidual programs are based on different job analyses or
no analyses at all. Job analyses are commonly regarded
as a necessary initial step in the development of
training and performance measurement technologies.
Some progsaras simply skipped this step, relying in-
stead on existing curricula or their own subject macter
expertise. Other programs performed their own analy-
ses or modified FAA task documentation. The lack of
acommon job analysis is reflected in the diversity of
terminal objectives provided by the institutions. Us-
ing the terminal option as an example, the final
learning objective ranges from performance equiva-
tent to the requirements of a Level I VFR facility to
performance expected at a Level V facility.

Thiscriticism begs the question whether or not the
schools should have performed their own analysis of
the controller's job. We think not. We assume chat all
of che programs are training students for the same
ATCS job. Therefore, they should all be besed on a
common set of requirements for the controller's job.
Without a common set of eraining requirements, the
FAA does not know the basic skills and knowledges
thac CTI-ATCS graduates bring to the job. Therefore,
we recommend that a single, common job analysis be
accepted by the FAA and that cach of the insticutions
link the analysis to cheir educational programs.

The FAA is already moving in that direction. In
August 1993, the FAA sent all CTI-ATCS institu-
tions a letter oudining the training objectives (job


CTi-ATCS Formative Evainatien

administering the OPM exam at the midpoint of the
program, studenis who do not have a strong aviation
background (80% of MnATCTC’s student body)
increase their chances of earning bonus points on the
exam. Second, the exan: is administered early enough
in the program so that the PAA can begin the sec: ity
check for students who pass the OPM exam.

Before the initiative co adopt the computerized
PTS test, Dr. Phillip Ackerman, 2 University of Min-
nesota consultant to MnATCTC, developed a written:
battery of teses ro select students for MrATCTC's
program. The tests were administered to students in
the firse seven classes as an experimental battery, bus
will be used vo assist in making selection decisions for
the eighth class. The batrery consists of tests that
ieasure reasoning, spatial ability, perceptual speed,
and perceptual/ psychomotor ability. A complete de-
scription of the rest battery and validizy results may be
found in Ackerman (2992).

The mean and standard deviation of scores for
MnATCIC’s selection test are 50 and 10, respec-
tively. Dr. Ackerman contends that the barcery hss not
been administered co enough students co warrant the
establishment of a formal cutoff score. However, his
data indicate that most students who score 40 or
above are successful in MnATCTC’s program. There-
fore, 40 has been established as a tentative cutoff. In
making selection decisions for the upcoming eighth
class, adminiszrators will look for a misimum score of
40 on the battery; but they will consider admitting
students who score in the upper 30s, but who have
excellent academic credentials.

Tuition

Presently, tuition is covered by federal funds, so
students incur no cuition costs. However, students
must pay fc books and materials and their own living
expenses; there is no financial aid available for these
expenses.

Reaaiting programs

MnATCTC uses a variety of recruiting methods to
inform candidates about its program. To recruit the
first class, MnATCTC relied primarily on a local bliz
advertising campaign. Since the first class, however,

MnATCTC’s recruiting activities have focused on
two other methods that have proved especially suc-
cessful: the viewbook and personal referrals. The
viewbook is a recruiting booklet provided to potential
applicants. The viewbook describes air traffic control
as a carcer, and the MnATCTC program. in particu-
lar. It also includes summaries of courses, 2 descrip-
tion of the Eden Prairie area, and an application form.
Personal referrals are recommendations of the pro-
gram made by careez placement counselors, FAA
employees, MnATCTC graduates, and MnATCTC
feculty and staff. MnATCTC feels chat referrals are
the most effective recruiting method. For the October
1993 class, for example, almost half of the applica-
tions were obtained through referrals.

Tc increase minovity enrollment, recruiters have
advertised MnATCTC’s program in minority publi-
cations. The Bleck Collegian has been particularly
successful in generating applications. MaATCTC also
uses its convacts with minority coalitions (e.g., Black.
Hispanic, women) within the FAA. One of the goals
of the minority coalitions is to educate fellow ...cm-
bers of minority populations about careers in air
traffic contvol. In that context, coalition members
make presentations to school and community groups.
During these presentations, MaATCTC’s coaiition
contacts publicize the program at MaATCTC.

Student characteristics

Some of MnATCTC's incoming students have
completed an associate’s degrec, but most hold
bachelor’s degrees. MoATCTC maintains detailed
ethnic and gender daza regarding the compesition of
its classes. Table 1 presents that information aggre-
gated across the seven classes. As can be seen in Table
1, a majority of MnATCTC's students have been
white and male. However, the table docs not show the
improvement MnATCTC has made in attracting
minority and temale students. For example, the first
class was 96% white and 10% female, whereas the
seventh class was 69% white and 53% femziec.
MnATCTC has established an affirmative action goal
of 60% total minority and female enrollment. To
meet chis goal, MaATCTC is willing ro accept fewer
than the optimal class size of 32.



Appendix & CYI-ATCS Formative Sveiuation

d. How do they keep abreast of changes in the ATC profession?
e. Do your instructors receive any instruction op. “training to train?”

8, The next questions concern your communications with the FAA regarding your CTI-ATCS program.
2. How do you communicate with FAA?

b. Does the FAA try to keep you abreast of changes to the ATC profession?

9. The following questions concern your personal expectations about your graduates.

a. Do you expect your students to be fully accepted in field training?
b. How do you expect they will fare (relative to Academy graduates) in over the longer (carcer) cerm?
¢. Do you expect your students to remain in this geographic erea once they scart their ACTS careers?

10. Finally, I have some questions to elicit your personal evaluations of the program here.
2. Is your opinion, how docs your prograin compare witn tne FAA Academy?

b. Doyou sec your program as competing with other CTI programs for students, funding, FAA attention, and
soon?

c. Whar are particularty attractive features of your program vis-a-vis the other CTI-ATCS programs?

d._ In general, what are your program’s greatest strengths (weaknesses)?


CTPL-ATCS Formative Eveiuavion

work. The computer system is capable of recording
who uses which CBI lessons, how long the person
works on each lesson, and posttest scores.

VER Tower Lab

The VER Tower Lab is used to train tower cab
operations in ATC/CRM 315 (Tower Lab 1), ATC/
CRM 316 (Tower Lab II), and for the first half of
ATCICRM 446 (ATC Facility Lab). The lab contains
atable top replication of a hypothetical airport called
Hampton Airport. The airport comes complete with
2 control tower; bangers and other buildings; scale-
model general aviation aircraft, jet aircraft, military
aircraft, helicopters; and scale-model rescuc and main-
tenance vehicles. Hampton Airport has two parallel
tunways and an intersecting runway crossing one of
che parallel ranways. There is also a grass runway. The
airport is equipped with functional runway lighting
which adds a measure of realism when students prac-
tice night operations. Each student pseudo pilot com-
municates with the tower cab on a five-channel
transmi/receive mobile radio headset.

A representation of a tower cab stands on a riser
behind and slightly away from the model airport. The
simulated tower cab is large enough to accommodate
three students and an instructor. The cab contains
beadsets, an operable light gun, a 24-hour clock, a
weather date display, a wind indicator, slots for hold-
ing flight progress strips, BRITE radar, an Automatic
Terminal Information Service line, and a telephone
used for communications with a simulated en route
center (Washington, DC). Members of 2 CCRM
team usually rotate among flight data, ground con-
trol, and local control ATC positions, while members
from another CCRM team serve as “pilots” of scale-
mode helicopters and fixed-wing aircraft, and “op-
erators” of rescue and maintenance vehicles.

During che first halfof ATC/CRM 446, the Tower
Lab is used in conjunction with the Radar Lab to train
joine tower and TRACON operations. One CCRM.
team acts as tower cab controllers, one team serves as
TRACON conrrofiers, and remaining CCRM teams
act as pseudo pilors. After team members have rotated
among controller positions for their facility (e.g..
among cower cab positions), teams change facilities.

ts

During ATC/CRM 446, students also apply the prin-
ciples of BRITE radar that were learned in ATC 302
(Tower Operations Il).

Radar Leb

The Radar Lab simulates the radae section of a
TRACON facility and an en route center. It is used in
ATCICRM 415 and 416 (Terminal Lab I and II),
Terminal Lab II, ATC 445 (En Route Operations),
ATCICRM 446 (ATC Facility Lab, and ATC/CR?4
499 (Combined ATC Operations). The simulation
system is ATCoach® and is described in detail earlier
in the context of MnATCTC’s program. The follow-
ing description highlights the unique aspects of this
system at HU.

At HU, the simulation hardware consists of four
radar scopes, cach of which is connected to a personal
computer that serves 2s a pseudo pilot/pseudo facility
position. When running 2 scenario, four students are
assigned to each station, playing roles of radar con-
troller, radar associete, pseudo pilot, and pseudo fa-
cilities operator. If necessary, one student can serve as
both pseudo pilot and pseudo facilities operator. Two
two-person CCRM teams may be used to run an
exercise such that one team functions as the controller
and associate, while the second team serves as pseudo
pilot and pseudo facilities operator. A three-person
CCRM team may run an exercise: one controller, one
assistant controller, and one pseudo pileti pseudo f2-

As students advance, they learn to coordinate with
other controllers in the same TRACON facility, as
well as with en route controllers and/or tower cab
controllers. When running coordinated scenarios, the
four radar positions can be configured to operate
interdependently. For example, the Hampton air-
space may be divided into feeder and arrival positions.
Thus, two student controllers working at adjacent
scopes may be running 2 joint exercise. When practic-
ing a joint TRACON/en route scenario, three scopes
may be setup as a TRACON facility (i¢., feeder,
aztival, and departure positions), with the remaining
scope serving as an cn route center.

During ATC 499 (Combined ATC Operations),
the Radar Lab is used to train joint TRACON and en
route operations. Depending on the size of the team,


CIIL-ATCS Formative Evaluation

products (graduates) just at the time when the marker
for those products was (az best) mature, and (at worst),
shrinking. Analysis of GAO and CAMI data through
1991 (Figure 2) only confirm the projections from
1989 data: There was likely to be a smaller market for
CTI-ATCS graduates in the 1990s than had been
anticipated. FAA controller workforce requirements
have been even further reduced by the age distzibution
of the rebuilt controller workforce, which has sup-
pressed attrition due to retirement. Moreover, the
tecent trend toward flattening the ATCS manage-
ment steucture has reduced attrition due co promo-
tion. As a consequence of these three factors, the
demand for new entry-level controllers has been sharply
reduced, and as a result, graduates of the CTI-ATCS
institutions are experiencing significant delays in get-
ting hired by che FAA. This is especially problematic
for the schools that attract older students who may be
close to the age-3@ hiring restriction.

In response, program administrators want a greater
commitment from the FAA to hiring and placing
CTI-ATCS graduates. Some of the administrators are
willing to reduce che numbers of students thar they
train in exchange for that commitment. Their prob-

Jem is getting from the FAA accurate, timely projec-
tions of the need for new controllers for planning
purposes. Clearly, the numbers of students dictace
some of the essential parameters of their programs,
such as the numbers of instructors, simulators, class-
toom space, and so on. To plan for these reductions,
the FAA needs to provide accurate projections of the
number of new controllers that it will need and to
allocate those slots among the five institutions. Fur-
thermore, these projections should provide estimares
for atleast five years into the future to plan for the next
year's incoming classes at the four-year institutions.
“These projections should chen be updated as develop-
ments occur. For instance, the projections should
consider the impact of ex-PATCO controllers who are
now allowed to compete for jobs in the FAA.

Tus Proceams Appear To Bz Functronine Wen.
All five CTI-ATCS programs are currently in op-
eration and appear to be functioning well. Several
observations led the evaluators to this conclusion.
First, each of the administrators projected a sense of
having clear training goals and reasonable strategies
for reaching those goals. The confidence of the ad-

Figure 2. Trends in the ATCS workforce based on 1982-1991 GAO and
CAMI data. (OUT figures for 1993 and beyond are projected.)

Ornueanaraods2REsa



Table 1.

Table 2.

Table 3.

Table 4.

Table 5.

Table 6.

Table 7.

Table 8.

Table 9.

Table 10.

Figure 1.

Figure 2.

LIST OF TABLES

Page

Gender and Ethnic Data for First Seven Classes in ATC Program at

Minnesota Air Traffic Control Training Center 9
Required Coursework for ATC Certificate at Minnesota Air Traffic Control

Training Cencer 9
Gender and Ethnic Data for te Two Classes Currently Enrolled in the Terminal

and En Route Airspace Management/Air Traffic Control (TEAM/ATC) Program

at Hampton University 15
Required Coursework for the B.S. in Terminal and En Route
Airspace Management/Air Traffic Control (TEAM/ATC) at Hampton University —-—--— 16
Required Coursework for the A.AS. in Air Traffic Control at the

Community College of Beaver County 22
Gender and Ethnic Data for Three Classes Currently Enrolled in the ATC Ops

Program at the University of North Dakota 27
Core Curriculum for Airway Science Program at the University of North Dakota —--—-—— 27
Required Coursework for the B.S. in Airway Science (ATC Ops Concentration)

at the University of North Dakors -- 28
Required Coursework for the A.A.S. in Air Traffic Control at che University of Alaska,
Anchorage 32
Examples of Innovations Identified at Each of the Five CTI-ATCS Institutions ——--——- 38

LIST OF FIGURES

Trends in the AFCS workforce based on GAO and CAMI data available in 1989 -~-------~ 35
Trends in che ATCS workforce based on 1982-1991 GAO and CAMi data ~~ 36



foundatior on which to build managerial expertise,
UND's curriculum includes courses co help students
understand human behavior at the individual level as
well as ac higher organizational levels. Managerial
courses are included in both the core curriculum an?

the ATC Ops program.

ATC Lab

The ATC Lab ac UND simulates the radar section
ofa FRACON facility. It can also be used to simulate
an en route center, but is not used as such because the
curriculum does not prepare students for placement
inen route centers. It is used in all ATC courses, with
the exception of ATC Tower Operations. The simu-
lation system is Virtual Controller, which was devel-
oped at UND.

Virtual Controller's software is divided into three
subsystems, which serve unique functions in building
and monivoring laboratory exercises. The subsystems
are summarized below:

1, The Profile Editor is an aircraft database and
includes U.S. and international civilian and military
airplanes and helicopters. Performance capabilities of
each aircraft (e.g., speed, climb and descent rate, turn.
profile) and call signs are included. The database cur-
rently contains data for 360 types of aircraft, but more
can be added.

2. The Aircraft Builder is used to create Laboratory
exercises by retrieving dara from the Profile Editor, by
space, and so on. UND uses the same airspace that is
used for training at the FAA Academy: Acro Center
(ZAE). However, Aircraft Builder is capeble of depict-
ing a hypothetical airspace or any existing airspace.

3. The Training Management subsystem maintains a
dacabese of the exescises completed by each student and
provides feedback on exercise performance. Feedback
consists of pointing our the error and providing 2 cefer-
ence in the FAA Handbook (Order 7110.65) that iden-
tifies che proper response. Feedback may be presented
immediately as the scenario progresses or as a final
scenario grade. The Training Management subsystem
can output feedback to the screen, to a printer, or both.
Instructors may pause a scenario, replay portions of a
scenario, or replay all of a scenario when discussing a
scudent's performance.

CTI-ATCS Formative Evaluation

The hardware consists of 10 radar scopes, each of
which is connected to a personal computer that serves
as a pseudo pilot/ pseudo facility position. Each radar
position includes a headset, radar scope, a keyboard,
a trackball, an area to hold flight progress strips,
simulated telephone links to en route and terminal
facilities, and a map display provided by an overhead
projection. In addition to controlling simulated air
traffic, the trackball is used in a menu-driven applica-
tion to load exercises. The screen can be set to project
in monochrome or in color. Coler projections are in
keeping with the new radar system, the Advanced
Auromation System (AAS) being developed for the
FAA. The ovethead projection area contains a map of
the Aeronautics! Center airspace and radio frequen-
cies for terminal facilities and en route centers used in
che exercises.

When running a scenario, a single radar position
may be supported by as many as four students assum-
ing the roles of radar controller, assistant controlier,
pseudo pilot, and pseudo faciliies operator. More
commonly, a radar position is staffed by two students
who assume the roles of radar controller and pseudo
pilot/pseudo facilities operator. While engaged in the
simulation, students communicate via headsets. Vir-
tual Controiler has voice activation capabilities elimi-
nating the need for a pseudo pilor, although someone
is still be needed to serve as a pseudo facilities opera-
vor.

As is true for ATCoach®, the workstations can be
configured to run independently or in a coordinated
scenario. The former is more appropriate for students
who are first learning radar control, whereas the latter
is useful in learning to coordinate with other conuol-
fers in the same TRACON facility, as well as with
concrollers at terminal facilities and en route centers.
For example, che simulated Aeronautical Cencer air-
space may be divided into arrival and departure sec-
cors. Thus, wo student controllers working at adjacent
scopes may be running a joint exercise, with one
student controlling arrival air traffic and another
student controlling departure traffic.


HU students, were quite successful. Of the first class,
22% were already attending HU, and 22% learned of
the CTI-ATCS piogram through other miscellaneous
methods. Of che second class, 33% were already attend-
ing HU, and 119% learned of the program through other
methods thar fall into the miscellaneous category.

Seudent characteristics

Students normally join die CTI-ATCS program
after completing two years of general college educa-
tion requirements at HU or some other accredited
institution. HU and E. L. Hamm maintain detailed
race and gender data regarding the composition of
their CTI-ATCS classes. Table 3 presents that in-
formation for the two classes currently enrolled at
HU. The two classes indicate a trend towards increas-
ing diversity: Approximately 61.1% of the first (se-
nior) class are either black or female, whereas 83.3%
of the second (junior) class are in either one of those
demographic groups. Particulasly large gains in recruit-
ing black females have been made in the junior class .

Table 3

Gender and Ethnic Data for the Two Ciasses

CILATCS Foruasive Evaluation

Program of Instraction
Carriceluzn

The two-year CTI-ATCS program at HU is de-
signed to be taken after completion of two years of
college coursework. It is actually six semesters in
duration, which includes two summer sessions. The
Program grants the B.S. degree upon successful comple-
tion of the coursework.

The HU program provides instruction on both the
terminal and en route options; however, the focus is
‘on terminal operations. Students teatn to work the
flight data, ground control, and local control rower
Positions; the terminal radar position; and the en
route data and radar positions. By the end of the
program, students should be able to handle 85 aircraft
per hour at the tower position, maintain an arrivai race
of 55 aircraft per hour at the terminal radar position,
and handle 45 aircraft per hour at the en route posi-
tion. The curriculum focuses on the teamwork aspect
of air traffic control, specifically coordination among
ATCSs and between the ATCS and pilot. It also

Currently Enrotied in Terminal and

En Route Airspace Management/Air Traffic Control (TEAM/ATC) Program at

Hampton University
Percent
Class Gender Race Number of Total
Senior Female Black $ 27.8
White i 55
Maile Black 5 278
White 7 38.9
Junior Female Biack 9 50.0
White 2 il
Mele Black 4 222
White 3 16.7



shave completed CCBC’s AAS. program in Air
Traffic Control,

be enrolled in a four-year program at one of two local
insticutions (Geneva College or Robert Morris Col-
lege) as a full-time student, and

not have completed more chan 90 semester or 120
quarter hours.

Three Eastern Region FAA facilities participate in
the Co-op program: Greater Pittsburgh International
Airport, Baltimore- Washington International Airport,
and Philadelphia International Airport. Although in
theory, students may be placed at any of the three
facilities, CCBC students are placed only at Greater
Pictsburgh International.

Student characteristics

While a majoriry of CCBC’s ATC students hail
from the local area, students from across the state of
Pennsylvania and from 24 states also are attracted to
the program. Most students enroll in the ATC pro-
gtam immediately upon graduation from high school.
The secoad largest age cohare comprises students who
graduated from high school rwo to four years previ-
ously. Many of this latter group of students are traas-
ferting general education credits to CCBC; others
have prior military service.

CCBC does not regui:ly maintain detailed race
and gender daca. The most recent compilation of
seudenz demographics was performed ro identify su-
dents who could meet graduation requirements in
May, August, and December 1993. From the 67
scudents whose demographic grouping could be de-
cermined, the overwhelming majority were white males
(57 or 85%). Other represented groups included 8
white females (12%), 1 black male (1.5%), and 1
black female (1.59%).

From the initiation of the CTI-ATCS program at
CCBC to March 1993, 89 studenes have been recom-
mended for direct hire by the FAA. Noce chat this
number ceflects individuals recommended by CCBC
for hire, and does not reflect the total number of ATC
graduates. Te be eligible for this recommendation.
students musz be (2) near completion of their third
semester, (b) close to obtaining their private pilot

2b

CTYL-ATCS Fermasive Evalzation

cestificates, and (c) rated as “promising” by the ATC
faculty. Of the 89 graduates of this program tecom-
mended for hire, 32 have been actually hired by the FAA.

Program of Instroction
Casricalam

The ATC program at CCBC provides formal train-
ing on the terminal option only, but includes some
instruction in both tower cab and TRACON opera-
tions. Students learn to operate the flight data, ground
control, clearance delivery, and local control tower
Positions as well as the terminal radar position. The
final objective of the program is to produce graduates
who can control the level of air treffic expected ro be
encountered at a Level HH terminal facility.

The ATC curriculum at CCBC was not developed
using a systematic model such as ISD; rather, it has
evolved over the 17 years of repeated tryouts and
revisions. Except for the addition of two radar control
courses, che curriculum has remained relatively un-
changed from the courses offered before CCBC be-
came part of che FAA’s CT1-ATCS program. One
exception is that CCBC added terminal rader to the
curriculum after learning that their graduates would
be required to perform radar operations as a part of
performance verification (PV).

Tabie 5, which is adapted from the CCBC Aviation
Handbook, presents the recommended course pro-
gression for ATC students. The ATC program can be
divided into three broad areas as follows:

1. General Education. General education require-
ments are spread throughout the two yea: period, but are
concentrated in the first semester. These general educa-
tion courses are intended to produce well-rounded
ATCSs with the background accessary to become ATC
managers for the FAA.

2. Private Pilot Certificate. Although 2 private pilot
certificate is not necessary to become a successful ATCS,
the ATC faculty believes that it gives students a berter
understanding of air traffic contro! from a pilot's per-
spective. Two courses are required to earn the Private
Pilot Certificare: PP 110 (Flight Theory) and PP 106
(Private Pilot). PP 110 prepares students for the FAA
written exam for the Privare Pilot Certificate, which is



Jastrection in team end msxzagerial skibs

The curriculum incorporates an approach to hu-
man factors training that HU terms “Conwoller Crew
Resource Mianagement” (CCRM). Crew Resource
Management (CR'M) has been used with much suc-
cess to train cooperation and team building skills for
cockpit crews. HU’s approach is to apply CRM piin-
ciples to air traffic control. Two goals underlie the use
of CCRM: (a) to teach interpersonal skills with the
intent of helping students become better human
interactors as employees, with other controllers, and
with pilots; and (b) to determine the extent to which
CRM model devised for cockpit crews applies to
controllers.

The curriculum does not include a separate course
for CCRM. Instead CCRM training is implemented
through classroom instruction, case study simula-
tions, and the teamwork requised to complete labora-
tory exercises. CCRM principles are presented in
classroom lectures. As part of the classroom instruc-
tion, students must enact and solve real-life problems
which are caused by interpersonal conflicts. The situ-
ations may be strictly interpersonal (e.g., sexual ha-
rassment), ot they may be ofa technical nature with an
underlying interpersonal problem (e.g., an near acci-
dent caused by lack of coordination among ATCSs).
Students aze grouped into two- or three-person teams
for laboratory exercises from the beginning of their
involvement in the TEAM/ATC program. For tower
cab exercises, students on a team alternate among
data, ground control, and local control positions. For
TRACON exercises, team members rotate among
radar controller, radar assistant, and pseudo pilot
positions. Team composition is changed each semes-
ter. This allows students ample opportunity to learn
to function within the dynamics that are unique to
each team, but puts them in as many different team
situations as are reasonable.

Another imporcant aspect of HU’s program is che
development of administrative and manzgerial skills.
HU feels chat their graduates are likely to attain
managerial positions by virtue of their college degree.
To ensure that graduates have a firm foundation on
which 00 build managerial expertise, HU’s curriculum
includes courses in aviation safety and management.

17

CTI-ATCS Fermasive Evaluation

Remediatios palicd

Given thar the curriculum requires students to take
a large number of courses consecutively, there is no
opportunity for students to retake a class if they fail.
To be fair co students, a rather detailed policy toward
remediation has been instituted. The policy stipulates
that students receive remediation and be retested on
lessons where they score below the passing cutoff of
70% correct. With the exception of final exams and
end-of-course research papers, all coursework is sub-
ject to remediation—whether it be homework, a lab
assignment, a midterm exam, or other assignment. If
students perform poorly on material covered in home-
work and lab assignments, they receive additional
instruction, but the grade is unaffected. In contrast,
students can improve their grade on midterm exams
and reports by retaking the exam or rewriting the
report. Students are allowed 4 maximum of two re-
tests. If students fail a midterm exam, for example,
they receive remedial instruction and retake the exam.
If students fail the retest, they receive remedial train-
ing again and are retested. If students pass the mid-
term on the first retake, their scores on the exam are
defined as 70%, regardless of actual score. If students
fail the second rezake, their scores on the exarn are
recorded as their actual test score.

CBI Leb

The CBI fab contains 10 DOS® -based personal
computers equipped with 80286 processors and a file
server equipped with an 80386 processor. CBI lessons
were developed by an E. L. Hamm computer pro-
grammer working in conjunction with the TEAM/
ATC faculsy. The lessons feature high resolution
graphics and quite a bit of animation. The lessons do
not incorporate sound, because sound would be dis-
tuptive in the open laboratory sereing.

The lessons, which cover rore material such as
aircraft recognition, are used primarily to reinforce
material taught in the classroom. Lessons are designed
to augmenc, not supplant, the instructor. Another
function of the CBI lessons is to provide remediation
when a student scores less than 70% on an assignment
or exam. CBI lessons may also be assigned as home-


CYI-ATCS Formative Kuainasion

CBI Leb

ATC CBI lessons aiz run on Virtual Controller.
Fifty CBI lessons are available for first- and second-
semester students. These lessons, developed for the
Minnesota Air National Guard, teach ATC basics.
Because the material is basic and somewhat repetitive,
the lessons employ animation to hold the student's
interest. The lessons are self-paced, and feedback is
immediate. The student may review previously pre-
sented material by reversing through the program one
screen ata time. Also, a student may exit the lesson at
any time. Finally, embedded questions are included in
the lessons so students can test themselves.

The CBI lessons have not been used as pact of
graded coursework in the past. Instead, students pri-
marily used CBI lessons to supplement: classroom
instruction at their own discretion. Beginning with
the fall semester of 1993, however, ATC Ops assigns
CBI lessons as homework. A written, knowledge-
based exam is administered to ensure chat students
complete the lessons and to assess their understanding
of the material covered in the lesson.

Class sixe

In the past, UND's class size has been sbour 10
students. With the current level of staffing, program
administrators feel that 10 students sre a comfortable
load. Alchough UND has enough radar simulators to
accommodate 25 to 30 students, more instructors are
needed to support a class of chat magnitude.

Instructors cad developers

UND's ATC Ops program employs four instruc-
tors. One of the instructors also serves as the director
of the ATC Ops program. Two instructors (including
the direcvor) are former Air Force controllers. The
other ewo instructors are recently retired FAA ATCSs,
with one having en route and the other, terminal
experience. All four instructors have prior teaching
experience: The former Air Force controllers have
extensive experience as OJT instructors in the mili-
tary, and the two retired FAA controllers taught at the
FAA Academy.

Univeastry or ALasxa, ANCHORAGE

The CTI-ATCS program at the University of Alaska

(UAA) evolved from an ATC progcam started in 197%
art the Anchorage Community College. When the
community college merged with UAA in 1988, the
ATC program continued within the university's Col-
lege of Career and Vocational Education. Before
being incorporated into the CTI-ATCS program, the
ATC program in Anchorage was designed to prepare
students to pass the OPM written test and to attend
the FAA Academy. As evidence of the older program’s
effectiveness, program administrators often cite the
fact that, of their graduates who were accepted at the
Academy, well over 90% successfully completed the
screen.
The ATC at UAA program was eccepted as a CTI-
ATCS institution in November 1991 for both termi-
nal and en route options. That same year UAA was
also approved as a CTI program in electronics tech-
nology, and is the only institution having both types
of CTI programs. In March 1992, UAA’s approval as
an Airway Sciences institution was extended. In addi-
tion to Air Traffic Control and Aviation Electronics
Technology, UAA offers three other aviation-related
programs of srudy: Aviation Administration, Avia-
tion Maintenance Technology, and Professional Pi-
foting. These latter programs are not yet part of the
approved Airway Science curricula. Aviation Mainte-
nance Technology has been submitted for approval,
but has not received fina! endorsement.

Whereas the ATC program has not been a direct
recipient of FAA funds, the Airway Sciences program
has received $6.9 million in fands from FY 1982-
1993. In addition, UAA recently received 2 $10.5
million dollar grant for capital improvements 20 the
program. Some of these funds will be used to build a
new training facility for ATC and for other improve-
ments to the Airway Science program.

The Alaska Region of the FAA actively participates
in the UAA program in Aviation Sciences. Represen-
tatives from the region sit on a board of advisors who
provide formal guidance and valuable subject matter
expertise co the ATC program at UAA. In rerarn for
their participation, the region hepes to gain from


CTLATCS Formative Evaluation

more closely, including such information as which specific
parts of the lesson are used, and scores on the post test.

Radar Lab

The Radar Lab simulates the radar section of an en
route center. Iris used in every course except ATCT 100
(nuoduction to Air Traffic Control). This lab em-
ploys the ATCoach® system and consists of 16 radar
scopes, each of which is connected to 2 personal
computer that serves as a pseudo pilot/pseudo facility
position.

Each radar position is equipped with a headset,
radar scope, a keyboard, a trackball, an area to hold
flight progress strips, simulated telephone links to en
route and terminal facilities, and an overhead projec-
tion map area. A pseudo pilot position has 2 headset,
akeyboard, a mouse, and telephone links w't the en
route controller. When running a scenario, ewo .-u-
dents are commonly used to support a single radcr
position. One student acts as the radar controller
while the other serves as the pscudo pilot/ pseudo
facilities operator. The controller and pseudo pilot
communicate via headsets.

‘The radar positions can be set to operate indepen-
dently of one another to train basic controller skills.
As students advance, however, they must learn to
coordinate with other controllers in the same center,
as weil as with controllers at other towers, terminal
radar control (TRACON) facilities, en route centers,
and flight service stations. When running coordi-
nated scenarios, the radar positions can be configured
to operate interdependently. For example, the simu-
lated airspace (a portion of the Minneapolis En Route
Center - ZMP, including Lincoln and Omaha, NE)
may be divided into high and low positions. Thus,
wo student controllers working at adjacent scopes
may be cunning a joint exercise, with one student
controlling air traffic above a certain altitude, and
another student controlling traffic below that altitude
within the same geographic area.

Ciass size

MnATCTC administrators feel their optimal class
size to be 32. In the past, MnATCTC has maintained
close to the optimal number of students, with an
average of 30 students across the seven classes.

12

Tnseractors and developess

MoATCTC employs five administrative staff per-
sons and ten instructors. The staff includes a director,
dean, recruiter, admissions officer, registrar, and an
instructional services coordinator. The instructors are
all former controllers, of whom about half have FAA
backgrounds and about half, military backgrounds.
The former FAA controllers left the FAA within three
years prior to their employment with MnATCTC.
The military controllers came immediately to
MnATCTC upon discharge. Also, as 2 group, about
half of the instructors have en route experience and
about half have terminal experience. Most of them
have training experience either in the military, at FAA
facilities, or at the FAA Academy. One is a former
public school reacher. The Minnesota Technical Col-
lege System requires that all instructors be certified.
Therefore, all of MnATCTC’s instructors are certi-
fied at the technical-college level, and they must meet
the system’s continuing education requirements.

Hamrron Univeasrry

Located in Hampton, VA, Hampton University
(HU) is a privately endowed, coeducational, nonsec-
tarian institution, and is a member of the Registry of
Historically Black Colleges and Universities. HU’s
CTI-ATCS curriculum is subsumed under the
university's Airway Science Program. The Airway
Science program offers the Bachelor of Science (B.S.}
degree with concentrations in Airway Science Admin-
istration, Airway Computer Science, Airway Elec-
tronic Systems, and Air Traffic Management.
Recognized by the FAA since 1985, the Airway Sci-
ence Program is an interdisciplinary curriculum de-
veloped by the FAA and the University Aviation
Association.

In February 1990, E. L. Hamm and Associates
contracted with the FAA through the Small Business
Administration to develop a prototype ATC training
program at HU. The result was the Terminal and En
Route Airspace Management/Ait Traffic Control
(TEAM/ATC) program, a separate ewo-year curricu-
tum leading to the B.S. degree. The first class began in
June 1991. The TEAM/ATC program has received
approximately $3 million from the FAA to develop
programs and to support students.



CYL-ATCS Formative Evalustion

CHAPTER 2
METHOD

Each of the five individual CTI-ATCS programs
was examined using a formative evaluation approach.
Formative evaluation is usually defined in contrast to
summative evaluation, the purpose of which is to
provide inferences about the outcomes of programs. A
suramative evaluation usually concerns quantitative
outcomes, such as the success of CTI-ATCS graduates
on the job, as measured by their rate of progress in on-
the-job training (OJT) at field facilities, or scores on
job performance evaluations. Te produce these sorts
of outcome data, however, a program must be fully
implemented and stable for 2 considerable period of
time. The five CTI-ATCS programs, in contrast, have
only just started to produce graduates: Only 61 gradu-
ates were hired during the firse 18 months of the
program. Consequently, 2 summative evaluation of
the program would be premature ax this point.

Whereas summative evaluations focus on program
outcomes, formative evaluations examine (and hope-
fully, improve) program processes at the early stages of
program implementation. Another contrast is thatthe
data collected in formative evaluations are usually
qualitative in nature. As originally differentiated by
Scriven (1967), however, formative and summative
methods are not alternative methods; rather, they are
complementary approaches ina comprehensive evalu-
ation strategy. The qualitative observations and specu-
lations of formative evaluation are seen as a necessary
first step 'n gencrating hypotheses co be rested in the
surnmative evaluation phase. Thus, the present find-
ings should be viewed as the initial phase in a continu-
ing evaluation process.

CTI-ATCS Srre Visris

Morrison and Fotouhi visited each of the five
participating CTI-ATCS programs in May and June
1993. At cach site, the evaluators collected and dis-
cussed program literature describing the programs,
such as catalogues and promotional materials. They
also examined training facilities. The evaluators were
able co observe classes in session only at CCBC and
MnATCTC because the school ycar had already ended
at the other three program sites. At each site, evalua-
tors conducted interviews with key program person-

nel including administrators, instructors, and stu-
dents when available. (Interview protocols are in-
cluded as an Appendix.) Students were not available
for interviews at HU and UAA. Also, none of the
instructors at UAA were available because they are on
campus only during the regular school year.

In addition to visits to the five CTI-ATCS sites, a
visit was made to the FAA Academy to acquire back-
ground information for the evzluation and to examine
the Academy terminal and en route training pro-
gtavas, It is important to note that the FAA Academy
is not a subject for evaluation in the present report.
Rather the Academy was intended to serve asa “bench-
mark” against which the individual CTI-ATCS pro-
grams could be compared. The Academy training
program is discussed only as a point of reference for
describing CTI-ATCS programs.

Evarvation Topics

Administrators of the CTI-ATCS programs were
questioned about their programs to probe their per-
ceptions of program goals. Some questions were asked
at the end of the interview to obtain more subjective
impressions zbour the progcam as a whole, such as the
perceived s.rengths and weaknesses. Most questions,
however, were designed to provide factual informa-
tion concerning three broad topics, each of which is
discussed below.

Recraitment

Administrators of the five CTI-ATCS programs
were asked to describe the overall approach they used
to attract students to their programs. In addition to
this general question, two specific recruitment issues
were pursued in depth. ‘The first issue was how the
institutions attracted students to a collegiate program
where students may have to pay their own tuition and
living expenses, compared to the FAA Academy, where
students are paid as GS-7 government emplovees
while attending a tuition-free program. In this con-
text, the program administrators were asked to esti-
mate tuition and other costs that are borne by the
students. The second issue was to determine the
institution’s straregy to attract women and minorities.


CTI-ATCS Formative Evaination

Table 6

Gender and Ethnic Data for Three Classes

Currently Enrotied in ATC Ops Program at the
University of North Dakota

i Percent
Classof Gender Race Number of Class

May 1993 Female = White 1 10.0
Male White 9 90.0
Mey 1994 Female «= White 2 2008
Make White 6 60.0
Black 2 20.0
May 1995 Female = White 3 188
Male White i 68.8
Black 2 125
Tabie7
Core Curriculum for Airway Science Program at the University of
North Dakota
Hows Arca Sampte Courses
30 General Studies Composition ¥
American Government
23 = Math, Science, & Technology College Algebra
Introductory College Physics
Meteorology
15 Aviation Introduction to Aviation
Introduction to Air Traffic Controi
Aerospace Legislation
9 — Computer Science Computer Programasing 1
9 Management Principles of Management



CIT-ATCS Formative Svaluation

the class size, one or two students serve as ghost pilots
for one tadar position. With practice, ghost pilots can
“Aly” several aircraft at a time.

The software simulates 2 portion of the Greater
Pieest argh airspace. In all, 22 exercises are available
for PP 247 (Approach Control I), and 20 exercises are
available for PP 248 (Approach Control II). In PP
247, as students first learn radar control, the radar
positions can be set to operate independently of one
another; that is, a single exercise can be run at all four
radar positions simulteneously, or a different exercise
¢an be run at cach radar position. In PP 248, students
iearn to coordinate with other controllers in the same
TRACON facility as well as with pilots and en route
controllers. In this case, the four radar positions can
be set to operate interdependently. For example, the
Greater Pittsburgh airspace can be divided into Ap-
proach North, Departure North, Departure South,
and Approach South with each scope displaying its
appropriate sector.

Beaver County Control Tower

CCBC is the only communiry college in the nation
that staffs and operates a functioning air traffic con-
trol rower. For the most part, air traffic is controlled
by students under the close supervision of their in-
structors who are experienced controllers. The con-
trol tower is used for on-the-job training (OJT) during
the third and fourth semesters in PP 239 and PP 240
(Facility Rating I and 11). The Beaver County Airport
is a general aviation airport onty, and as a result,
seudents do not have an opportunity tc control com-
mercial jet aircraft, However, students do acquire
hands-on experience controlling a variety of general
aviation aircraft, including helicopters.

Before working in the tower, students must have
complete PP 326 (Advanced ATC Ell) and passed the
CTO writen exam. Students begin OJT in Facility
Racing I by working in the flight data and ground
control positions. In PP 240 (Facilicy Rating 1),
stadents work in the local contro! position. OJT
culminates with a student's earning the facility rating
for Beaver Tower.

24

Ctess size

CCBC’s ATC classes are limited to 24 students.
For lab courses (PP 239, 240, 247, and 248), classes
of 24 are divided in half; that is, balf of che class meet
for ewo hours on Mondays aad Wednesdays, while the
other half meet for two hours of lab work on Tuesdays
and Thursdays. No classes meet on Fridays.

Tnstrector: cad Gevelepers

CCBC's ATC program employs six instructors.
These instructors work full-time (i.¢., 37 hours per
week) in the Beaver County control tower and serve as
part-time instructors at CCBC. Three instructors are
retired FAA ATCSs: Two have ATC experience at
non-FAA control towers, and one is a former FAA

employee.

University oF Noat Dakota

The University of North Dakota (UND) is a state
supported institution located in Grand Forks, ND.
UND"s Center for Aerospace Sciences (CAS) is the
second largest college at the university and offers
taining in 2 variety of areas, ranging from flight
training to atmospheric study. Subsumed under CAS
is the Department of Aviation, which offers four
distinct four-year curricula leading to the B.S. degree:
Aviation Administration, Airport Administration,
Aeronautical Studies, and Airway Science.

In February 1991, UND signed a letter of under-
standing with the FAA to become part of the CTI-
ATCS program. This program is incorporated in
UND’s CAS as Air Traffic Control Operations (ATC
Ops), one of six concentrations in the Airway Science
curriculum. ATC Ops has not received FAA funding
directly for program development or student support,
bur the Airway Sciences Program has been the recipi-
ent of $24.7 million in grants during the period FY
1982-1993.

Students
Incoming studeats must meet the general univer-
sity admission criteria, which includes a high schoo!
diploma or equivalency and a minimum score of 21
on the American College Test (ACT). In addition,


FORMATIVE EVALUATION OF THE COLLEGIATETRAINING INITIATIVE —
AIR TRAFFIC CONTROL SPECIALIST (CTI-ATCS) PROGRAM

CHAPTER I
INTRODUCTION

The present report describes an initial formative
evaluation of the Federal Aviation Administration's
(FAA's) Collegiate Training Initiative - Air Traffic
Conrrol Specialist (CTI-ATCS) Program. The pur-
pose of the CTI-ATCS program is to test “. . . rhe
concept tat non-federal, post-secondary educational
institutions can develop, deliver, and implement air
traffic control recruiting, selection, and training pro-
grams” (Draft Revision of FAA Order 3120.26 dated
January 16, 1991). The CTI-ATCS Program is man-
aged by a National Steering Commitree comprising
representatives from Air Traffic, Human Resource
Management, the FAA Academy, and the Civil
Aeromedical Institute. The committee is chaired by
the FAA’s Higher Education and Advanced Technol-
ogy Staff (AHT-30). The detailed responsibilizies of
these organizations with respect to the governance of
the CTI-ATCS Program are described in Order
3120.26. The evolution of this program is summa-
rized in the following section.

CTI-ATCS Procram Bacxcrounn

In 1988, two comprehensive studies (Means,
Mumaw, Roth, Schlager, McWilliams, Gagné, Rice,
Rosenthal, & Heon, 1988; Northern NEF, Inc., 1988)
were undertaken to review the FAA’s system for train-
ing Air Traffic Control Specialists (ATCSs). One of
the more provecative recommendations of the North-
ern NEF study was that nonfederal, post-secondary
institutions (i.e., rwo- and four-year colleges and
universities) be selected to develop and test academic
programs for training fundamental skills and
knowledges related to air traffic control (ATC). The
Northern NEF study acknowledged that their pro-
posal for a collegiate training program wes not a new
idea. In fact, two ongoing FAA programs had been
specifically designed to attract college students co
ATC-related careers: The College Cooperazive (Co-
op) Education Program and the Airway Sciences Pro-

gram. However, both of these programs were con-
ceived as providing training only in rudimentary ATC
skills and knowledges in preparation for more in-
depth training at the FAA Academy. In contrast, the
collegiate program conceived in the Northern NEF
report was more comprehensive, ellowing graduates
vo bypass entry-level training at the FAA Academy and
be placed directly into ATC facilities.

Different arguments have been made in favor of
establishing initial ATC taining programs in non-
federal educational institutions. Some of these asgu-
ments were advanced in the Northern NEF (1988)
study and others have evolved since chen, A synthesis
of arguments from that study and informal conversa-
tions with FAA personnel reveal four general justifica-
tions for the CT1-ATCS program:

1. The knowledges and skills acquired through 2
college education promote better performance and flex-
ibility on the job. At a general level, a college education
is thought to produce a “deeper and wider knowledge
base” in ATC-related subjects, such as communications,
meteorology, and aerodynamics. This knowledge base is
thoughr to be required to cope with anticipated in-
creases in the demands of the controller’s job. At the
same time, collegiste programs can address more spe-
Gific educational needs, such as deficiencies in personne!
management skills.

2. Additional ATC taining programs expand the
current pipeline of trained ATCSs. This expansion is
viewed in terms of both the grearer numbers required to
meet staffing requirements for the National Airspace
System (NAS) and in diversity to include a greater
proportion of minorities and women in the profession.
As staffing needs have abated in recenz years, the poten-
tial of colleges to increase ATCS work force diversity has
become much more important.

3. College training promotes the professionalism of
the ATCS work force. It has been argued thar the level of
controller education should be more congruent with the


EXECUTIVE SUMMARY

This repore describes the initial formative evalua-
tion of the Federal Aviation Administration (FAA)
College Training Initiative — Air Traffic Control
Specialist (CTI-ATCS) Program. The purpose of the
CTI-ATCS program is co test “the concept that non-
federal, post-secondary educational instirutions can.
develop, deliver, and implement air traffic control
recruiting, selection, and training programs” (FAA,
1990) The background and evolution of this program
are presented, along with descriptions of the insti-
tutions that are involved in the training of CTI-
ATCS participants.

The report concludes that the programs at the five
pardcipating educational institutions appear to be
functioning well. Specifically, the programs are mak-
ing innovations in recruitment, selection, and train-
ing that may be of benefit to the FAA. Some progress
in recruiting women and minsrities is being made.
However, the sharply curtailed demand for control-
lers poses a significant challenge to che CTI-ATCS
program. Improvements in program management and
communications are also needed. Overall, the CTI-

ATCS program appears to be gencrally successful, but
further study is required.



Sondents

In addition to meeting general admission criteria at
HU (e.g., high-school diploma or equivalency, col-
lege entrance exams), applicants to the CTI-ATCS
program must meet further, more stringent standards.
The minimum admission requirements for the CTI-
ATCS program include:

* US citizenship;

* fluency in the English language, 2s assessed by an oral

interview and a written essay;

© younger than 28 years of age at the time CTI-ATCS

classes begin;
© two or more years of post-secondary education from.
an accredited junior/community college or univer-
sity, with a minimum grade point average
(GPA) of 2.7; and

+ ability to meet OPM hiring requirements, including
the rnedical evaluation, background security investi-
gation, and the OPM written exam.

The recruiting material (described below) publi-
cizes telephone numbers for HU’s TEAM/ATC pro-~
gram. As a result, the first contact most applicants
have with the program is by telephone. During this
initial contact, applicants respond to a short, stan-
dardized questionnaire that functions asa preliminary
screen. Questionnaire items employ a yes/no format
and address the citizenship, English language ability,
and age requirements for the CT1-ATCS program at
HU. Callers are also asked whether they had ever been
arrested or convicted of a crime, if they have any
medical conditions they feel wouid hinder them as an
air traffic controller, and whether they had completed
two years of post-secondary education at an accred-
iced college or university. Finally, callers are asked
their college GPA and are told about the tuition
waiver for CTI-ATCS courses. If callers provide the
desired responses, their names and addresses are re-
corded, and application forms are mailed to them.

The application form asks for written responses to
the telephone screening questions and for a short essay
about why the applicant wants to be an sir traffic
controller. The primary purpose for the essay is to
evaluate the applicant's command of written English.
A secondary purpose is to examine the applicant's
commitment to air traffic control as a career.

3

CTI-ATCS Formazive Evaluation

if applicants meet the minimum admission ce-
quirements, as determined by their responses on the
written application, they are then asked to visit HU’s
campus for an in-depth interview with TEAM/ATC
staff personnel (If a face-to-face interview is not fea-
sible, the interview is conducted via telephone). The
interview continues the process of evaluating the
applicant's verbal language skills and assessing his or
her commitment to a career in air traffic contrel. In
addition, the applicant's post-secondary education is
discussed in detail during the interview. The purpose
of this discussion is to determine either that appli-
cants have already completed HU’s prerequisites in
general education (40 semester hours) and Airway
Science-related (13 semester hours) courses, or that
they can complete the prerequisite coursework re-
quired by the TEAM/ATC program.

The OPM exam is not a requirement for admission
to HU's TEAM/ATC program. Instead, the exam is
administered a few months prior to graduation. In
agreement with MnATCTC, HU administrators feel
chac delaying che OP exam until after coursework is
completed increases their students’ chances to carn
bonus points. In addition, HU administrators pointed
out that OPM scores for CTI-ATCS graduates are
kept on file for three years. By administering the exam
near the end of the program, test scores are valid for
three years after graduation, as opposed to one year if
administered as a selection device prior to enrollment.

HU is also investigating the use of three personality
tests as admission criteria: the Myers-Briggs Type
Indicator, the California Psychological Inveatory,
and the Employee Aptitude Survey. All students ad-
mitted to the TEAM/ATC program have taken these
tests. HU plans to validace scores on these personality
tests against teacher and student subjective evalua-
tions, and, if possible, against FAA subjective evalua-
tions of performance after they are hired. Because the
personality tests are administered only co those indi-
viduals who are accepted into the program, results
from this research only indicate how well these instru-
ments can predict who will succeed

There are ewo goals to the personality research
program at HU. The first is to determine whether
tzaining in team skills, which is integrated throughout
the two-year program, affects the personalities of the



program outcomes defined ia terms of the progress of
CTI graduates in field training and the success of the

gtaduares on the job.

Oncanizatton oF Reroar
The next chapter of the report (Chapter 2) provides
a description of the methods used to perform the
present evaluation. Chapter 3 describes various as-

CTTI-ATCS Formative Evaluation

pects of the programs in terms of their progress in
recruiting, selecting, and training students. Chapter 4
presents our conclusions and recommendations about
the CTI-ATCS program as it presently stands.


members from one or two CCRM teams act as 2
TRACON facility while a similar configuration of
teams serves as an en route center (i.e., controller,
assistant, pseudo pilot, and pseudo facilities opera-
tor). After team members have rotated among con-
troller positions for their facility (¢.g., among
TRACON positions), teams change facilities; that is,
students who were working as 2 TRACON facility
now work as an en route center, and vice versa. The
teamwork approach is thought to facilitate the devel-
opment of interpersonal skills among controllers within
a facility, among controllers in different facilities, and
between controllers and pifors.

In addition to the radar positions, the Radar Lab is
equipped with a part-task trainer based on the Verbex
sound board and supplied by the manufacturer as an
option to the ATCoachO system. This option is a
voice-activated system in which the voices of ATCSs
and pilots are simulated via a voice generation/recog-
nition system. The Verbex-based system is, in essence,
apart-task trainer used to train fundamental TRACON
and en route skills, such as vectoring and aircraft
separation. ft can be run by 2 student with little or no
oversight from an instructor. There is no concern
about students picking up bad habits because che
system will not respond to improper phraseology,
improper vectorings, and so on.

Although the Verbex-based system can recognize
almost anyone's voice within 4-5 hours under calm
conditions, the voice activation system is quite sensi-
tive co stress-related and other changes in the user's
voice. To bypass difficulties in recognizing particular
voices, the system also recognizes commands entered
from the keyboard. Typed commands, like verbal
commands, must use proper phraseology. The system
responds to a combination of verbal and cyped com-
mands and typed commands can be used to override
verbal commands. For example, if che system cannot
recognize students because they have a cold, the stu-
dent can type, rather chan talk, to the system. It is
unclear whether Verbex is effective at all as a part-rask
trainer, much less used in this fashion where the mode
of responding has been changed.

CTI-ATCS Formative Evaluation

Class size

The first two TEAM/ATC classes at HU each
comprised 18 students. With the current equipment
and level of staffing, HU faculty feel chat they could
comfortably accommodate 2 maximum of 24 stu-
dents. Twenty-four students also fits the two- to
three-person CCRM team configuration currently
used.

Inctractors and

The TEAM/ATC faculty includes two full-time
HU employees and five E. L. Hamm employees!
consultants. In <ddition ro teaching responsibilities,
the HU employees also perform administrative func-
tions for the CTI-ATCS program. One HU employee
is a pilot by background, and the other has experience
as a military air traffic controller. Both were instruc-
tors at HU in Airway Sciences before the CTI-ATCS
program was initiated.

Each E. L. Hamm employee and consuitant has
over 20 years of ATC experience. Two are retired FAA
ATCSs: one en route and one terminal operator. In
addition te controlling traffic, the retired en route
controller served as a facility trainer and has been
retired for two years; the retired terminal operator
assisted with on-the-job training and has been retired
for eight years. The remaining three E. L. Hamm
employees have military ATC experience. Two are
prior Air Force air traffic controllers, and one con-
trolled sir traffic in the Navy. All three prior-military
controllers worked in a tower/terminal radar/non-
radar environment, and all were facility trainers. One
of the prior Air Force air traffic controllers caught at
the Air Force’s ATC technical school for 10 years; the
other is a former high school instructor.

Communary Coutece oF Beaver County
The main campus of the Community College of
Beaver County (CCBC) is located in Monaca, PA.
CCBC’s Aviation Sciences Center is located in a
separate facility in Beaver Falls, PA and is about 15
miles from the main campus. The Aviation Sciences
Center is adjacent to the Beaver Couaty Airport,


DOTIFAAIAM-96/6 A Format:ve Evaluation

of the Collegiate Training
initiative—Aiz Traffic
Oifice of Aviation Medicine Control Specialist
Washington, D.C. 20591 (CTI-ATCS) Program

John E. Morrison

Carolyn Hili Forouhi

Human Resources Research Organization
International, Inc.

Alexandria, Virginia 22314

Dana Broach

Civil Aeromedical Insticute
Federal Aviation Administration
Oklahoma City, Oklahoma 73225

February 1996
Final Report

This document is available to the public
through the National Technical Information
Service, Springfield, Virginia 22161.

@

U.S. Department
of Transportation ©

9960312 006 = saan

ma ee DTIC QUaiiz¥ UsSPRCTED 1



CUl-ATCS Formative Evaination

revealed another, more fandamentai, problem: some
apparent ambivalence of the FAA towards the CTI-
ATCS program itself. The FAA, might consider ap-
pointing a single manager of the CT1-ATCS program
who is knowledgeable about the dezails of each insti-
tution and isa forceful advocate of the programwithin
the FAA. Furthermore, this manager could serve as the
conduit of all communication between all FAA osgani-
zations and the programs, but have the power to make
operational decisions affecting CTI-ATCS, without nec-
essarily consulting the entire stecring committee.

Tus CTI-ATCS Procram 1s General.y
Svccessrvr, sut Fuatuer Srupy ts Neepep

In summary, the CTI-ATCS program was con-
ceived as a bold initiative in ATC training. In large
part, the initiative has succeeded. The individual
programs developed at the five nonfederal post sec-
ondary institutions have demonstrated that they can
develop, deliver, and implement air traffic contol
- recruiting, selection, and training programs. Further-
more, the programs have developed end implemented
thought-provoking innovations in ATC waining.
These encouraging observations should be tempered
by cautioning, once again, that the training effective-
ness of the programs has yet to be measured by
summative evaluation techniques.

Despite the apparent overall success of the pro-
gram, there have been difficulties in che edministra-
tion and management of the program at the level of
both the local institution and the FAA. The schools
still have problems specifying pre-...'y what their
cousses train and what their students are learning.
Also, the insticutionsl recruiting programs have been
less chan fully successful in recruiting large numbers
of women and minorities. From the management
side, the FAA has at times failed to communicate
effectively with the institucions and to make timely
decisions that affect the future of che program. Some

of the reticence of the FAA management may be traced
to their ambivalent attitude toward the CTI-ATCS
program as a whole. These problems are not insur-
mountable and can be corrected through actions taken
at the insticutional or federal level. However, these
corrections will require close cooperation between the
institutions and the FAA, with an eye toward explicit
goals for the program.

‘The most serious problem facing the CTI-ATCS is
the reduction in demand for new entry-level contzol-
lets. Unlike the other problem: discussed above, this
problem is not easily addressed by actions at the
instivutional or federal level. Moreover, it brings into
question one of the fundamental assumptions of the
program: that the FAA will continue to hire these
newly trained controllers. CTI-ATCS graduates are
currently experiencing significant delays in hiring,
and the situation appears to be getting worse. It is
incumbent upon the FAA co determine whether or not
there is enough demand for newly trained controllers
to support five different CTI-ATCS institutions and
the FAA Academy. If not, the role of che CTI-ATCS
institutions must be reexamined vis-a-vis the FAA
Academy and the entire ATC training system. For
instance, the FAA might examine whether the CTI-
ATCS institutions take on other functions, such 2s
providing refresher training and college-credit pro-
gtams for the FAA regions. Alternatively, the FAA
might explore the possibility of redefining the
Academy's role; for instance, UAA has suggested chat
the CTI-ATCS institutions provide all initial training
with the Academy taking on the function of agraduate
school of air traffic control. These possibilities are not
meant to be exhaustive or even representative. Rather,
they are meant to suggest the wide range of questions
and alternative actions that che FAA should consider
with respect to improving the CTI-ATCS program, a
program that otherwise has promise in providing
innovative ATCS selection, recruiting, and training.


CTI-ATCS Formative Evaination

Training

To gain a concept of the overall goal of ATC
training at each of the institutions, the evaluators
asked program administrators to define the level of
job performance that their graduates are expected to
attain as the result of training. Fez terminal ATCSs,
this is usually described in terms of the aumber of
operations per hour, which can be transiated to termi-
nal level. For instance, the goal of FAA Academy
tower training is to produce students who are able ro
handle 60-65 operations per hour, the equivalent of a
Level 11 facility. In contrast, the final objective for the
en route program at the FAA Academy is stated in
terms of fulfilling a particular en route job. As an
example, the goal of FAA en route training is to
produce a student who is a qualified Radar Associate,
that is, completed Phase VIII of the 1988 Instruc-
tional Program Guides (IPGs). The 1988 IPG (p.
VIIIA-3) requires that at completion of Phase VII,
“che developmental will, in accordance with Hand-
book 7110.65:

1. Identify duties of the radar-associated/nonradar
position;

2. Identify aircraft, permanent echoes, weather, etc.
on radar displays;

3. Define beacon code assignments;

4. Describe radar identification procedures;

5. Deseribe transfer of radar identificairon proce-
dures and standard operating practices;

6. Stare radar separation minima;

7. Identify when to integrate the use of nonradar
procedures into a radar environment to ensure positive
separation;

8. Describe procedures for verifying and using Mode C;

9. State procedures for issuing IFR clearances using
radar and nonradar data for departure and airfiles;

10. Enter computer messages for a radat-associated/
nonradar position;

11. State the transition procedures to and from the
primary back-up system; and

12 State the requirements foe position relicf briefings.”

Two specific training issues were pursued in detail.
The first was to describe the curriculum and the
methods by which it was developed. In particular,
evaluator questions were direcred toward uncovering
any task analytic and training design models that were
used to develop the ATC curriculum. The second
issue was the identification of innovative training
technologies that were in place or planned. Innovative
technologies included instructional procedures (e.g.,
team taining) or instructional delivery systems (e.g.
multi-media machines).

Another aspect of training pursued by the evalua-
tors was the number of trainers ard their background.
Acrucial issue pursued was the currency of instructor's
experience. The FAA assigns active ATCSs as instruc-
tors in the Academy. The concern was that CTI-
ATCS instructors were substantially Jess current in
their knowledge of current ATC cules, regulations,
and techniques.


The institutions were also requested to provide data to
substantiate their progress in this regard.

Selection
Pre-Training Screen

From the outset of the program, all CTI-ATCS
students have been required to take the 4-hour written
ATC examination administered by the U.S. Office of
Personnel Management (OPM). This test is designed
to identify chose applicants who have aptitude for the
ATC profession and to select students for the FAA
Academy. The individual CTI-ATCS ~...grams were
originally required to develop “additional” methods
for selecting studexts who demonsizate aptitude for
the ATC profession. The original goal was co develop
a cost-effective method i replace the 9-week screen-
ing process at the FAA Academy. Recently, however,
the FAA has developed a 5-day, computer-adminis-
tered Pre-Training Screen (PTS) to select ATCS stu-
dents immediately prior to the beginning of training.

FAA validation studies, as reported by Broach and
Brecht-Clark (1993), indicated that the PTS was
successful in predicting outcomes in the FAA Acaa-
emy Nonvedar Sczeen as well as progress in on-the-job
training at field facilities. There were suggestions in
May 1993 that all CTI-ATCS students would be
required to cake the PTS bactery as a condition of
employment by the FAA. The plan, et that rime, called
for students to cravel to the FAA Office of Aviation
Careers Assessment Center, located at the Mike
Monroney Acronautical Center in Oklahoma City, to
take the PTS at some point prior to employment by
the FAA. The FAA was to assume student travel and
lodging costs. However, in a letter to the CTI institu-
tions dated December 22, 1993, the FAA’s Assistant
Administrator for Human Resources Management
“suspended” the requirement that the CTI-ATCS
graduates successfully complete the PTS.

The evaluators also soughz to identify any other
admission requirement that effectively selecced stu-
dents for the program. In chac regard, the institutions
were asked to characretize their “typical” students
demographically to obtain an impression of the out-
come of beth che recruiting and selection processes.

CTI-ATCS Formaszive Evaiuation

Performance Verification

Upon completion of training, the FAA plans to
require all CTI-ATCS students to submit to “Perfor-
mance Verification” (PV), an evaluation process con-
firming that students have acquired the basic
competencies needed to start field training at an ATC
facility. PV has two components: (a) a knowledge
component astessed by traditional (i.¢., paper-and-
pencil) methods; and (b) a performance component,
which is conceived as a simulated, bur realistic, ATC
exercise designed to assess those skills and knowledges.
At the outset of the CTI-ATCS Program, the indi-
vidual institutions were initially required to develop
both components of PV chat could be administered
within the objectives and resources of their particular
programs.

More recently, the FAA has moved to standardize
the performance component of PV across CTI-ATCS
programs and the FAA Academy to ensure that con-
trollers areain the same minimally acceptable level of
performance before assignment to a field facility.
Responsibility for PV for both the Academy and the
CTI-ATCS programs was assigned to a single FAA
division (ATZ-400). With regard to the performance
component, it is the responsibility of ATZ-400 to
establish PV requirements in the form of simulation
scenario events (e.g., numbers of aircraft, types of
aircraft, weather) for both terminal and en route
versions of the examination. The individual programs
are required to develop a simulation scenario using
their own cesources to meet these PV requirements.
The institutions administer PV locally, wich ATZ-
400 monitoring the testing process. This new PV
concept was agreed upon in March 1993, and ATZ-
400 provided PV requirements for both en route and
terminal eptions in August 1993 (Manger, Training
Requirements Program, & Manager. Performance
Verification Program). Because of this recent change
in concept, the institutions had not incorporated PV
inco cheir curriculum at the time of the evaluation.
Nevertheless, program administrators were asked to
comment on the PY program and how it might be
implemented at their institution.


CTLATCS Formative Evaluation

Teble 10
Exampies of Innovations identified at Each of the Five CTI-ATCS institutions

Tastitution
* Innovations

Minnesota Aix Traffic Control Training Center (MRA TCTC)

development ard use of unique criterion-related selection instrument

use of computer-based ISD software to develop program

development of CBI courseware that complements curriculum
incorporation of team training to improve management as welll as controller skills

Hampton University (HU)

active recruitment of female and minorities

systematic methods for screening applicants
incorporation of team taining to improve controller interactions
library of locally developed CBI courseware

Community College of Beaver County (CCBC)
incorporation of actual tower training in curriculum

use of BC Airport controllers as instructors

pilot orientation

Tegional interest in program

University of North Dakota (UND)
© locally developed ATC simulation
© — active recraitment and outreach to Native Americans
© — development of administrative and managerial skills
e pilot orientation

eoone eeee

oeoeve

University of Alaska, Anchorage (UAA)
femalefminocity scholarshi

flight orientation with regional focus

use of FAA controllers as instructors

active regional involvemen: in program



CTI-ATCS Formative Evalaaticn

primarily of publicizing the program through UND
student support groups such as the Women’s Center,
Veteran Services, International Scudies Support Cen-
ter, Enrollment Services, and the Dean of Students.

Because Native Americans are the largest ethnic
minority in North Dakota, UND's recruitment ef-
forts have been concentrated on this particular popu-
lation. Initial contece began with ATC Ops faculty
making recruiting visits to the five tribal community
colleges in North Dakota: Fort Berthold Community
College, Little Hoop Communiry College, Turtle
Mountain Community College, Standing Rock Col-
lege, and United Tribes Technical College. These
initial visits did not result in many applications to the
UND prograra. However, the faculty learned that
traditional recruiting approaches are not effective
with the Native American community and developed
programs to address some of the lessons they learned.

One problem encountered in recruiting Native
Americans is that many do no: want to leave the
reservation. To address this problem, UND intends to
establish a distance learning satellite link with North
Dakows’s five tribal colleges beginning with the fall
semester of 1994. The distance learning program is
designed to facilirate transition from the reservation
to UND, and tothe outside world. UND will provide,
via satellite, one aviation course for cach semester of
the freshman and sophomore years. During the sum-
mer between their freshman and sophomore years,
students will artend UND to earn their private pilot
certificate. Students will chen uansfer to UND for
their junior and senior years. The distance learning
program is targeted toward students who may be
interested in a variety of aviation-related careers, not
just air traffic control. Nevertheless, Airway Science
facuity at UND anticipate that the distance learning
program will result in increased Native American
enrolimsnt in che ATC Ops program.

A second outreach program is aimed at Native
Americans in junior high and high school. UND
administrators specify two reasons for reaching out to
pre-teens and reens. One reason concerns the reluc-
tance to leave the reservation described above. it is
hoped that some junior high students will become
interested enough in aa aviation career co consider
leaving the reservation. If so, emotional seperation

from the reservation can begin early and comdinue
slowly over several years, thereby increasing the
individual's probability of succeeding in acereer away
from the reservation. A second reason for reaching out
toward pre-teens and teens concerns alcoholism. Al-
coholism is such a pervasive problem on reservations
chat many potential ATC candidates are ineligibie for
FAA employment due to their inability to pass the
security evaluation. UND hopes to develop enough
incerest in aviation among young people that they will
be motivated to remain drug and alcohol free.

Suedent characteristics

Tabie 6 presents the gender and ethnic breakdown
for ali three classes currently enrolled in UND's ATC
Ops program. Early on, UND established recruiting
goals of 10, 20, 35, and 50% diversity for the first
through the fourth ATC Ops classes, respectively.
Note that UND exactly mez its diversity goal of 10%
women and minorities for the first class (1993). UND
more than met its 20% diversity goal for the second
class (1994) with 40% of the class consisting of
women and minorities. The third class (1995), which
includes 31.2% women and minorities, fell slightly
short of che gos! of 35%.

Program of Instroction
Carricolnm

The ATC Ops program at UND is 125 semester
hours in duration and spans five semesters (including
onc summer session). The prog7am provides instruc-
ton on the terminal option with a focuson TRACON
operations. Students learn to operate the termina!
vadar and radar associate positions. The final training
objective is to produce graduates who are able to
“...utilize ATC skills to control airplanes in a terminal
environment in sccordance with rerformance vetifi-
cation requirements set by the FAA” (personal com-
munication, Bill Droke, July 18, 1993).

Because the ATC Ops program is a concentration
within the Airway Sciences program, students must
fulfill 86 semester houss of Airway Science Core
Curriculum course requirements (Table 7 presents an
abbreviated version of the core curriculum.) One of
these core courses is Avit 102 (Introduction to Avia-
tion), wherein students can earn the Private Pilot



REFERENCES

Ackerman, P. L. (1992). Predicting individual differ-
ences in complex skill acquisition: Dynamics of
ZZ, 598-614.

Assistant Administrator for Human Resource Manage-
ment (AHR-1). (1993, December 22). [Letter to
Robert Powell, Community of Beaver
County]. Washington, DC: Federal Aviation Ad-
ministration Office of the Assistant Administrator
for Human Resource Management.

Broach, D., 8 Brecht-Clark, J. (1993). Validation of
the Federal Aviation Administration Air Traffic
Control Specialist Pre-Training Screen. Air Traf-
fe Consrol Quarterly, 1, 115-133.

Federal Aviation Administration. (1990). A7CS Pre-
bire Demonstration program. (FAA order 3120.26).
Washington, DC: Author.

General Accounting Office (1989). Aviation safety: Seri-
ons problems continue to trouble the ir control
werk force (GAQIRCED-89-112). Washington,
DC: Author.

Manager, Training Requirements Program (ATZ-100)
& Manager, Performance Verification Program
(ATZ-400). (1993, August a ILeser co Garry
Gerome, Hampton University]. Washington,
Federal Aviation Administration Office of Air
Traffic Program Management.

CT1-ATCS Formative Evaluation

Means, B., Mumaw, R., Roth, C., Schlager, M..
McWilliams, E., Gagné, E., Rice, V., Rosenthal,
D., & Heon, S. (1988). ATC training analysis
study: Design of the next-generation ATC training
system (OPM Work Order 342-036). Alexandria,
VA: HemRRO International, Inc.

Northern NEF, inc. (1988). Aligh: plan for the future:

Training controllers for the National Air Traffic

System (OPM Subcontract Agreement Sub-
OPM88-04). Colorado Springs, CO: Author.

Personal Strengths Publishing (1590). Strength Deploy-
ment Inventory. Pacific Palisades, CA: Anthor.

Schlager, M., Roth, C., Roiling, W., Mumaw, R., &
Gagné, E. (1989). AAS transition training study
techuical report (HII Report FR89-19). Alexan-
dria, VA: HumRRO International, Inc.

Scriven, M. (1967). The methodology of evaluation. In
R. M. Tyler, R. M. Gagné, & M. Scriven (Eds.),
Perspectives of curriculum evaluation (pp. 39-83).

Vogler, D. E. (1989). Performance instruction. Eden
Prairie, MN: Instructional Performance Systems,
ine.


applicants to the ATC Ops program must meet fur-
ther, more stringent standards. Minimum admission
requirements for the program are:
completion of freshman year (24 hours) with a mini-
mun GPA of 2.5 on a 4.0 scale;
completion of Avit 102 (Private Pilot), or an FAA-
+ enrolled in/completion of Avit 303 (Introduction to
ATC) with a minimum grade of C;
FAA Class I Medical Exam - ATC Type;
© US citizenship:
*ability to graduate prior to age 30; and
capability of graduating in five semesters after start-
ing the CTI-ATCS program.

Scudents begin the admission process by complet-
ing a written application. In addition to background
deta (¢.g., mame, address), the application requests
information regarding the applicant’s education.
College transcripts, Private Pilot Certificate, and Class
Il Medical Certificate must be submited with the
written application. The completed application is
then used to determine whether the candidate meets
the minimum admission requirements for the CTI-
ATCS program.

Students who meet the minimum admission re-
quirements must complete a battery of three standard-
ized tests. One of these is the Raven Matrices, which
is an aptitude test designed to measure analytic ability
and visual processing and has been termed 2 “nonver-
bal intelligence test.” The students also take two
better known psychological instruments: (a) the Min-
nesota Multiphasic Personalicy Inventory (MMPI), 2
clinical assessment instrument; and (b) the Keirsey
Temperament Sorter, a temperament inventory. The
aptitude, personality, and temperament measures ace
administered, scored, and interpreted by a faculty
member from the UND Department of Psychology.

In addition to the standardized tests, prospective
students must submit to a selection interview. The
selection interview is conducted by a three-member
panel that includes che ATC Ops program director, an
instructor, and a representative of CAS Student Ser-
vices. The structured interview consists of six ques-
tions that are intended co assess the candidate's
motivation for a career in air traffic control. Each
interviewer provides a numerical rating for the

25

CTS-ATCS Formative Evalzasion

candidate's response to each question. Upon comple-
tion of the interview, the interviewers compare their
ratings and attempt to reach a consensus regarding the
candidace’s potential for success as an air traffic con-
troller.

In making selection decisions, the results of the
assessment measures and the structured interview are
reviewed collectively. A score of 20 on Raven Matri-
ces, corcesponding te a percentile score of 85, is used
asa tentative cutoff score for the program. This curoff
is tentative in chat adminiscrators will consider admit-
ting students who score lower than 20 if they have
excellent academic credentials and score well on the
interview. Results of the MMPI and the Keirsey
Temperament Sorter are used primarily to correbo-
tare the results of the selection interview; no strict
selection criteria have been adopted for these inseru-
ments. Although all four measures are used to make a
selection decision, the interview is weighted more
heavily than the others.

As in the previous programs, the OPM exam is not
& requirement for admission to UND's CTI-ATCS
program, Instead, the exam is administered during the
student's final semester. UND agrees chat this policy
increases the students’ chances of earning bones points
and lengthens the period of time that the OPM test
scores are valid.

Taition

Students must assume al] tuition and other costs
associated with obtaining their degree. Tota) cost
{ie., tuition and fees, room and board, and books and
supplics) for one year at UND ranges from $5,166 for
residents of North Dakota to $8,274 for non-resi-
deats. In addition to these costs, CTI-ATCS students
must bear the cose of obtaining 2 private pilct certifi-
cate, which is approximately $5,000. Financial aid is
available on a demonstraced need basis.

Recruiting progrems

UND's recruitment effort for the ATC Ops pro-
gram focuses on women and minorities. Much of the
early recruitment for the program targeted students
currently enrolled at UND, specifically those enrolled
in courses provided by CAS and the Deparument of
Aviation. These efforts, which are on-going, consist


CTT-ATCS Formative Evaination

job responsibilities and civil service rank (GS-12 to -"4)
of an air traffic controller. Also, consistent with other
professions, students should bear a significant portion
of the cost of their own education. A corollary to this
argument is that the CTI-ATCS program can poten-
tially reduce the costs to the federal government for
enery-level ATCS raining.

4, Collegiate prograrns tap technical training exper-
tise available at colleges and universities to develop
innovative approaches to taining. These approaches
include improvements to training systems (analysis,
design, development, implementation, and eveluation), 2s
well as the employment of specific advanced training tech-
nologies (e.g., computer-based tutorials and simulations).

In response to agency and congressional interest inthe
potential of collegiave ATC programs. the FAA en-
tered into a cooperative agreement with the Mid-
America Aviation Resource Consortium (MARC) in
February 1990 for the establishment of an ATC train-
ing program in Minnesota. The FAA was directed in.
its FY 1990 appropriation (P_L. 161 - 164 [H.R
3015], Noveraber 21, 1989) to provide $3.4 Million
to MARC. Similarly, the FAA provided a grant of $5
Million to Hampton University (HU) in 1990. Con-
cinued interest in the program led the FAA to formally
establish the CTI-ATCS program in Janusry 199?
and to solicit the participation of additional schools.
To be selected for the CTI-ATCS program, posi-
secondary institutions were required to submit formal
proposals. The FAA rated the proposals they received
on a number of factors, such as capability to develop
a valid ATC training program, the employment of
advanced training methodologies, and strategy for
aggressively recruiting minority and female students.
Three additional schools were selected in 1991, bring-
ing the total to five institutions:

1. Minnesota Air Traffic Control Training Center
(MnATCTC), Eden Prairie, MN;

2. Hampcon University (HU), Hampron, VA;

3. Community College of Beaver County (CCBC),
Monaca, PA;

4. University of North Dakota (UND), Grand Forks,
ND; and

$. University of Alaska, Anchorage (UAA), Anchor-
age, AK.

These five institutions have developed subszan-
tially different approaches to ATCS recruiting, eelec-
tion, and training. In face, the programs themselves
were initially selected to be highly diverse in at least
two respects. First, the institutions are geographically
dispersed, allowing the CTI-ATCS program to draw
from a large and diverse population of potential ATCS
candidates. For instance, UND and UAA are geo-
gtaphically well suired to recruiting from Native
American populations, whereas HU recruits from an
urban center that has large numbers of African Ameri-
can and other minority inhabitants. Second. the se-
lected institutions themsetves are markedly differenc
in affiliation and charter. The institutions incinde
two public universities (UND and UAA), a privately
endowed, historically black universicy (HU), 2 two-
year, private community college (CCBC), and a non-
degree, six-month technical training program
(MnATCTC).

The diversity in approaches to ATCS training is
consistent with the FAA's desire for the CTI-ATCS
programs to devise innovative approaches to recruit-
ing, selecting, and training. Ac the same time, how-
ever, the diversity of programs makes evaluation ore
difficult. Clearly, the programs should not be directly
compared to determine “the best” approach. Rather,
the present approach assumes that there are certain
fundamenul goals and objectives that every CTI-
ATCS program must address. At the same time, it
acknowledges that the approaches or processes used to
achieve those goals may be substantially different
across programs.

Evauation Onyectives

As a member of the steering committee for the
CTLATCS program, the FAA's Civil Aeromedical
Institute (CAM1) is tasked with monitoring and evalu-
ating the program. The present research is part of this
ongoing evaluation effort. The specific objectives of
this initial formative evaluation are to determine the
degree of innovation that the five selection institu-
tions demonstrate with regard to theis (2) recruiting
activities, (b) selection procedures, and (c) training
methods. This research is part of an larger evaluation
strategy that calls for a furure summative evaluation of


CTT-ATCS Formative Evatuation

includes courses in aviation theory, safety, and
management. Table 4 presents HU's CTI-ATCS
curriculum.

Under contract with the FAA, E. L. Harm con-
ducted sn ATC job and cognitive analysis seview, as
well as an FAA training review. The contract specified
the use of an

ISD approach to curriculum development end the
inclusion of material contained in FAA Academy
IPGs Phases I - EV for the terminal option and Phases
I - 1H for the en rouce option. Course development
began during the summer of 1990 when a training
development tearn was identified. The team consisted
of a military ATCS, a civilian ATCS, a pilot, a
computer programmer, and an industrial psycholo-
gist. The team visited several FAA rower, TRACON,

Table 4

and en route facilities across the US as well as the FAA
Academy. During these visits, the carticulum devel-
opment team gathered information about the job
functions of terminal and en route ATCSs. Material
covered in the designated IPGs were obtained from
the Academy.

As in MnATCTC’s program, HU'’s program is
explicitly designed to integrate theory and application
as soon as possible in the learning experience. Due to
the bands-on focus, the laboratory exercises drove the
course development process; that is, lab exercises were
designed first, and then course lectures were written to
support the exercises. In keeping with this principle,
classes were scheduled approximately one hour before
the lab. In this way, the material covered in class
would be immediately applied in laboratory exercises.

Required Coursework for the B.S. in Terminal and En Route Airspace
ManagemenvAir Traffic Control (TEAM/ATC) at Hampton University

Semester Code Course Title Hours
First - i2 hrs ATC301 ‘Tower Operations 1 4
ATC/CRM3i5 = Tower Labi 3
ATC321 Aviation Theory I 3
Second-12hrs ATC302 Tower Operstions II 4
ATCICRM316 = Tower Lab i 5
ATC322 Aviation Theory IT 3
Third-12brs = ATC401 Terminal Operations 1 4
ATC/CRM41S = Termina) LabI 5
ATC421 Aviation Safety 3
Fourth- 12brs = ATC402 Termdéna! Operations II 4
ATCICRM416 = Terminal Lab Ti $
ATC422 Aviation Management 3
Fifth - 12 hrs ATCA23 Contemporary Aviation Projects 3
ATC431 En Route Operations 4
ATCICRM445 En Route Lab
Sisth-12%s = ATC432 ATC Facility Applications 3
ATCASS ATC Facility Lab 4
ATCACRM499 Combined ATC Operations 5



waffic, The first year includes academic instruction in
ATC history, regulations governing ATC, aviation
weather, and basic air navigation techniques. The
first-year series also includes an internship that incor-
porates field trips to local FAA air traffic facilities.
This intemship is designed to provide an early intro-
duction to ATC problems that are peculiar to Alaskan
airspace.

The emphasis in the second year is on learning
specific ATC procedures and applying them under
realisticly simulated conditions. AT 241 (Aisporr
Traffic Control) provides prerequisite instruction on
terminal controller procedures, which are then prac-
ticed in che simulated facility (Weatherfo:.: Airport)
used in AT 242 (PCIT 1). PCIT I also provides
instruction on basic non-radar contro! techniques. In
contrast, AT 243 (PCIT I) disseminates information
on basic en route radar techniques, which are prac-
ticed on radar simulations in AT 244 (PCIT Ill). Also
included in second-year courses is AT 240 (Opera-
tions in 2 Flight Service Station). This course is
designed to provide the student controller background
knowledge on the flight service station (FSS). Itisalso
helpful ro the student who is too old for employment
as 2 controller but may be eligible for a position in an
BSS.

Terminal Leb

Despice the program's ecaphasis on Lev-i # skills,
terminal controller skills are trained in the context of
a fictional Level [II facility: the airspace around
“Weatherford International Airport.” Procedures are
learned in a lab using scale model aircraft and the
runway configuration painted on the floor. This labo-
ratory is similar in layout and function to the “table
tops” facility at che FAA Academy and Hampton
University.

Kedar Lab

Radar skills are trained with the ATCoach® sys-
tem. The latest version of ATCoach® was installed
after the end of spring semester and was not yet
functioning at the time of the site visit. The capabili-
ties of the system are assumed to be similar co those
described for the ATCoach® systems at che CTI-
ATCS programs at HU and MpATCTC.

CTI-ATCS Formative Evaination

Flight simulation

A key component of UAA’s training program is the
flight simulator, which is incorporated in two ATC
courses. The simulator is a Frasca 141

GAT-1B, which is a single-engine, general aviation
trainer that simulates IFR conditions only. This simu-
lator has been in use in these courses and in other
flying courses at UAA for more than 20 years.

Planned inacvatiens

Several new trainiag technologies are planned for
the UAA program but have not yet been implemented:

1. With the help of the FAA board of advisors, UAA
is planning to build and operate a VFR tower at
Birchwood Airport in the northern part of the munici-

pality of Anchorage. Along the tines of the CTL-ATCS
Program at CCBC, the intent is to provide students
ATC program. Action on this initiative awaits the reso-
lution of several issues, not the least of which is the
acquisition of funding for constructing the tower.

2. Administrators are also seeking to acquire a three-
window, part-task trainer for simulating terminal facil-
ity operations. Program administrators are now
examining candidate systems to be acquired under the
recently awarded Airway Sciences grant. The present
cost goal for this syszem is less than $500,000.

3. Long-range plans for the UAA program include
the incorporetion of ATC and other Aviation: Sciences
courses in UAA's distance delivery system. This system
sepresents a collection of related training technologies
(eg., video instruction, satellite delivery, multi-media
computer systems) that enable college instruction to be
delivered directly to rural Alaskan citizens et some loce!
site. This goal will probably not be realized until lare
1996.

Chass size

As of February 1993, 50 students were entoiled in
both classes of the ATC program. Because of the oper.
enrollment policy, program administrators expect
considerable aterition in the first year. The ideal size
for the program is ebout 20-30 students for first year
and 15-20 for the second. However, UAA program
administrators sre anxious to avoid training more


CTLATCS Formative Evaluazion

students than the FAA can hire and are willing to
adjust that figure downward given reliable projections
of FAA hiring policies.

Instractors

The ATC program presently has only one full-time
instructor—the program director. The university is
seeking authorization for an additional full-time in-
structor. At che time of our visit (June 1993), the
authorization had received approval by the Alaska
state legislature and was awaiting the Governor's
signature.

In 1992-93, the program employed five part-time
instructors. The part-time employees include three
active FAA controllers who teach ATC procedures.
Two additional part-time instructors are used to teach
the first-year incroductory courses: One has « Ph.D. in
human factors; the other has a background as a. com-
mercial professional pilot. Each semester, instructors
receive in-service training through the university on
topics related to instruction, such as coping with
different student personalities/leatning styles, and
discussions of the pros and cons of different testing
techniques.


CTILATCS Formative Evaluation

the final exam for the course. PP 106 enables students to
obtain the required flying hours necessary to earn the
Privace Pitot Certificate. The latter course is sdminis-
tered by fixed base operaters (FBOs) associated with
CCBC and located nearby: Beaver Aviation, Moore
Aviation, and Stensin Aviation at the Beaver County
Airport; Haski Aviation at the New Castle Airport; and
Beaver Aviation at the Butler County Airport. Sradents
are encouraged to use one of these FBOs to fulfill their
flight requirements. Studenzs whe have 2 Private Pilot
Certificate prior to enrollment in the ATC program may
place out of chese ewo courses.

3. Terminal ATC Operations. While carning gen-
eral education requirements, students take prerequisite
ATC courses, where they learn such basic principles as
meteorology, aircraft recognition, and basic phraseol-

ogy. Procedures specific to tower cab and terminal radar
operations are caught in separate courses, which run
concurrently during the third and fourth semesters. The
program has a strong emphasis on the practical applica-
tion of ATC knowledge and skills. With PP 239 (Facil-
ity Rating 1) and PP 247 (Approach Contre! 1), both
third semester courses, students begin co apply the
knowledge learned in the classroom to control air traffic
in the laboratory and in the Beaver County Conrrol
Tower, an FAA Level Il VFR facility. The practical
application of lecture material begins within che Ersx
week of the classes, Furthermore, practical exercises
immediately follow lecture presentations. After attend-
ing a 'ecture on vectoring aircraft, for example, studencs
complete vectoring exercises in the lab that afternoon or
the next morning.

Table 5
Required Coursework for the A.A.S. in Air Tratfic Control at
the Community College of Beaver County
Semester Code Course Title Credit
First- 16/17 credits PP110 © Flight Theory 4
PP123 Airman’s Knowledge 3
HEISS Eaglish Composition 3
MAISS — College Algebra 3
or
MP161_ —s Physical Science 4
Social Science Elective 3
Second - 17 credits PP106 Private Pitot 3
PP121 Aviation Meteorology 3
PP136 Theory of Instrument Fight 4
PP2360— Advanced ATC Ii 4
HE1S6 —_— English Composition If 3
Third - 16 credits PP239 ~—s Facility Rating I 6
PP247 = Approach Control I 4
HL260 Logic 3
Social Scieece Elective 3
Fourth - 16 credits PP240 ~— Facility Rating IT 6
PP248 = Approach Control I 4
BD150 —slatroduction t0 Computers 3
General Education Elective 3



TABLE OF CONTENTS

Page
CHAPTER 1. INTRODUCTION - 1
CTI-ATCS Program Background 1
Evaluation Objectives -~ 2
Organization of Report om 3
CHAPTER 2. METHOD. — - 4
CTEL-ATCS Site Visits ~ 4
Evaluation Topics 4
CHAPTER 3. PROGRAM DESCRIPTIONS —-. 7
Minnesota Ait Traffic Control Training Center 7
Hampton University 12
Community College of Beaver County 19
University of North Dakota 24
University of Alaska, Anchorage 30
CHAPTER 4. CONCLUSIONS AND RECOMMENDATIONS 35
The Demand for CTi-ATCS Graduates Has Been Sharply Curzailed 35
The Programs Appear To Be Functioning Well 36
Programs are Employing Innovations in Recruiting, Selection. and Training —~—— 37
Programs are Converging on Some Common Approaches to ATC Training aeemcncccerenmmncne 37
Instructor Currency is Not a Serious Problem — 39
Some Programs Have Made Progress in Recruiting Females and Minorities 40
Programs Do Not Maintain Consistent Student Demographics 40
Programs Were Developed from Different Job Analyses or No Analyses at All or)
The OPM Test Is Not Used to Select Srudents 4)
Institutions Do Not Understand the Requirements of Performance Verification —----------———. 41
The Effects of Combined Terminal/En Route Curricula Are Unknown 4)
Management of the CTI-ATCS Program Should Be Improved 41
The CTI-ATCS Program Is Generally Successful, But Furcher Study is Needed —-----------------— 42
REFERENCES 43
APPENDIX Al



skills and tasks) and subject areas (factual knowledge)
currently being trained at the FAA Academy. Pro-
grams were advised chat their curricula “. . . shall, ata
minimum, contain the same subject areas and meet
the same training objectives . . . as the FAA Academy
curriculum.” Although this listing of objectives and
subject areas is not a comprehensive job-task analysis,
it provides some of the much-needed information
provided in such an analysis, namely a listing of the
skills and knowledges required to begin OJT at an
ATC facility. Each of the CTI-ATCS programs should
formally respend to the document, including any
omissions that are evident from their own analyses or
their experience training ATC subjects.

Tue OPM Test Is Nor Use to Sexect Srupents

Unlike the FAA, the CT!-ATCS institutions do
not use the OPM test as a selection instrument.
Rather, the programs encourage their students to take
the vest ar the end of their coursework. The institu-
tions point out that delaying the test increases the
chances of carning bonus points for knowledge of
aviation and lengthens the period of time chat the
scores are valid. Indeed CCBC pointed out that their
typical incoming students are not eligible to sit for the
exam. Nevertheless, the OPM test was designed to
measure aptitude for the ATC profession, not to serve
as a measure of what is learned in training. Therefore,
the OPM test should be scheduled to be taken as early
as possible in a student's course of study to determine
if they are good prospects for the controller's job. Ar
the same time, processing the OPM test requires some
peried of time. If students take the test at the end of
their coursework, they are likely to experience a delay
in hiring. 4 reasonable compromise appears to be the
procedure used at MnATCTC, wherein students take
the OPM test about halfway chrough their coursework.
This allows enough time to process OPM test data, as
well as security and medical information, to be com-
pleted by the time the student graduates.

Instrrurions Do Not UNpeastaNo THE
REQUIREMENTS OF THE PERFORMANCE VERIFICATION
In addition to successfully completing training
objectives, students must demonstrate their attain-
ment of the knowledges and skills required for OT by

4

CY1T-ATCS Formative Evaluation

Atthe time of our interviews, the requirements for the
PV exam had not deen detailed by the FAA, and the
institutions had lizerally no idea of the content of this
test or how it would be implemented in their pro-
grams. A model performance test (perhaps the onc
adsninistered at the FAA Academy) is needed ro dem-
onstrate the test content and procedures to the CTI-
ATCS programs. This model and the list of training
objectives and subject areas described above should
provide enough grist for the CTI-ATCS programs to
develop performance exams that can reasonably be
administered at their facilities.

Tue Errscts oF Comsivep TraMinal/En Route
Curnicuta ARE UNKNOWN

UND, CCBC, and HU all offer both terminal and
€n route options. Unlike the FAA Academy, students
in these programs are not assigned to one or the other
option; rather, they receive instruction on both career
options. It could be argued that this single curriculum
is a waste of training time, because students will be
assigned as either a terminal or en route controlier—
not both. On the other hand, it could be argued that
training in both options provides valuable knowledge
about and an appreciation for the job of other control-
less with which any controller must interact. Whether
or not this cross training is beneficial to performance
thould be 2 topic for the summative phase of this
ongoing evaluation. The FAA also needs tc consider
the implications of such cross eraining for controller
career paths. For instance, the FAA needs to consider
whether or not CTI-ATCS graduates from programs
offering both opticas be allowed to change career
paths, thatis, from terminal to en route controller jobs
or vice versa.

Manacement of THE CTI-ATCS Procran SHoutp
Br Improved

More than one program administrator commented
that communications with FAA headquarters was
sporadic and that decision making was extraordinarily
slow. Part of the problem seems to be FAA’s matrix
management method that seeks to involve a cross-
section of organizations directly affected by the CTI-
ATCS program. Our discussions with FAA employees


noted oa the RTF software. Although the RTF may
offer a higher fidelity simulation of radar functions,
the acquisition of such a high-fidelity system such as
the RTF would be cost-prohibitive for institutions in
the CTI-ATCS program. Thus, these low-cost sys-
tems continue to be a reasonable choice for the CTI-
ATCS institutions.

Chuak-and-Apply Approach te Lestructional
Desiga

Three of the CTI-ATCS instructional programs
(MaATCTC, HU, and UND) present students with
practical exercises in controlling aircraft 2s carly as
possible in training. This training design is consistent
with an instructional design principle that training
developers at the FAA Academy call “chunk and
apply.” This training strategy specifies that related
knowledges and skills must be trained together (i.c., as
a chunk), closely followed by their immediate appli-
cation in a realistic problem. Because this approach is
currently in ase at the Academy, it is not, strictly
speaking, an ATC training innovation. Nevertheless,
it represents convergence toward a modern anprosch
to instructional design that is consistent with modern
information processing theories of learning and
memory. (For discussions of the implications of in-
formation processing theory to ATC training, see
Means et al., 1988; and Schlager, Roth, Rolling,
Mumaw, & Gagné, 1989.)

Pilot Perspective ia ATC Training

Three of the CTI-ATCS programs recognize the
value of learning the pilot’s perspective in controlling
air uraffic: UAA requires their students to take four
courses related to flying, including ground instruc-
tion in a simulator; UND and CCBC require chat
their students actually obtain a private pilot's certifi-
cere. Conversations with Academy administrators
indicated that they, too, think the students’ under-
standing of the pilor’s perspective of ATC is worth-
while. However, pilot training cannoc be implemented
at the Academy within the current time and resource
constraints. Thus, this sort of extensive background
instruction is a unique advantage of these CTI-ATCS

39

CTI-ATCS Formative Evateztisn

Training in Team Skills

Both MnATCTC and HU have incorporated ex-
plicit training in team skills, although their emphases
are different. At MnATCTC the emphasis is on team-
building skills in preparation for management and
controller positions, whereas the focus at HU is on the
controller's work to improve inveraccion between pi-
lor and conwoller, as well as among controllers. De-
spite these differences, both programs recognize the
importance of team skills in the ATC profession. As
discussed above, however, it remains to be seen whether
training in team skills has a positive effect on job
performance at either the controller or managet level.

Hands-On Training in VFR Tower

As identified in Table 10, one of CCBC’s most
important innovations is their use of the VFR control
tower at Beaver County Airport to train their CTI-
ATCS students. This is ar: important innovation in
thaz it allows students ro apply the knowledges/skills
they learned in class and labs in an actual job setting.
UAA cecognizes the advantages of the control tower at
CCBC, and has begun to explore the possibility of
building a similar operational tower for training stu-
dents.

Distance Learning Technologies to Reach Remote

Populations

Both UND and UAA are crying to recruit and train
rural citizens (particularly Native American popula-
tions) who are located largely in remote and isolated
communities. These citizens are often reluctant to
move from their homes to a residential campus. A
strategy is evolving at both schools to export some
aviation-related courses to students at the remote
sites. In addition to providing instruction in aviation-
related subjects, it is hoped that these courses will
facilicace the rural scudent’s eventual transition wo
campus life.

Ensranuctor Currency 5s Nor a Serious Proaien
When asked to identify their most important as-
sets, administrators of the CTI-ATCS institutions all
identified their knowledgeable and dedicated faculty.
As the same time, FAA Academy personnel had specu-


CTI-ATCS Formative Evaluation

Taba i
igender and Ethnic Data for First Seven Classes in ATC Program at Minnesota Air
Tratfic Control Training Center
Gender’ Percent
Ethnic Origin Number of Toul
Female
Asian 3 14
Black 6 28
Hispenic 1 os
Native American 0 0.0
White 75 35.2
Male
Asian 8 3.8
Black M $2
Hispanic 6 28
Native American 2 09
White 101 474
Table 2
Required Coursework for ATC Certificate at Minnesota Air Traffic Contro!
Training Center
Quarter Course Cade Course Title
First ATCT100 Introduction to Air Traffic Control
ATCT112 En Route Radar Fundamentals
ATCT131 Introduction to ATC-Radar Associate
Second ATCT211 Advanced Radar
ATCT232 Advanced Noa-Redar
ATCT251 Advanced Radar/Radar Associate



nurturing, assertive, analytical) through self- and peer-
report questionnaires. Then, the individual and peer
complete the questionnaire by assuming two life con-
ditions—normal and stressful. After identifying their
personal styles, individuals work in groups to com-
plete exercises that require working with others who
have similar ordifferent personal styles. In theory, this
teaches students co appreciate different personal styles
and how to work with those differences. After faculty
members become proficient with SDI, they intend to
incorporate it into the current ream building lessons.

To remain in the program, students must maintain
aC” average or better in cach class. Due tothe intense
schedule and the cumulative nature of the program,
there is no opportunity for students to retake a class if
they fail. Therefore, MnATCTC has developed 2
procedure to maximize the studenr’s chances of suc-
cess. Upon entering the program, each student is
assigned a faculty rdvisor, who is typically assigned
three advises. The advisor receives input al most daily
on the progress of his or her advisees through informal
discussions with and voice/electronic messages tof
from other faculty members. The advisor also meets
with his or her advisees on a weekly basis to discuss
their progress. if a student is performing poorly, the
advisor may provide one or more remedial actions,
such as providing tutorial services, referring the stu-
dent to the Computer-Based Instruction (CBI) Lab,
and arranging for tutoring from the course instructor.
Ifa student continues to have academic trouble, the
advisor may counsel the student about withdrawing
from the program.

CBI Leb

The CBI Lab contains eight work stations, each
equipped wich an DOS-based personal computer with
an 80386 processor actworked to a single file server.
MnATCTC’s courseware was developed by its own
computes programmer, and can be categorized into
four groups or phases. The classifications reflect the
evolution of MnATCTC’s program with 2 different
goal underlying che lessons in cach group. The CBI
lessons developed in Phase I provide a crash course in

CTI-ATCS Formative Evaluation

aviation for students who enter the program with no
aviation background. These lessons concern phonetic
alphabet and numbers, phonetic numbers and usage,
weather, aircraft performance, and airspace. Phase II
CBI lessons supplement ATC material taught in the
classroom. These lessons are designed for student self-
remediation and refresher training, and include topics
stich as airspace reclassification, maps, three-letter
identifiers, non-radar homework problems, time and
distance problems, and D-side keyboarding. Deve!-
opment of Phase fll lessons is currently underway.
‘These lessons are designed to serve as part-task train-
ets of specific skills, such as a vectoring exercise
wherein the student vectors aircraft through a maze.
These lessons are also expected to serve particular
functions within the curriculum, such as homework
assignments. MnATCTC’s fourth stage of lesson de-
velopment will include the production of interactive
videodisc programs, as well as faculty authorship of
lessons.

CBI lessons are self-paced, with students able to
enter and exit at any point in the program. Each lesson
incorporates a post test consisting of multiple-choice
items. These tests serve different purposes, depending
on the phase during which they were developed. In
keeping with the self-help philosophy, the post tests
for Phases I and IT provide prompts and immediate
feedback. For example, students may review part of
the lesson before answering a question. If they select
an incorrect response, information is provided to
explain why their response was incorrect. A correct
response is rewarded with positive feedback. In con-
trast, the Phase [II post tests (under development) do
not provide assistance nor is feedback immediate;
however students have an opportunity to review their
answers. After reviewing their responses, students
must indicate that they have completed the test, at
which time the test is computer graded.

The compucer system currently has some limited
capabilities for wracking student use. For example, the
system is capable of recording who uses which CBI
lessons and which lessons are used. Stage If lessons
are intended to be an integral pare of the curciculum.
Along with these kessons, MnATCTC personne! are
writing software programs that will track CBI lesson use


Technical Documentation Page

7. Report No. 2. Goverment Accession No. 3. Recipient's Catalog No.
DOT/FAMAM-96/6

4 Tite cnc Subtitle 5. Report Gate
A Formative evaluation of the Collegiate Training Initiative - February 1996

Air Traffic Control Specialist (CTI-ATCS) program

& Pertomning Organization Code

7. Author(s)
Jehn E. Morrison, Carolyn H. Forohui, & Dana Broach

8. Performing Onganzation Report No.

9, Pertorning Orgon2ation Narne and Address

Human Resources Research Organization International, Inc.
6&6 Canal Center Plaza, Suite 400, Alexandria, VA_22314

10. Work Unit No. CIRAIS)

FAA Civil Aeromedical Instirute
P.O. Box 25082, Oklahoma City, OK 73125

13. Contract or Gtant No. 7

12. Sporeoring Agency Nome and Address
Office of Aviation Medicine
Federal Aviation Administration

800 Independence Avenue, S. W.
Washington, DC 20591

18. Type of Report and Period Covered:

1S. Supplementary Notes
‘This research was conducted under task AAM-93/94/95-B-HRR-133

16 Abstract

This report describes the initial formative evaluation of the Federal Aviation Administration (FAA) College
Training Initiative - Air Traffic Control Specialist (CTI-ATCS) Program. The purpose of the CTI-ATCS

program is to test “the concepe that non-federal, post-secondary educational institutions can develop, deliver,
and implement air taffic control recruiting, selection, and training programs” (FAA, 1990) The background
and cyolution of this program are presented, along with descriptions of ‘the institutions that arc involved in the

training of CTI-ATCS participants. The report concludes that the programs at the five participating
educational institutions appear to be functioning well. Specifically, the programs are making innovations in
recruitment, selection, and training that may be of benefit to the FAA. Some progress in recruiting women and
minorities is being made. However, the sharply curtailed demand for controllers poses a significant challenge to
the CTI-ATCS program. Improvements in program management and communications are also needed.
Overall, the CTI-ATCS program appears to be generally successful, but furcher study is required.

17. Key Words 18. Distibution Statement
Air Traffic Control Specialist (ATCS) Document is available to the public
Training, Program Evaluation, Education through the National Technical
Recruitment, Selection Information Service, Springfield, Va. 22161
19. Seeunity Casal. (of fhis rapert) ‘2. Security Clossif. (of thts poge) 21. No. of Pages 22. Price
UNCLASSIFIED UNCLASSIFIED 52

For DOT F 1700.7 (6-72)

Reproduction of compisted page authored


CTLATCS Formative Evaluarion

programs provide basic (i.c., undergraduate) training
to prepare students for an enzry-level job at lower level
facilities, such as a VFR terminal rypical in Alaska. At
some later point in their ATCS career, they would
then attend the Academy for :dvanced (.¢., graduate)
training to prepare them for a job at a higher level
facility, such as those in Anchorage International
Airpore.

The UAA curriculum has evolved through the
process of deliveting and modifying instruction in
ATC for more than 20 years. Because the curriculum
is not the product of a systemztic training develop-
ment effort, the FAA requested in 1992 that UAA
examine their curriculum with respect to critical skills
and knowledges trained under the old Academy Screen.
The analysis indicared that most of these critical skills
and knowledges were currently being tained in the
curriculum; only minor course revisions were needed
to correct the omissions.

Table 9

The required coursework, detailed in Table 9, is
designed to be completed in two academic years (i.e.,
four semesters) although some students take longer to
complete the progtam. The curriculum calls for 46
credit hours of required coursework in ATC. in addi-
tion, UAA requires the following credit hours in non-
ATC courses: (2) three hours in oral communications,
(b) six hours in written communications, and (c) six
hours of general coursework (humanities, math, 2atu-
tal science, or social science). Thus, a total of 61
credits are required to graduate with the degree of
Associate of Applied Science (A.A.S.). Students wish-
ing to complete a four-year program can apply all
credits earned in che ATC program towards a Bach-
elor of Science, Technology (B.S.T.) in Airway Sci-
ences.

The first year of the curriculum is devoted to
providing classroom instruction on basic skills and
knowledges that are prerequisites to controlling air

Reuired Coursework for the A.A.S. in Air Traffic Control at the

University of Alaska, Anchorage

Course Credit
Year Code Course Title Bours
First = AT140 = Air Traffic Comcol History 3

AT 141 Aviation Weather 3

AT 142 Control Eavironment 3

AT 143 Air Traffic Contre! Regulations 3

AT 144 Air Traffic Control Airborne [ab 2

AT 14S Principles of Flight 3

AT 146 ~~ Az Traffic Control Intern Program 4
Second AT 200 _ Instremeat Grouad School 4

AT 240 = Operations in Flight Service Station 3

AT241 Airport Traffic Coatrol 3

AT 242 Pros. for Control of Instr. Traffic (PCIT) I 3

AT 243 Proc. for Contro} of Instr. Traffic (PCTT) IT 3

AT 244 Proc. for Control of Instr. Traffic (PCIT) UI 3

AT 245 —_- PilovComroller Techaiqeves 3

32


CY?2.ATCS Formative Evaluation

which is approximately 30 miles northwest of the
Greater Pittsburgh International Airport. The CCBC
Center/Airport complex includes the Beaver County
Control Tower, which is equipped, staffed, and oper-
ated by CCBC students and their instructor/supervisors.

CCBC offers four distinct two-year programs in
Aviation Sciences: Professional Piloting, Aerospace
Management, Avionics Technology, and Air Traffic
Control. Inaugurated in 1976, the Air Traffic Control
(ATC) program is a two-year program leading to the
degree of Associate in Applied Science (A-A.S.). In
1982, CCBC began offering an ATC Co-op program
in conjunction with the FAA Eastern Region. In
August 1990, CCBC signed a letter of understanding
with the FAA to join the FAA's CTI-ATCS program
with a terminal option. Az soon 2s the letter was
signed, CCBC began placing their graduates at FAA
Level Il] rower facilities. The program at CCBC dees
not currently receive FAA funds for developing pro-

gtams or for supporting students.
Seadents
Admission and requiremests

Aside from a high school diploma or its equivalent,
there are no additional requirements for admission to
the ATC program. Students must take math and
English proficiency exams developed by CCBC to
assess their readiness for college level math and En-
glish courses. If they are deemed unprepared, they
must take remedial courses, but they are aot usually
denied admission. Although not an admission re-
quizement, students are encouraged to obtain an FAA
Class I medical certificate prior to entering the ATC
program. Until they odtain this medical certificate,
students cannot enroll in PP 106 (Private Pilot) or PP
239 (Facility Rating 1), which are second and third
semester courses, respectively.

Practical considerations prevent CCBC from using
the OPM test as « selection instrument. Mast of
CCBC’s ATC students are recent high school gradu-
ates. In arder vy sit for the OPM exam, an individual
must have thre: years of general experience, four years
of college, or a combination of education and experi-
ence equaling three years of general experience. There

are exceptions to the above experience/education re-
quirements, but the zxceptions still require more
experience and/or post secondary education than an
18 year old hes had an. opportunity to obtain.

Tuition

Tuition fees at CCBC can vary, depending on
whether the student is a graduate of (2) a Beaver
County high school sponsor of CCBC ($3,556), (b) 2
fon sponsoring in-state high schoo! ($7,652), or (c)
an out-of-state high school ($71,748). CCBC admin-
istrators estimate that the total cost of obtaining the
AAS. in Air Traffic Control ranges from $12,000 to
$16,000, including tuition, the cost of obtaining the
private pilot certificate, books, fees, and living ex-
penses. Ir should be noted that during the course of
their studies, students are required to earn a Conzro!
Tower Operator (CTO) certificate and the Facility
Rating oral and hands-on exam for the contre! rower
acehe Beaver County airport. These exams are edmin-
istered at no cost to the student by FAA CTO exam-
iners from the Greater Pittsburgh Airpore Traffic
Control Tower.

Programs

Recruiting fo: the CTI-ATCS program is incorpo-
rated into CCBC’s overall recruiting program. For the
most part, the ATC program relies on the strength of
its reputation to recruit students. To encourage mi-
nority enroliment, CCBC has formed relationships
with 87 organizations having ties to the minority
community in the Pitteburgh area, such as the Urban
Lcague and the NAACP. CCBC sends speakers to
mectings of these organizations to discuss educational
opportunities at the college. In addition, CCBC con-
ducts some specialized recruiting for its Aviation
Science programs through national advertising in
aviation-related magazines.

Although not highly touted, CCBC’s participation
in the FAA's Co-op program is an effective recruit-
ment tool. Participation in the 12-month Co-op pro-
gtam is highly competitive and is not guaranteed. For
a CCBC student to be eligible for selection into the
FAA Co-op program, he or she must:


NOTICE

This document is disseminated under the sponsorship of
the U.S. Department of Transportation in the interest of
information exchange. The United States Government
assumes no liability for the contents or use thereof.



APPENDIX A
Questions for Administrators of CTI-ATCS Institutions
Name:
Tide:
Phone Number:
1. I want to start by getting some basic information about your program.

a. Prior to the ATCS program, did you have an Airways Sciences program in place? How long has it been in
existence?

b. Does your CTI-ATCS program focus on the en route option, the terminal option, or both?

c. How many students have graduated from the CTI-ATCS program? Where have they been placed?

d. What is your present rate of graduation? —_ Do you expect that to increase or decrease in the near future?
e. What are the tuition and other costs that the students must pay?

2. Thave some questions about your overall goals for the program.
a. According to your understanding of che CT1-ATCS program, what are the minimal objectives that al! five
CT1-ATCS programs must meet? Prioritize these objectives.

b, What unique objectives does your program strive to attain? What are their priorities relative to the common

3. The next questions pertain to your acceptance policies.
a. Describe how scudents apply to your program.

b. What are your criteria for acceptance?

Al



CTL-ATCS Fermasive Evaluation

Certificace required for admission to the ATC Ops
program. Like CCBC, UND program administrators
feel that this instruction is valuable in understanding
air traffic control from a pilot's perspective. Students
who have an FAA-certified Private Pilot Certificate
prior to enrollment in the program may place out of

As specified in their agreement with the FAA,
UND developed theis ATC Ops curriculum from an
analysis of the controller’s job. Working with cask
data provided by FAA, UND course developers iden-
tified and documented the specific tasks to be trained
and the level co which those tasks should be trained
(.c., informational, cognitive, or performance). They
then cross-referenced these tasks to the course in
which they are taught. Documentation is available for
Avit 303 (Introduction to ATC), a prerequisite for
admission into the CTI-ATCS program, and for the
ATC Operations courses (i.¢., Avit 306, 310, 401,
404, 406, and 412).

As shown in Table 8, the 39 semester hours of ATC
courses can be divided into three broad areas: ATC
operations courses, aviation related courses, and other
required courses. As mentioned previously, UND's
ATC Ops program focuses on TRACON operations;

Table &

just one course addresses tower cab operations. In Avit
310 (ATC Tower Operations), students are exposed
to the principles of tower cab operations, but they
receive no laboratory training on this subject.

As in MnATCTC’s and HU’s programs, UND’s
curriculum is designed to integrate theory and appli-
cation early on in the learning experience. With the
exception of Avit 310 (ATC Tower Operations),
which is described above, all ATC courses are sched-
uled for two-hour classes. Theory and principles are
presented in the first hour; students then apply those
principles in laboratory exercises in the second hour
ATC Lab. Avie 412 (Advanced ATC Radar/Nonradar
ID) is a comprehensive, lab-oriented course that is
exclusively devoted co application. Students must
demonstrate comprehensive ATC abilities in a UND's
include coordinating with other controllers in the
same TRACON facility, as well as with controllers at
terminal facilities and en route centers.

An important aspect of UND’s ATC Ops program
is che development of administrative and managerial
skills. UND expects that their graduates will be con-
sidered for managerial positions by virtue of their
college degree. To ensure that graduares have a firm

Required Coursework for the B.S. in Airway Science (ATC Ops
Concanivation) at the University of North Dakota

Substantive Area Code Course Tide Hours
ATC Operations — Avit 306 ATC Radar Operatioas I 4
Avit 310 ATC Tower Operations 2
Avit 401 Advanced ATC Radar Operations II 4
Avit 404 Advanced ATC Noaradar t 4
Avit 406 Advanced ATC Noaradar IT 4
Avit 412 Advanced ATC Radar/Noaradar Ill 4
Aviation Related = Avit 250 Human Factors 2
Aviv 402 Airport Planning and Administration $s
‘Other Required Mgmt 310 Organizational Behavior 3
Comm 210 —_ Intro. to Interpersonal Communication 3
Psy 360 Iniro. to Personality 3
PSi 437 Administrative Processes 3



ministrators in theit own programs was evidenced by
their unanimous responses (a) that their ATC pro-
gtams were as good as or better than the FAA Academy's
and (b) thar their graduates would be well accepted in
FAA facilities. Second, in addition to a clear vision of
program strategies and goals, che administrators evi-
denced attention to the tactical details of running a
CTI-ATCS program (e.g., remediation policies) that
were not spelled out in their initial agreements with
the FAA. Third, the programs are operating satisfac-
torily without the benefit of vast resources in equip-
ment and insttuctors. Overall, our impression is that
the individual CTI-ATCS programs have quickly
acquired the knowledge and experience necessary to
train ATCSs in a collegiate context. The FAA aeeds to
tap that experience and act more in partnership with
the institutions to determine the future of the CT3-
ATCS program.

We purposely emphasize the term “appear” to indi-
cate that these comments are largely based on our
subjective reactions to the CTI-ATCS programs, rather
than on objective fact, that is because it is not known
at this point if the schools are producing graduates
who are ready for training at an FAA facility. As
graduates are absorbed into the system, their perfor-
mance must be documented to amass the sort of
evidence required to mount a summative evaluatici:
of the program.

ProGRaMs ane Eupiovinc INNOVATIONS IN
Recxurmne, Sizscrion, AND TRAINING

One of the principle purposes of the CTI-ATCS
demonstration program was for the institutions co
develop and implement innovations in ATC recruit-
ing, selection, and training. Even a cursory examina-
dion reveals that the CTI-ATCS institutions have
developed methods and technologies that are differ-
ent from the FAA and different from each other. Some
of the innovations observed at the institutions are
summarized briefly in Table 10. (More detail on these
and other innovations was provided in Chapter 3.)
This table documents that, as mandated in the origi-
nal FAA order, the CTI-ATCS institutions are in fact
developing and implementing diverse approaches and
technologies related to ATC training.

37

CTI-ATCS Formezive Evalxazion

As encouraging as these innovations may sound,
this does not imply that they are necessarily effective.
Ac the least, however, the innovations imply a variety
of hypotheses that can eventually be tested. For in-
stance, does the team training at HU actually improve
controller efficiency in a realistic work environment?
Does the tower training provided at CCBC facilicate
transition to tower operations on the job? CAMI
should work with the insticutions to desig: research
for testing these, and other hypotheses, as a part of the
future summative evaluation.

ProGrams axE CONVERGING ON Some Comnon
Approacnes To ATC Traintnc

Whereas the programs evidence diversity in inno-
vations, che programs have also developed some com-
monalities in their approaches. These common themes
may have resulted from communication and cross-
fertilization of ideas among CTI-ATCS institutions.
The adoption of the approaches from other programs
also demonstrates that the collegiate ATC programs
are starting to model one another, rather than at-
tempting to duplicate the FAA Academy. This tend
could encourage the development of innovative ap-
proaches to ATCS training. The following provides a
discussion of some of the common innovations that
22 plsscuity impler_zuicd, orare planned to beimple-
mented in the future, at rwo or more programs.

Low-Cost Simulation Systems for Radar Training

The programs have all adopted relatively low-cost
radar simulation systems based oa networks of per-
sonal computers. The system at CCBC was developed
especially fer them by Wesson, whereas the system at
UND (Virtual Controller) was developed by UND
persoanei. The remaining three institutions use the
same system: ATCoach® developed by UFA. A sys-
tematic examination of the ATCoach system at
MaATCTC was performed by the FAA in 1992. The
FAA evaluators noted 66 deviations from fuli fidelity,
49 of which were rated as “critical.” Ac the same time,
they noted that the Academy's high-fidelity VAX-
based Radar Training Facility (RTF) also deviates
from full fidelity: Of the 49 critical deviations noted
in the ATCoach® software, 8 similar deviations were


CBI Lab

Although some lessons address edvanced ATC pro-
cedutes, che CBI lessons are used primarily wo zein-
force material taught in prerequisite ATC courses.
The lab contains 12 personal computers equipped
with 80386 processors. Each computer serves as 2
stand-alone device. Some of the commonly used les-
sons are stored on the hard drive of each computer;
other lessons are available on diskette.

With the exception of an air traffic contro! applica-
tion that utilizes the Beaver County airspace, the CBI
lessons were provided by the FAA Academy. Included
in the Academy-provided lessons are 156 lessons thar
can be grouped into four brosd categories: (a) general
information on FAA employee procedures (e.g., pay
system, employee benefits, drug awareness), (b) intro-
ductory ATC materials (e.g., aircraft identification,
weather, navigation),

{c) en route procedures and responsibilicies, and
(¢) termina) controller responsibilities. Also, among
the Academy-provided software are 87 lessons on
operations at terminal facilities

The CBI lessons are not used as part of graded
coursework; that is, lessons are not assigned as home-
work, posttest results are not counted 2s part of course
gtades, students cannot earn “extra credit” for com-
pleting CBI lessons, and so on. Students use CBI
lessons to supplement classroom and laboratory in-
struction ar their own discretion.

Facility Lab

The purpose of the Facilicy Lab is co train tower cab
operations in PP 239 (Facility Rating 1) and to tran-
sition from tower cab to radar operations in PP 240
(Facilicy Rating Il). The lab contains a table-top
reproduction of the Beaver County Airport, complete
with a control tower, hangers, and scafe-model air-
planes, helicopters, and rescue vehicles. Behind one
corner of the model airport is a representation of 2
small section of the interior of Beaver County's cermi-
nal, including an operable beacon light, a 24-hour
clock, an altimecer, an electronic weather vane, sloes
for holding flight progress strips, and atelephone used
for simulated communications with the Greater Pites-
burgh tower. The table top model replicates Beaver

CTI-ATCS Formative Evaluation

County Airport’s single 4,500 foot runway, bur the
model can be alrered to accommodate at: additional
intersecting or paralle} runway.

In PP 239 (Facility Ktaring 1), laboratory instruc-
tion begins with students moving toy airplanes and
helicopters in response to air traffic control com-
mands issued by fellow students. To transition be-
tween tower and radar operations, the Facility Lab is
equipped with a “master” computer, a transparency
projector, a large viewing screen, three “ghost pilor”
computers, and headsets for both local control and
ghost pilots. The master computer, which is operated
by an instructor, interfaces with che transparency
projector to project Beaver County's airspace on a
large screen. The master computer controls every-
thing that is projected onto the screen, including the
size of the airspace, che number and type of aircraft
that appear in the airspace, the direction from which
the aircraft enter che airspace, weather conditions, and
the presence or absence of a data box beside each
aircrafz.

In PP 240 (Facility Rating II), students are beyond
using the table top replication. In this class, they
primarily use the Facilicy Lab’s consputer capabilities.
Students rotate among flight data, local conrrol, ghost
pilot, and Pirsburgh tower positions. Because the
computer capabilities replicate a radar environment,
the ground control position cannot be simulated.

Rader Lab

The Radar Lab simulates the radar section of a
TRACON facility and is used in PP 247 and 248
(Appcosch Control f and 1), The syscem was devel-
oped specifically for CCBC by Wesson at a cost of
approximately $150,600 and has been in place since
the summer of 1992. The hardware consists of four
tadar positions and eight ghost pilot positions. It is
functionally similar to che ATCoach system described
for MnATCTC and HU. The present section high-
lights the unique features of the system ac CCBC.

The radar positions are each equipped with a head-
set, radar scope, a keyboard, a trackball, an area to
hold flight progress strips, aad an overhead projection
area. A single radar position is connected to and can
contro! up to four simulated aircraft. Depending on


CTTL-ATCS Formative Evaluation

CHAPTER 3
PROGRAM DESCRIPTIONS

This chapter presents results from the interviews.
The answers are organized as descriptions of the
individual CTI-ATCS programs. These essentially
qualitative but objectively verifiable descriptions pro-
vide a common basis from which the individual pro-
gtams can be compared and contrasted. Evaluative
cornmenss on all five programs are provided in Chapter 4.

This chapter is divided into five sections corre-
sponding to each of the five CTI-ATCS institutions.
Each section is divided into two subsections. The first
subsection (Students) describes the sorts of students
that have beer: recruited and selected in the programs,
thereby integrating those two evaluation ropics. In
addition, the first subsection includes the specifica-
tion of tuition costs incurred by students. The second
subsection (Program of Instruction) contains a dis-
cussion of training issues, including detailed descrip-
tions of curricula, instructional laboratories, class
size, and instructors. This framework permits com-
parisons of the instructional programs at each of the
five institutions. In addition, the descriptions high-
light any major aspect of recruitment, selection, or
instruction thaz is unique co a particular program.

Minnesota Aim Trarerc Controt Traininc
Center

The Minnesota Air Traffic Control Training Cen-
ter (MnATCTC) is located at the Flying Cloud Air-
port in Eden Prairie, MN, which is about 15 miles
west of Minneapolis. Unlike other CTI-ATCS pro-
grams, MoATCTC is not located on or directly asso-
ciated with 2n educational institution. However, in
partnership with the Minnesota Department of Trans-
portation, MnATCTC has a consortium agreement
with several educational systems, including the Uni-
versity of Minnesota, the Minnesota State University
System, the Minnesota State Board for Community
Colleges, the Minnesota State Board of Technical
Colleges, and the Minnesota Higher Education Coor-
dinating Board, through the Mid-America Aviation
Resource Consortium (MARC).

MnATCTC was the first institution to be selected
for the CTI-ATCS demonstration program. In Febru-
ary 1990, the FAA entered into a cooperative agreement
with the Minnesota Deparement of Transportation to
develop and administer 2 prototype six-month non-
degree ATC program. The first (pilot) class began in
January 1991. At the time of the visit (May 1993), the
seventh class of students was enrolled in the
MnATCTC program. From FY 1990-1993, the
MnATCTC program has received $10.65 million in
congressionally mandated funds to develop the pro-
gram and to support students.

Stadents

Admission procedures and requirements

To be admitted to MnATCTC’s program, candi-
dates must be U.S. citizens and younger than 30 years
of age. Candidates begin the admission process by
completing a written application form. They must
submit college cranscripts demonstrating successful
completion of a two- or four-year degree progzam. If
an applicant’s degree is not in aviation, he or she must
also present a certificate indicating completion of at
least one aviation course, obtained from either a fixed
base operator or an accredited college or technical
school. In addition, candidates must submit an FAA
Class II medical certificate. In the final step, candi-
dates must agree to a structured interview w:th repre-
sentatives of the program. Interviews are conducted
face-to-face if possible, but can be conducted by
telephone if need be. The interview is used to assess
the applicanes’ communication skills, level of matu-
tiry, and ability to pass a security check. Ic is also used
to provide a realistic preview of the job of an ATCS,
as well as what to expect while a student at MnATCTC.

Although some candidates take the OPM exam
prior to applying to the program, the exam is not an
admission criterion. Instead, the OPM exam is ad-
ministered at the end of the frst quarter (approxi-
mately 11 weeks into the program) to students who
have not taken the exam previously. There are two
teasons for this policy. First, OPM examinees can earn
bonus points for their knowledge of aviation. By


CTI-ATCS Formative Evaluation Appendix A

4. Related vo acceprance issues are rwo FAA-mandated selection instruments.
a. How does the OPM written test fit into your program, and does it determine who you accept or retain in
the program?

b. How does the FAA five-day computer-based screen fit inte your program, and does it determine who you
accept or revain? .

5. The next questions relate to your recruiting strategies.
a. What sorts of recruiting strategies do you have ro attract students to your program?

b. I’m particularly incerested in how you artract students to attend your CTI program and pay tuition when
students at the Academy are, in essence, paid to attend their program.

¢ Do you have any specific Affirmative Action (i.c., diversity goals)?
d. What strategics are you using or plan co use in atcracting, retaining, and graduating women and minorities?

6. The next questions concern your curriculum.
a. Please describe your curriculum in general cermys and tell me how it was developed. [Any printed materials
would be belpful.}

b. Describe any training innovations that you have implemented or are planning to implement. [“Innova-
tions” are defined relative to the Academy]

c What is terminal learning objective for the program? {What job will your graduate be able to perform
immediately, or what level of job proficiency will chey have attained at graduation?}
7. _Thave some specific questions regarding your instructional staff.
a. How many inscructors do you currently employ?

b. Whar is their background?

c. Does your stuff have FAA controller experience? How recent is that experience?


CTI-ATCS Formative Evaluation

Program of instruction
Carriculum

The ATC curriculum at MnATCTC, derailed in
Table 2, is 22 weeks in duratien and is divided into
two 11 -week quarcers, The curriculum is very intense,
requiring that students spend approximately eight
hours per day in class or in lab. Although MnATCTC
does noz confer an academically recognized degree,
students are awarded a certificate upon successful
completion of the program.

MnATCTCis the only CTI-ATCS institution that
provides instruction for che en route option only.
Students learn to work both the en roure radar contro!
and the radar associate positions. However, the final
training objective is that students be able to fill the
tadzr associate position upon placement in an en
rouse center. MnATCTC’s agreement with the FAA
outlined certain procedures that were to be used in
curriculum development. Specifically, MnATCTC
was to conduct an ATC job and cognitive analysis
review, as well as an FAA training review. The agree-
ment further specified the use of an Instructional
Systems Design (ISD) approzch to curriculum devel-
opment and the inclusion of marerial contained in
FAA Academy Instructional Program Guides (IPGs),
En Route Phases [ - HE.

After conducting the required reviews, MnATCTC
used the Vogler Curriculum-Pedagogy-Assessment
{CPA) Model (Vogler, 1989) to develop their CTI-
ATCS curriculum. The Vogler CPA Model is an ISD-
type approach to curriculum development, which
ensures that the knowledge and skills taught in the
curriculum are directly traceable to specific job tasks.
MnATCTC uses PEAKS, a curriculum development
software packege, which applies the Vogler CPA
Model, zo produce all their courses, lessons, and
exams.

A general principle used by training developers at
MnATCTC was to integrate theory and application as
soon as possible in che learning experience. This
principle was implemented by che following three-
step format. First, theory and principles are intro-
duced in a shore lecture format. Second, students
immediately begin to apply those principles by work-
ing on performance-based, written exercises in class

10

uader the guidance of the instructor. Third, students
move (after a short break) to MnATCTC’s Radar Lab,
where they apply the osincipies in laboratory exercises.

Tastraction in team skills

Because their incoming students have college de-
grees, MnATCTC anticipates that its graduates will
eventually move into FAA managerial positions. To
help prepare students for management, as well as
controller positions, MnATCTC has included team
building exercises in their curriculum. These exercises
are designed to demonstrate (a) the importance of the
senses (i.¢., speech, auditory, vision, and touch) in
communication and task performance, and (b) the
ways that people rely on each other to communicate
and to complete tasks. The activities require students
to work togethers in small groups to accomplish simple
tasks. The casks are simple physical activities that are
unaffected by aptitude or athletic abilities. For ex-
ample, one exercise requires that group members
stand in a circle with hands joined. Two members join
hands through a piece of surgical cubing. Keeping
their hands clasped, the group must pass the tubing
around the circle once. The exercise is timed so the
group must work as quickly as possible.

In addition to the activities described above, stu-
dents develop team building skills by working in
study groups and with iab partners. The faculty deter-
mines lab partner ossigaments (i.¢., radar controller
and pseudo pilot operator), and assignments change
each day. By changing partner assignments, students
learn to work with people who have a variety of work
styles. Team skills are particularly relevant in
ATCT251 (Advanced Radar/Radar Associate) where
studeats are organized into groups of four (i.e., radar
controller, radar associate, pseudo pilot, and reliever)
to complete complex exercises. Students are also ex-
posed to Crew Resource Management (CRM) train-
ing, which has been successful in training team building
skills for cockpit crews.

Faculty members are currently learning the Strength
Deployment Inventory (SDI), an off-the-shelf rrain-
ing program designed to build team skills (Personal
Strengths Publishing, 1990). In the first stage of SDI,
individusls identify their own personal style (e.g.,