ET Training Performance

To obtain information on the training activities of project evaluatior
participants, a Training Progress Report was sent to supervisors of
participants with one or more years in the agency. Responses were received
for 12 ASC HIRES and 12 TRAD HIRES. These sample sizes were too small to
analyze categorical data statistically; ther2fore, the findings are reported
for descriptive purposes only. The majority of ETs, regardless of hiring
status, completed at least one directed study/correspondence course within
their first year; half of the ASC HIRES and slightly more (63.6%) TRAD HIRES
received Computer-Based Instruction; 40% of TRAD HIRES and 43% of ASC HIRES
completed one or more field courses; and 58.3% of ASC HIRES and 63.6% of TRAD
HIRES enrolled in one or more resident/ Academy courses. Only one TRAD HIRE
failed any coursework. Twenty-five percent of ASC HIRES successfully
completed one or more performance examinations whereas two-thirds of TRAD
HIKES completed at least one.

The training progress of most participants was thought to be on schedule or
ahead of schedule (see Table 6). Again, this information is provided as
background on how the two samples are performing and does not imply how ASC
and TRAD HIRES will do in the future. Finally, supervisors were asked to:
(a) characterize the participant's ability to apply coursework; (b) assess
the pace of the person's OUT progress; (c) evaluate his or her ability to
apply what was learned in 0JT; and (d) rate the person's training performance
as comparec with a1] other ETs. Although the sample sizes are small,
statistical comparisons of the rating data using ANOVA were possible. The
ratings were completed using : five-point scale in which a rating of "3" is
average. In al} cases, both AS and TRAD HIRES average ratings are at least
3.0 suggesting that neither group is subpar. It was found that the pace of
the ASC HIRE sample's OJT progress (M = 4.17) was rated as slightly (but

significantly) faster than that of TRAD RIRES (M = 3.42) (F (1,22) = 5.09, p
< 05).

ASI-OP Training Performance

Training Progress Reports were also administered foi both ASI occupations.
Data were available for 12 ASC HIRES and 27 TRAD HIRES. Within the past
year, the majority of participants completed at least one resident/Academy
course (ASC: 100.0%; TRAD: 65.4%); received OJT (ASC: 100.0%; TRAD:
66.7%); and worked independently (no lcngsr receiving OUT) on at ieast one
major job function/task (ASC: 66.7%; TRAD: 92.6%). Again, this information
cannot be analyzed statistically at this point but is provided to describe
the training activities of participants. Most participants were either
consistent with or ahead of what was expected with regard to their training
progress (see Table 6). ASI supervisors were asked to rate the same aspects
of training performance as ET supervisors. The only statistically
significant difference between ASC and TRAD HIRES was with the perceived pace
of OJT progress (F (1,35) = 5.02, p <.05). ASC HIRES were perceived as
moving at a somewhat slower pace (M = 3.67) than TRAD HIRES (M = 4.32)
although the ASC HIRES average rating on this item was acceptable.

ASI-AW Training Performance

Training Progress Report responses were received for 7 ASC HIRES and 50 TRAD
HIRES. Once again, information is provided only for descriptive purposes and

18


as

20

15

10

NUMBER OF RECOGNIZED PROGRAMS

AWS MGMT AIC SYS COMP sc?

YR OF RECOGNITION

1936
1985
1984

AY MNT SLEC SYS

AREA OF CONCENTRATION

nN

wav Science Recognized Programs:

Areas of Concentration by Year.


having tasks that used many skills and abilities (M = 3.22) than did their
TRAD HIRES counterparts (M = 3.92) (F (3,186) = 2.65, p = .05); the remaining
differences within each occupation vere non-significant. No differences
between ASC and TRAD HIRES were observed for the following perceptions
regarding the extent to which each was provided by tneir FAA job: (a)
opportunity to work with competent people; (b) tasks that require a number of
skills and abilities; or (c) the opportunity to make a substantial impact on
the lives of others. in addition, tne foliowing factors were rated as no
More important to one group than the other: (a) intellectual challenge; (b)
autonomy; (c) working with competent people; (d) the opportunity to make an
impact on the tives or work of others; or (e) the opportunity to contribute
to decisions that affect one's job.

Project Objective 7: To Increase Femaie and Minority

Representation Within the FAA

There was insufficient data to adequately address this issue. Statistics
comparing the demographics of ASC and TRAD HIRES we~? possible only with the
ATCS (T/E) sample. No differences in the representation of women and
minorities were observed. However, in both ASI~OP ana ASI-AW, the
percentages of women and minorities in the ASC HIRES sample were
substantially higher than those of TRAD HIRES (refer to the relevant Project
Participant sections earlier in this report). Again, without statistical
comparisons, it cannot be determined whether this was likely to bea
consistent pattern or merely due to chance. Clearly this is an aspect of the
Project that requires close examination in the future.

Project Objective 8: To Improve Employee Perceptions of the FAA

This project objective is designed to examine feelings that employees have
regarding their job, supervisor, and coworkers. Several questions from the
Biographical Questionnaire relate to expectations regarding satisfaction with
general work factors. Four satisfaction expectation scales were developed,
two of which were relevant to this objective. Again, these data were
examined only for ATCS ( /E). No differences were found with regard to the
extent to which participants expected to be satisfied with the "basics"
(e.g-, salary, benefits) as measured by the Basics Scale. In addition, the
Scale of Management Focus, which measured the extent to which employees
believe in the FAA management structure, was developed. No statistically
significant differences were observed between ASC and TRAD HIRES.

Of greater interest is the reported satisfaction of employees with experience
in the agency. Five factors were identified from the satisfaction items in
the Status Survey. The resulting scales tapped general job satisfaction,
supervisor satisfaction, coworker satisfaction, and persona? contro?. No
statistically significant differences between ASC and TRAD HIRES were
observed with the General Job Satisfaction Scale, Supervisor Satisfaction
Scale, or the Coworker Satisfaction Scale. With regard to pay satisfaction,
ASI-OQP ASC HIRES reported slightly less satisfaction with their pay (M =
2.00) than did their TRAD HIRE counterparts (M = 2.94) (F (3,186) = 3.58, p <
05). No differences du2 to hiring status were significant with the other
occupations.

22


Demonstration Project during its first 4 years. The agency report on which
this paper is based (Clough, 1987) provided the empirical support for an FAA
request to the Office of Personnel Management to extend the duration of the 5
year demonstration project an additional 4 years (with 1 more year for
evaluation purposes). Those interested in more cpecific information on many
facets of this evaluation effort are referred to the complete summative
evaluation report (Clough, 1987).

Organization of This Report

In the Method section, demographic and other characteristics of the
previously described project evaluation samples are presented. In addition,
the questionnaires, supporting databases, and data collection procedures used
to execute the project evaluation are briefly reviewed. The accumulated
evidence for each of the eight project objectives is then presented in the
Results section.

The Results section was organized on the basis of organizational goals which,
in turn, subsume specific Airway Science project objectives (see Appendix A).
An initial attempt was made to combine the data across occupations when
occupation per se was not of central importance to the project objective.
The relative influences of hiring status and occupation could then be tested.
However, at times it appeared that there were real differences in the
response patterns of individuals in each occupation. When empty cells
prevented an analysis of the interaction between hiring status and
occupation, this statistical approach was abandoned and examination of only
the ATCS occupation was reported. In other words, when it was not possible
to compare ASC HIRES and TRAD HIRES within each occupation, only information
relevant to the ATCS ASt and TRAD HIRES is presented. Ali statistical
analyses of the items in the Biographical Questionnaire include only Air
Traffic Control Specialists (Terminal/En route) for it was the only
occupation in which both ASC HIRES and TRAD HIRES responded.

Two occupational groups are not described after the next section of this
report although both are currently under the purview of the Demonstration
Project evaluation. The occupations are (a) Computer Specialist (GS-334) in
which one Airway Science candidate had been hired and there were no
comparison group members, and (b) Air Traffic Control Specialist (Flight
Service Station) in which one Airway Science candidate had been hired. It
was inappropriate to even include these people in overall comparisons between
ASC and TRAD HIRES since it was not possible to determine occupation-specific
influences. [Note: The original summative evaluation report (Clough, 1987)
stated that a second Computer Specialist had been selected; the individual
declined the appointment following the completion of that report and
therefore was not included in the above counts.]

METHOD

Project Participants

Air Traffic Controi Specialists (Terminal/En route} [ATCS (T/E)]. As of May
32, 1987, S8 ASC HIRES and 2774 TRAD HIRES had entered on duty. (Actuatiy,

according to our records, 116 Airway Science candidates had been selected;
however, all were not yet with the agency.) Both the ASC and TRAD HIRES
samples were predominantly men (ASC: 83.7%; TRAD: 85.7%) anc nonminority

3


O4TH TOUCT3TPO4) = GYYL 64TH edUueTOS ADMITY = SMY  ‘SSeUTYZJOMUTY - 4OJDedSUT
Ay@j0$ UOTJOTAY = MY-ISVY  {sUOyIDUEedD - 4oyDedsUT AJesDS UOTIOTAY = dO-ISY ‘UOTOTUYDe, SOTUOUZDeTZ = 13 = ‘O3I0N

£2 t O° 0 o t) £'8 t 0 0 6 1 UOTIDATIOW JO 4907
oO” 0 o" 0 o 0 O° 0 aot @ 0° 0 03 peeuBy AT TOUy6T40
Udy} STQDTyOAY ew) BuTUTOU, sseqy
O° 0 0° 0 0° 0) O° t) O° 0 0° 0) sesunog Burssog
AATMOTISTG POH 4O/puD peTToO; SoH
£6 4 com L e4 L oO 0 O° 0 oO oO @TQDT TOAy
ION SOSUNOD JUEPTsey Asosseoon
188 9 ef ase 9 es6 ¢2 £16 UL ese OL 6°06 OL elqoottddy yon |
s@TNPEeYydsS PpuTyeg ST vosoeY
zh 6h tus 4 60" 6 S'Sh Ss o'sz cuz ¢ UOT OATIOW UGTH
oO 0 a 0 ° oO 0 £8 t 0° 0 03 peeuby AT Touy6y40
VOY ETQDTTOAY ewT) BuyUTOUL evow hat
sez ft ° 0 get ¢ o 0) £'8 L ze z eouet4edx3/eGpetmouy sotud
sesunog jueptsey so AIT Tqoryoay
e“e zt gez 2 $'Sh ob s'4s 9 ees £ ss 9 e1qnotTddy 30N
te[TNpeyos jo poseuy S} uosDeYy
zor 6s oO 0) “ae L 2's t “gol 2 L'6 L @{mpeyss purneg
6% 2 999 4 9'6z OS ese of “iy G gle eInpeyss Jo poeyy
6% %@ eee 2 4°99 BL eso o£ “iy Gg geo £ Pez00dxz UITM 7UEISTSUCD
:pezoedxy 03 EATIOTeY SSeuBoug TONQoy
s u $ u ¥ u % u % u s u
aval osv aval osv Ove osv @4TH JO ODAL
AV-ISV dO-ISV 13 *NOTLYdNI90

(2861 ‘zt Aow yo so)
SOUTH TOUOTZTPO4) puD edUeToS Aomaty so sseuboud Guyupouy “9 eTqD)


References

Clough, D. L. (1986a). Airway Science Curriculum Demonstration Project: An
initial formative evaluation and a proposal for modification of the
research design. Oklahoma City, OK: Federal Aviation Administration.

Clough, D. L. (1986b). An interim report on the Airway Science Curriculum
Demonstration Project: Clarification and elaboration of the project
design. Oklahoma City, OK: Federal Aviation Administration.

Clough, D. L. (1986c). Airway Science Curriculum Demonstration Project: The
second formative evaluation report. Cklahoma City, OK: Federal Aviation
Administration.

Clough, D0. L. (1987). Airway Science Curriculum Demonstration Project: The
first summative evaluation report. Oklahoma City, OK: Federal Aviation
Administration.

Collins, W. E., Boone, J. O., VanDeventer, A. D. (1981). The selection of air
traffic contro} specialists: History and review of contributions by the
Civil Aeromedical Institute, 1960-80. Aviation, Space, and Environmentat
Medicine, 52, 217-240.

25



(a) most hires after their first year are stil} in training status (ATCS
Supervisors in particular responded negatively to evaluating the management
potential of an individual who has not yet been able to demonstrate whether
he or she is capable of being a competent controller); and (b) in the ASI
Profession especially, ASC HIRES enter the agency at a much lower grade than
the majority of TRAD HIRES and therefore might .be expected to have Tess
opportunity to demonstrate supervisory skills than their TRAD HIRES
counterparts.

As expected, there were significant differences between the managerial skills
of the ASC HIRES and TRAD HIRES samples within certain occupations (as
Measured by the MSS) (F (3,675) = 2.90, p < .05).  £&T ASC HIRES were
perceived as showing greater skilis than ET TRAD HIRES. This measure is
likely to become increasingly accurate as each supervisor has a. longer time
to interact with his or her employees.

The next set of items was contained in the Status Survey and was answered by
Participants with one or more years in the FAA. No significant differences
were found with regard to the extent that respondents: (a) felt
intellectually challenged by their jobs; (b) perceived advancement
opportunities; (c) indicated that becoming a manager or supervisor was
personally important; or (d) were satisfied with their chances of receiving a
promotion.

Project Objective 6: To Attract Individuals with Greater Awareness

of and Skills in Maintaining Positive Human Relations

Relevant Biographical Questionnaire items referred to the importance of and
expectations regarding skill utilization, participative decision-making, and
other intrinsic factors of the job. Selected expectation questions were
combined to create. the Work-Reiated Challenge Scale to address aspects of
this issue. Again, the data were analyzed using only the ATCS (T/E) sample
and no significant. differences on the basis of hiring status were found. In
fact, both ASC and TRAD ATCS expected to be challenged by the job; the
average response for the two groups was 4.1 on.a_ five-point scale.
Similarly, the occupational choice items of intellectual challenge, autonomy,
opportunity to work with competent people, ability to control workload, and
Prestige of the job were examined and were found to be unrelated to hiring
Status. Because of the agency's strong concern with the human relations
skills of its supervisors, several of the questions previously discussed
under Objective 5 are relevant. ‘The MSS includes interaction skills such as
Oral communication and interpersonal skills. No significant differences were
observed with these individual ratings.

Several questions on the Status Survey also pertain to an interest in or
feelings toward human relations concepts. It was found that ASC HIRES as a
Group perceived slightly less autonomy in their jobs (M = 2.88) than did TRAD
HIRES (M = 3.42) (F (1,186) = 6.40, p < .05). Although the Hiring Status x
Occupation interaction overall was not statistically significant, the above
finding was especially apparent with ASI-OP participants wherein ASC HIRES
had an average rating more than one point lower than the average of TRAD
HIRES. In reflecting on the entry grades (and probable experience} of the
two groups, differences in actual work assignments may account for the
previous finding. Also, ATCS ASC HIRES assigned slightly less importance to

21


initial qualification screen was established and terminal/en route option
placement was moved to after the pass/fail screen. As a result of the
changes to the pass/fail screen program, the former terminal and en route
screens and the current “common screen" must be examined separately. One
component of the screen which remained consistent during the transition was
Phase II. This was exclusively classroom instruction designed to provide
students with basic knowledge of aviat*on and air traffic control. In
comparing the performance of ASC and TRAD H:RES in Phase II, mo differences
on the basis of hiring status were observed.

Former Terminal and En route Screen Programs. Since there appeared to be no
pattern in the assignment of ASC and TRAD HIRES into terminal and en route
screens, the two screens were combined for the purposes of analysis. Thirty-
five ASC HIRES and 323 TRAD HIRES participated in the project evaluation. As
Table 4 indicates, the pass rate of ASC HIRES was significantly ower than
that of TRAD HIRES (ad = 10.53, p < .01).

Table 4. Pass/Fail Rates in Non-Radar Training
for Airway Science and Traditional Hires
{as of May 12, 1987)

FORMER TERMINAL AND EN ROUTE Pass Fail With raw/ Total
PROGRAMS : Incomplete
n % n % n % n h
Airway Science Hires 15 42.9 19 54.3 1 2.9 35 100.0
Traditional Hires 211 «65.3 so 27.9 22 (6.8 323 100.0

CURRENT COMMON SCREEN PROGRAM:

Airway Science Hires 27 44.3 27) 44.3 7 11.5 61 100.0
Traditional Hires 1358 65.2 596 28.6 128 6.1 2082 100.0

Common Screen Program. Sixty-one ASC HIRES and 208Z TRAD HIRES had completed
the Screen as of May 12, 1987. As Table 4 suggests, athe pass rate of ASC
HIRES was significantly lower than that of TRAD HIRES 0? = 11.68, p < .005).
It would be useful to look at the relative performances of the various
subgroups of ASC HIRES as classified by the eligibility scheme previously
described; however, cell sizes were too small to identify any trends. Since
it is important to determine whether specific subgroups of ASC HIRES are
performing much worse (or better) than the others, larger sample sizes are
necessary to fully assess ATCS training performance.

Components of the non-radar screen were examined to identify those aspects of
the program responsible for the lower ASC HIRES performance. Briefly, it was
observed that ASC HIRES scored significantly lower on the laboratory

16


substantially between the two groups, although proportionally more ASC HIRES
took more aviation credit hours. Tests of statistical significance could not
be run on any demographic information due to the small sample sizes.

Since the Airway Science register permits entry at the GS-7 level only, all
ASC HIRES began at that grade; on the other hand, over 80% of the TRAD HIRES
started as GS-11s or GS-i2s during their ASI appointment which coincided with
their participation in this project. Once again, GS-7 is a lower than usuat?
entry-level grade in the ASI occupation. The TRAD HIRES sample consists of
higher grade leveis because it was designed to resemble “typical” TRAD HIRES.
Also, all ASC HIRES began in the specialty area of Genera? Aviation or
“Other,” whereas 55.7% of TRAD HIRES began in the Air Carrier speciality; an
additional 22.2% of the sample initially were in General Aviation. Two ASC
HIRES since converted to the Air Carrier specialty area.

The remaining occupations. The databases for the remaining occupations ATCS
[Flight Service Station (FSS)] and Computer Specialist (CS), contained
relatively little information. The primary reason for this was the low
number of Airway Science candidates that had been hired into these
occupations. With only one ATCS (FSS) and one CS ASC HIRE member, no
meaningful information could be provided. Although 144 TRAD HIRES had agreed
to participate in the project evaluation from the ATCS (FSS) occupation,
there were no TRAD HIRE CSs. Briefly, the ATCS (FSS) qualified on the basis
of a non-Aviation major and relevant work experience during or following his
education. The CS ASC HIRE was an Aviation Management graduate.

Bata Sources. Table 3 describes the questionnaires and supporting catabases
used in this project. With the exception of the Biographical Questionnaire,
all other instruments were developed specifically to support the evaluation
of the Airway Science Curricuium Demonstration Project. Readers are referred
to Appencia D of the complete summative evaluation report (Clough, 1987) for
more detailed information on the development and psychometric characteristics
of the questionnaire subscales.

Procedure

Participation in this project was obtained by the signing of consent forms by
willing new hires in the occupations of ATCS, ET, ASI, and CS. All ATCS new
hires received the participation consent form during ATCS Indoctrination;
Civil Aeromedical Institute (CAMI) staff described the project and requested
participation. Regional human resources management offices were responsible
for obtaining the consent of all ASC HIRES and for requesting the
participation of mon-ATCS TRAD HIRES. Once a participation consent form was
received by the Human Resources Research Branch of CAMI, the individual was
entered into the Airway Science databases, career tracking began, and
questionnaires were distributed as needed.

The current version of the Biographical Questionnaire (8Q) as well as the
Supplement to the BQ have been distributed since July 1985. Mailings of the
other questionnaires were initiated in November 1986. At that time, an
attempt was made to "catch up" and therefore all questionnaires that should
have been mailed to the participant and his or her supervisor on or prior to
November were sent. The preferred mailing date is trigesred by the
entry-on-duty date which corresponds to the individval's Airway Science
project participation. Follow-up mailings to non-respondents and mailings

8


oot
OL Oo
i
ray
“
oO
¥
o's2 ix4 On
4
z
t
o4 “ oF
8
ee
i)
ut
0°03 = 08 OF
€
oe g oe
bd u %
TOOL

eg

+00Mr- O

toen

@UON

“SUOTQDETES EDUeTOS ADMJTY 91 40¥ 6uTsstw evem SUOTOOTSTSSOTD

os

oO 9
l

6

L

°

L
o'eL eL
L

L

L

oe €
@

8

L

4
ost SL
i)

o t0)
s u
uot zOONDy
BurMotqtods
/Sutung

St

tone e oocoo nnonno

°
oo

* ou

uoTyoonpy
j3O NOTT ut
{0} sotad

@0UETVOOXZ YOM WUDABTEY

(2861 ‘zt Aow go so)
(aynos ugF/ToUTWIAL) SAISTTOTIedS TOUWUOD OFJJOUL ATV JO UOTIDOTSFSSOTO AXTTTATBT13 eouetos Aomuty

TozOL

@UON
6utsstw
4OUI0
quewe6ou0W

SOFUCIZOOT A
e0UeTSSg 4eqyndwop
vofow UOT ZOTAY-UON

4eurI0
ABoTouyse,
suotzosedg,
Jofow uorqotay

4aUuI0d
queweBouow
AGotouyse;
suotzouedg

eyONPOUD UOTIOTAY

qwiy eoverog Aomuty
@ONPOIg eDUETOS ADMUTY

UOT JOoONPA yuvAeTeYy

“L @TqQoL



{ASC: 87.2%; TRAD: 92.9%). The average age of ASC HIRES (M = 25.88) and
TRAD HIRES (M = 25.83) did not differ. As has been reported in the past
(Clough, 1986a; 1986c), ASC HIRES were much more likely than TRAD HIRES to be
certificated as a ground instructor, flight instructor, and pilot (student,
private, commercial, airline transport). Six percent of the ASC HIRES and
21.6% of the TRAD HIRES had worked for the FAA prior to their appointments
which coincided with their participation in the Airway Science project. The
vast majority (79.5%) of ASC HIRES had completed a Bachelor's degree, whereas
oniy 29.7% of TRAD HIRES had attained that educational level}. Of the ATCS
hires who had attended college, significantly more ASC than TRAD HIRES had
earned credits in such subject areas as management, computer science, and
aviation.

Respondents were asked to classify their Associate and Bachelor's degree
aviation coursework into discrete categories. For example, aviation
coursework taken toward an Associate and/or Technical/Military/
VYocationai-Technical degree was described as: {a) vocationally oriented
(i.e., training directed toward a specific occupation and/or FAA
certificate(s) rather than toward further education in a Bachelor's degree
program); (b) baccalaureate-transfer oriented (i.e., training d’. ected toward
further education in a Bachelor's deoree program rather ‘**en toward a
specific occupation or FAA certificate); or, (c) other.

Aviation coursework taken toward a Bachelor's degree was placed into one of
the following categories: (a) aviation operations (i.e., education focusing

ser of aircraft on the ground or in the air such as Flight
> A Traffic Controi, or Professional Pilot}; (b) aviation
ogy (i.e., education focusing on ground support functions such as
nics and Aviation Maintenance); (c) aviation management (i.e., education
- G on the management of personnel and/or operations or systems such as
A n Administration/Management, Air Transportation Management, or Airline
gement).

focus of the aviation coursework taken by the two samples differed with
ortionally more ASC HIRES concentrating in aviation management; the vast
majority of TRAD HIRES had completed coursework focusing on aviation
operations.

were Classified according to their eligibility for the Airway
register. As Table 1 illustrates, 60.0% of AoC HIRES qualified, at
n part, on the basis of an Aviation Bachelor's degree. In addition,
sires graduated from FAA recognized Airway Science programs (two of the
2d not yet entered on duty). Three Aviation graduates had prior FAA
2 as Air Traffic Assistants (GS-2154} as did one of the non-Aviation
. In additien, one Aviation graduate had worked with the FAA in a non-
4TC-related position. Six Aviation graduates and three non-Aviation majors
entered the FAA Academy more than once; it is not known whether these
enroliments occurred pricr to or following their Airway Science appointments.

Technicians FET]. As of May 12, 1987, 48 ASC and 22 TRAD HIRES
this stugy. (Our records indicated 50 selections, with 48 having
ported for duty.} Most ASC HIRES qualified for the Airway Science register
the basis of a combination of work experience and college education (see
ble 2). All ASC HIRES were men as were 90.9% of the TRAD HIRES; also, the
jority of both samples were nonminority (ASC: 88.6%; TRAD: 72.7%). The

Bu a O]M
A a3 oo

4



simulation problems as measured by both the average technica? and instructor
assessments (F (1,1983) = 13.92, p < .0005; F (1,1983) = 15.60, p < .0001).
In addition, the average score of ASC HIPES on the non-radar Laboratory
Controller Skills Test was almost nine points iower than that of TRAD HIRES
(F (1,1981) = 26.33, p < .0001).

Two predictors of non-radar screen performance which have been established
from years of research at the Civil Aeromedical Institute are age at the time
of training and performance on the Office of Personnel? Management Air Traffic
Controller Seiecticn Test (Collins, Boone, VanDeventer, 1981; P. Kegg,
personal communication, June 1987). Generally, younger people are more
likely to succeed in training, as are individuals who score higher on the
Selection Test. [It was found that the average ages of ASC and TRAD HIRES at
the time that they entered the non-radar screen did not differ.]

The typical selection approach is based solely on performance on the ATC
Selection Test. In a sense, the Airway Science hiring program is being
compared to the Selection Test with regard to its predictive power. Since
only 10% of the ASC HIRE rating is based on the Selection Test, it is not
surprising that ASC HIRES scored significantly lower than TRAD HIRES on the
test (F (1, 2021) = 61.09, p < .0001). In an attempt to determine whether
the lower test performance could be a factor in the lower non-radar screen
performance, an hierarchical regression was completed in which any variation
in the screen final score which was explained by performance on the ATC
Selection Test was statistically removed. The differences between ASC and
TRAD HIRE non-radar screen performance disappeared (see Table 5). This
finding should not be interpreted as a recommendation to abolish the Airway
Scierce ATCS selection standards. However, this aspect of the hiring program
merits close examination as the number of ASC HIRES, particularly the grossiy
underrepresented Airway Science graduates, increases.

Performance in the non-radar screen is just one aspect of an ATCS's training.
It was, and will continue to be, closely monitored since it operates as the
initial screening mechanism for entrance into the field of air traffic
control in the terminal and en route environments. However, field training
also is an important measure of the technical competence of ATCS jiacumbents.
The capacity of ASC HIRES as compared with TRAD HIRES to perform in the field
will be investigated in the future.

Table 5. Hierarchical Regression of Selection Criteria on
Official Non-Radar Lab Composite Score

Step Selection Measure R2 8 t
1. Transmuted Composite ATC Selection Test +059 2243 10.68 ****
2. Hiring Status (Airway Science versus 059 Oil 0.51

Traditional Hire)

eeet op < 001

17



average ege of ASC HIRES (M = 33.98) and TRAD HIRES (M = 39.13) did not
differ significantly. In addition, over half of both ASC and TRAD HIRES
received veteran's preference (ASC: 65.2%; TRAD: 75.0%). Starting grades
of TRAD HIRES ranged from GS-3 to GS-12. The majority of ASC HIRES started
in the specialty option of Technical Management or NAVAIDS/Communications
whereas TRAD HIRES tended to be more evenly distributed across the
specialties. Some change in specialties has been observed in both groups.
Relatively few of either sample had earned a Bachelor's degree (ASC: 27.6%;
TRAD: 7.7%), although an additional 31.0% of ASC HIRES (and 38.5% of TRAD
HIRES) had completed an Associate degree.

Aviation Safety Inspectors - Operations fASI-OP]. As of May 12, 1987, data
were available on 19 ASC HIRES and 82 TRAC HIRES. One ASC HIRE had an Airway

Science degree, 42.1% had completed an Aviation Bachelor's degree and all had
prior work experience (see Table 2). Four of the 19 ASC HIRES (21.1%) were
women, whereas only three of the 82 TRAD HIRES (3.7%) were women. Also,
three (15.8%) ASC HIRES were minority (Hispanic); only 5% of the TRAD HIRES
sample were minority (two were American Indian, *.: Black, and one Hispanic).
The average age of ASC HIRES (M = 33.42) was lower than that of TRAD HIRES (M
= 42.86). TRAD HIRES were much more likely than ASC HIRES to have received
veteran's preference (ASC: 38.9%; TRAD: 81.8%). Although the entire ASC
HIRES sample began in the specialty option of General Aviation, at Teast 29%
of the TRAD HIRES entered within the specialty of Air Carrier. Two ASC HIRES
have since converted to the Air Carrier specialty. Also, all ASC HIRES began
as GS-7s; the majority of TRAD HIRES entered as GS-1is or GS-12s due
presumably to their extensive flight and other relevant experience. The TRAD
HIRES sample was designed to resemble (and therefore generalize to) the
“typical” person hired on the basis of traditional hiring criteria; entrance
at the GS-7 level is very rare in this occupation.

Biographical information was available for only eight ASC HIRES and 49 TRAD
HIRES. Of those responding, six ASC HIRES (75.0%) and 22 TRAD HIRES (44.9%)
had earned a Bachelor's degree. An additional ten (20.4%) TRAD HIRES had
completed a Master's degree, and one person (2.0%) reported receiving a
Doctorate or professional degree. No ASC HIRES had earned advanced degrees.
The aviation coursework taken by the two samples differed consistently;
proportionally more ASC HIRES took Associate degree coursework that was
Bachelor's degree oriented and Bachelor's degree coursework concentrating in
aviation operations.

Aviation Safety Inspectors - Airworthiness [ASI-AW]. Ten ASC HIRES and 115
TRAD HIRES were participating in this study as of May 12, 1987. All ASC
HIRES except one qualified, at least in part, on the basis of prior work
experience (see Table 2). Two of the 10 ASC HIRES (20%) were women, whereas
only three of 115 TRAD HIRES (2.6%) were women. Similarly, five of the eight
ASC HIRES (62.5%) for whom minority status information was available were
nonminority; in contrast, 95.5% TRAD HIRES were nonminority. The average age
of ASC HIRES (M = 33.80) was considerably lower than that of TRAD HIRES (M =
43.23). Most TRAD HIRES received veteran's preference (92.0%); however, only
30.0% of the ASC HIRES were veterans. Biographical information was available
on six ASC HIRES and 78 TRAD HIRES. Of those responding, two ASC HIRES
{33.3%} had received a Bachelor's degree, one (16.7%) an Associate degree,
and the remaining three (50.0%) no degree. Of the TRAD HIRES, 61.5% had
earned no college degree, 21.8% an Associate, 15.4% a Bachelor's degree, and
1.3% an advanced degree. The types of aviation coursework did not differ

7



DISCUSSION

As stated at the outset, the purpose of this report was to describe the
status of the Airway Science program reiative to its eight objectives shortly
before the end of its fourth year of implementation. This report also served
as the basis for the agency's request to the Office of Personnel Management
for an extension to the 5 year demonstration project. Issues addressed
include global intentions of the pregram such as the desire to enhance the
ties with coilegiate aviation education as well as areas of concern specific
to the FAA such as recruiting and selecting individuals with managerial
potential and a readiness to accept technological change. At this stage oF
the program, few consistent differences were observed between Airway Science
and traditional nires.

Within this evaluation report, two aspects of the Airway Science project were
noteworthy. First, interest on the part cof the academic community was
evident and appeared to be strong. Although the majerity of institutions
with recognized Airway Science programs received FAA recognition in the first
year of the project, submission rates for curriculum proposals appeared to be
up markedly in 1987. In addition, it has been suggested that the type of
academic institution pursuing participation in the Airway Science program may
be changing to some extent. Within the past several years, proportionally
more of the colleges and universities submitting Airway Science curriculum
proposais were relatively new co aviation education. Thus, the Airway
Science program may be promoting the growth of collegiate aviation education.

The second finding that should be mentioned is the relatively lower pass
rates of Airway Science hires as compared with traditional hires in the FAA
Academy's air traffic control (terminal/en route) screen program.
Preliminary evidence suggests that the lower pass rate may be linked to lower
scores on the OPM ATC Selection Test. Since test performance accounts for
only 10% of a candidate's Airway Science rating, a higher weigh*ing of the
Selection Test may be warranted. As @ greater number of Airway Science hires
complete the program, it shculd be possible to determine the relative impacts
of college education and work experience that may relate to success or
failure in the screen.

The original premise of the Airway Science program was that a broader-based
background may become increasingly important as an individual rises into
supervisory and managerial ranks. The evidence used to evaluate the program
to this point has largely focused on technical competence and new employee
perceptions and expectations about work. These short-range measures were all
the information that was available to date; however, they do not adequately
assess the longer term aspects of the Airway Science project. For example,
to determine project participants' ability to accept and adapt to
technological change, their performance and perceptions will need to be
examined as such changes occur within their workplace. Also, as previously
mentioned, it was thought that the knowledge, skilis, and abilities obtained
through completion of an Airway Science degree would be valuable as
individuals attempted to bid on and perform in supervisory positions. Hiring
in support of the Airway Science program began in 1984; it will be several
years before any movement into supervisory positions can be anticipated.

Finally, it must be recognized that oniy five Airway Science graduates have
been selected inta the FAA (others qualified for the Airway Science register

23



pee Lo

aN

a

ah

- te

te *


being initiated following November are being sent on this “Airway Science
date." Questionnaires are mailed to appropriate respondents and supervisors
on a monthly basis.

RESULTS

Only statistically significant differences were documented in this paper.
When a sample size was 1000 or more, a “p value” of at least .01 was required
to describe a finding as statistically significant. With smaller sample
sizes, the cutoff "p value" was .05. The statistical pc~er was sufficient
with relatively large samples to warrant a more stringent -equirement. With
a “p value" of .05 and over 1600 members in the sampte, many findings were
statistically significant that were of questionnable practical significance.

Project Objective 1: To Assess the Unique Impact of an

FAA Recognized Airway Science Degree

This objective focuses exclusively on the perceptions and behaviors of Airway
Science graduates as compared with those who do not have an Airway Science
degree. Unfortunately, in May 1987, no reliable mechanism was in place to
identify Airway Science graduates who entered the agency as a result of the
traditional (rather than Airway Science) selection approach. When sufficient
data are available, two specific areas will be investigated in support of
this objective. (At this time, the number of Airway Science graduates is too
few to compute meaningful statistics.) First, it was hypothesized that Airway
Science graduates might be more committed tc a career in aviation. Second,
and related, it was hypothesized that less attrition might be evident with
Airway Science graduates than with others.

Two types of attrition are possible and both will be examined. Specifically,
an individual may change occupations but remain in the FAA; this is referred
to as occupational attrition. The second type is agency attrition which is
exit from the FAA. One hypothesis is that all attrition may be lower with
Airway Science graduates. On the other hand, if occupational attrition is
necessary (@.g., due to a fa‘lure in training), it may not result in agency
attrition because the individuals may be more committed to remaining with the
FAA and may, due to their varied backgrounds, be qualified for multiple
positions.

In addition to the two areas mentioned above, this objective also provides
the basis for a focused analysis of Airway Science graduates within the
larger experimental groups of ASC HIRES for project Objectives 3 through 8.
It is important to recognize the significance of this objective to th:
jong-term project evaluation. It is expected that the number of Airway
Science graduates selected into the FAA occupations of interest will be
sufficiently large to permit an examination of this objective in the future.

Project Objective 2: To Enhance FAA Ties with

Universities/Colleges Offering Aviation-Related Degrees

The data relevant to this discussion were compiled in April i987 by the UAA
from its Airway Science Curriculum database. At that time, there were 29
colleges and universities across the country with recognized Airway Science

10


*SU0}S5jwgns
1kS0dosd wnina}4sN2 |2Ua}2g AeMIIy vo vaxey (Sa7eP PUL)
SUOjIIE {1B SOPA[ IV, aSeqRzepP F449 “UOLZRPIOSSy UO\RyAY AZ; SsaaquN,

OY} AG paupeqUsew pur paUb;saq saseqezRg wNINd}JIND eDUa,IG KeMEY “¢

*arueUIOsJad yO Seunseaw jeqol6
$B (LAM SP UaasIG AJOPRIOGe, IePes-YOL ayy pur I] @8PYg Wouy
$9309$ As03Rs0gR{ pur 3882 L1R 380) UOLZIG1GS Div WdO 243 UO Sas095

SBPM;OU] saseqeyeG UO;I29LBS (3/1) SOLW O9NZIQSUT LRIyPawWOIPY LhALD +z

*UOLIEAIIE pur *(SzuaWeLe BAOGY 949 Uy SebUEY?) BuyAIEIa

Jansed *(6uj3es S2ueMsO, sad JeNUUe *1aAd! AJOsyAsaaNs ‘UO; ber

“AIyLy2@s ‘dKy yuounujodde ‘apRs6 '441 1290S ‘uO}yedNI20) uO44)S0d

6u,9-098 ‘aouasesasd $,uRsaz9A ‘snzRIs AZ) JOU;W ‘KES SapNyruL

OSRGRIED $142 WOIs ¥zRQ “WaISAS jaUUOSVAd PaZWOINE S.v¥s Oy} SP
SBAsIS (SIWdd) WaISAS UO JBUO,UT JUAWOHRURW |eUUOSIeg pazeP;{OSUOD *}

bASVUVIWO ONL LbOse

*yoafoud aauayog Kemayy aya AQ pesaaod got e uy E1498 Aue NG

$9149S [RUO}FRINIIO (CU;H4sO J; 9YY UL 495U0, OU ave GYM Syued}I}QIed

40 SJOS;AJAdNS BY} OF GUaS $) OSE KODUT Siu, "4Lasas AQ

quas S$) 9) ‘SOLV 4I4M *320day StasBO4g Bujuyes) ay} 03 papuadde

$4 3) “SISW PUR S13 424M ‘BADGE paud;juaW sy *A; juaADAsSQn:

seak yore pue azep Aynp-uo-Asqua Buymol,oy seadk auo A, ojewyxoudde
S40$,AsVGNS ,$3URGy2}9ILd 07 PaINgysys4p S| JuaWNszsUy SyyL FasNparoug

“paysanbas ase sosjAsadns v oq ARP duo UBD LeNPYA\HUy Bu9

Ru} POOYILAAL, 94} Bu;pseBas SUu0}32;PHsd ‘UO; Q;OPe Ul *juawebeuRW

BIINOSSI VEWNY BAL }IG,F% YIFM PUB, 1 e Ap asol> Si. h 4S BU FIQLUKD
$]URd} 247480 YDPaSSs YO}UM 0} 1UAIKS OY) aINSPaW OL :9S00INg

14 9UaI0g (e)rabeueK 40 KapuL

"Al seek payatcwor $) quownsysuy $149
*ayep Aynp-vO-Asqua $,9URG| 249257 BY JOIVR WeaA BuO Kp azeW,XOUddE
GujuLybag “ssosparadns ,syuRd)24I4ed AQ Pays; duo> ase YO;Um

SIS¥ PUP S13 4Oy Padolarep UVEG BARY S3vOGay YuasayyIg seuNpadoid
+m0( 0 PAQs1ISaP 10j3UaIOg [Bj 4aBeUPH
40 X@PUT |Ya $1 B4;eUUO SAND Fj Y7 JO PUs ay 7e PezRI0dsO5u;
OSiY "SONOIB UOS}sLGWOD PUL [PIUGW,IIdKD BY ZO SHesHosd Buju; esa
@Y] GsedWOD 03 OLIAI}II BIR}IdOIdde @pyAosd 03 NUR sOWDedGSU]
AyaseS UO TePAY PUR UP}I}UYIa, $94U0IZIALJ JO Suo}ZedNII0 ayy

uy $3ued}I}qued YuRasss yO $Sas60Id Buyuyesy ay} YUauNIOP Of :asoding

$340day s$auboug Guyupesy

*Y¥4 Ud UIFA FL19S 612d) 24 94ed YouRasas
1B 0} VERA e BIUO Pa,;ew Sp JuauMszsUL F4GI "BIRD Aynp-uo-haque
$,juedja;qved Oyz vazse seat aud Apazew) xossde Buyuuy6ag sesnpaco'g

*pasnspaw ase s19S91 Ws OUI pur ‘ws

ay] UEUIEM a[Os 'S4aysOMOD ‘yOSyAsadNs ‘Gof 4ay 40 S44 P4eMO]

apny433® PUR jo SU0}3dadsad $,quapUcdsas aur ‘osiy [*asPaEzeP

[Suuossad pazewoqne S,v¥y a4} UF PazOU SU0} IIE Buyuyesy 40

[@uUossad ayesoqo/s09 93 pauBysap $1 SUL] ‘sesodurd Buyy2047
30j S@BURYD Jaa JO/PUR {RUO}IRINPS Auv azRI~PUy O4 wsOdINg

Kanang $0yeyy

"palo

aUy US YRWOW YZsINOJ 4}ayI 02 104 4d YUaS BO ISM J] “POUSTIAEISE

uaaq Sey r2afOsd BY2 UE UO}Id;I;I AED yey) BUG (Sd,y-UOU)
$qURG,>;74Rd YOuRasas (1k 07 PaL;eW Sy KaAsNs SpYL ZauNpaI04g

*{aos (euo,IeZjUebI0

Panyarsad *suojzepIadKs saasR> yys 89;0U2 | eUOLzedHII0

40g Suoseas **a' 4) UO PRMIOJUy 1 RUFPNILAIE pue [Se2u9;sadxa

AIGM YRADIAs ~ BdUBLIG AVMIpY “SU; 7RI/S9ILI ZIIIBD * (Oy IN~D

UOEZELAT GBuppALIV4) PeI2{|dWOI yIOMASANGD |B4/{OI 4O MapAIBAO

ue ‘papuarae sayiisueajun/sabaryoo *()soujw *($)so(ew *(s)aasbap
[euopeINpa ‘*9" 4) uo, zeuOzu; [€21UdeIHO}G SPyAdsG 0 sasCdsNY

BupwUUOLISANO aIUayIS AOMIGY [ELIE

*O@ au2 buyno,10, 41 a3e) Pau
S_Rjuawdo|aaap Soiv AQ payadwor Si yuawnszsur Sid) FasNpsroig
+ $aoua,s00Ka
WIOM QURAW!as = BDUd;Ig APAIFY PUE [PapUaI IP S947; SsdAjUT
ysaba, (oo *(s)soujw ‘(s)4ofew **aty] puncsbyIeq uo} yRINpD
S.2 BpuoGsas BY} UO 104 IPULOZU, PALVeIBE BOW APyAGsd 0, sasOdiNg

quauaidons 08

+ (buyuiPsy

Awapedy Ws 03 30420) A949 PWOYR LAD UE ADVM Yssiy 41aqT buy snp
S,CIUMUOLaAMP SdLv L107 U|A,H Sp JspeuUo; ISN Syqy  tasnparcag

[aos (puos iezjuebs0 parjarsad *$u0yyei290xa

saoues yg '89\049 |PUOIIdNII0 JO, SuOSees **a"y) UOVIMWO, Uy

[PUjPAZ,99e pue (5b, 704/$990I4 449499 '{Uo,JeINPs UO, ILIAE

Buypnyau;) Paya dwO2 %sOMBSINOD 69, (03 4O MALAIAAC UF *(s)aasbap
[eu9; PIN ‘ay } VOLJeWIOZU) {PI)NAes1H0; aPyAOIG Oy sasodung

(09) v2pPuve) sang ;ed14desbo4g

Sdu) v1 15 a0

UOLyBN] RAZ 3DeFOsg |IUa;Ig ARMIY 947 IsOddNS 03 PAs SOSRGLIRO [RUOLIFPPY PUR Sas;euUO}ISANO “E a14eL



SBOUTUZIOMUTY - 4029edSUT

Ajej0S UOTIOFAY = AY-ISV tsuotqouedg - suozoedsuy Azejz0g UOTIDTAY = dO-IS¥ fupToyUYyIe, SOTUOUZIETZ = 19 *@ION
“uoTzOeTeS 13 GUO Joy 6utsstw SOM UOTIOOTJTSSOTD

oL 6t 64 t 0 L 4 ue 6. , ¢ 8 62 YOIOL
z ‘ a z t ray ouoN
0 © ) 0 0 o (i) (i) 1 6utsstw
0 1) i) 2 € Z L € t yeu
O 0 ° o L 0 L 0 0 queweGou0w
° 0 t 0 ) iL 0 O 9 soTu0Uz0OTR
oO 0 0 oO L 0 oO 0 L eoUEeTOS seqndwod
9 8 ez oO 0 L z Ss eL z € 6 Jofow UucyIOTAY-UON
° 0 0 0 0 0 t¢) L to) 4eur0
° Oo 0 ) ti) t (0) 0 ° ABoTouyoes
0 0 0 0 0 0 0 0 L suoy osedg
0 + z 0 (i) 0 0 0 L ct) L L uofow uoptyoTAy °
0 Qo t() 0 t °o 0 0 0 4030
L 0 0 t z 0 0 t 0 quewebouow
(0) f0) 0 C1) 0 ti) L 0 2 AGoyouyoe1
0 0 0 0 @ 0 0 z 0 suoyqouedo
g 8 z L 0 0 L SG 0 L ¢ z OJONPDIDH UOYZOTAY
9 0 0 t t (0) 0 (0) () quoy sweyshs 3j049uTY
\ l i) te) oO ) L L 0 0 C 0 eyonpowg eouetos AomMutTy
av-1Isvy dO-ISv 13 av-Isv dO-IS¥ 43 My-IS¥ dO-ISV 43 AY-IS¥ dO-ISV 43 uoyqodns99
uozzoonpy uoT3DONpA vot oONpPZ yuoreTeYy
ButmoT {os go NeTT Ut
[oyoL @UON /6utung [04 sotad

eouetuedxy YOM JUDAETOY
(ce6t ‘zt Aow 30 80)
VOTIOOTJTSSOTD AYTTTATHTTZ eovetos Aomuty “Z @TQoL


not to assess group differences between ASC and TRAD HIRES. As with the
ASI-OP. occupation, most participants had completed one or more
resident/Academy courses (ASC: 6 of 7 (85.7%); TRAD: 84.0%); received OJT
(ASC: 5 of 6 (83.3%); TRAD: 90.0%) and were able to work independently
(ASC: 4 of 6 (66.6%); TRAD: 88.0%). The majority of ASC HIRES were
described as ahead of what was expected with regard to their training
progress (see Table 6). ASC HIRES were rated significantly lower than TRAD
HIRES in their ability to apply what had been learned in OJT; however, once
again, even the average rating of ASC HIRES was acceptable (M = 3.4).

Project Objective 4: Jo Develop Within the FAA Workforce an

Increased Acceptance of Technological Change

The data to respond to this objective came from two sources: the
Biographica’ Questionnaire in which the incumbents' expectations regarding
increased automation were asked and the Status Survey which was seit to
employees after being with the agency for one year or more. Due to missing
data, the expectation questions were analyzed for only ATCS ({T/E). No
statistically significant differences were observed between ASC HIRES and
TRAD HIRES with regard to t': likelihood that: (a) greater automation will
occur in the future; or {(b) negative feelings will result from increased
automation in the workplace.

The remaining questions were asked of those who had been in their positions
for one or more years; all four occupational groups were considered.
Regarding the respondents' predictions that increased automation will affect
job tasks in the future, when examining differences between ASC and TRAD
hHLRES within each occupation, ASI-OP ASC HIRES predicted less Jikelihcod of
change than ASI-OP TRAD HIRES (F (3,182) = 3.01, p < .05). All other
intra-occupational differences were non-significant. The only observed group
differences occurred with the four occupational groups and their feeling that
increased automation would influence their effectiveness. Of interest in the
project evaluation are differences between ASC and TRAD HIRES in general, and
then within each occupation if the main and interaction effects,
respectively, are statistically significant; general differences between the
occupations as a whole are not relevant to this review. No differences were
found when considering respondents’ feeling that increased automation would
affect job satisfaction or job challenge. In summary, hiring status had very
little influence on expectations, knowledge, or feelings regarding increased
automation at work.

Project Objective 5: To Recruit and Hire Individuals

with Managerial Potential

The Managerial Skills Scale (MSS) was designed to identify the extent to
which project participants demonstrate the skilis linked to effective human
resource management. Also, respondents (the participants' supervisors) were
asked to predict whether the participant has the potentia? at some point .o
be a supervisor and when that might 5e. (See Appendix D in Clough, 1987 for
a thorough description of the contents and psychometric characteristics of
the scales described in this and later sections.)

Conclusiens drawn from the MSS survey must be considered preliminary since:

20


Appendix A
Airway Science Curriculum Demonstration Project Objectives

ORGANIZATIONAL GOAL #1: To explore the possibility and to determine the
eventual consequences of restructuring the FAA career system iink with
the academic community. As presently envisioned, aviation education and j
training traditionally borne solely by the FAA will be shared by the :
individual and academic institutions.

Gbjective 1: To assess the unique impact of a tailor-made
baccalaureate curriculum (i.e., an FAA-recognized Airway
Science program) on:
a. One's interest in pursuing an aviation-related career
b. Cccupational and organizational attrition
c. Objectives 3 through &

Objective 2: To enhance FAA ties with universities/colleges offering
aviation-related degrees

ORGANIZATIONAL GOAL #2: To devetop a cadre of highly qualified individuals
well-suited to occupations integral to the future National Airspace
System
Objective 3: To improve the technical competence of the FAA workforce

Objective 4: To develop within the FAA workforce an increased acceptance
of technological change

Objective 5: To recruit individuals with managerial potential

Objective 6: To attract individuals with greater awareness of and skills
in maintaining positive human relations

ORGANIZATIONAL GOAL #3: To promote socially responsible personnel policies and
practices

Objective 7: To increase female and minority representation within the
FAA

Objective 8: To improve employee perceptions of the FAA

26 US. GOVERNMENT PRINTING OFFICE. 1968 |


Control Specialist {ATCS) (GS-2152); Aviation Safety Inspector - Operations

(ASI-OP) (GS-1825); Aviation Safety Inspector - Airworthiness (SI-AW)

are Computer Specialist (CS) (GS-334); and Electronics Technician (ET)
S-856) .

The Demonstration Project evaluation design was discussed in detaii in the
report, “An Interim Report on the Airway Science Curriculum Demonstration
Project: Clarification and Elaboration of the Project Design" (Clough,
1986b). As the report explained, two separate research questions are meshed
within the Airway Science Curriculum Demonstration Project objectives. The
original project design was based on the premise that individuals with an
Airway Science background were not being hired into the relevant FAA
occupations through traditional methods. As a result of this assumption, an
alternative selection strategy was developed which abandoned many of the
traditional selection criteria in favor of evidence of an Airway Science
background. Thus, new rating guides were developed for each occupation which
attempted to identify those knowledges, skills, abilities, and other
characteristics (KSAOs) that completion of an Airway Science degree provided,
and then to equate work experience and college education with respect to
these KSAOs.

The current study compared Airway Science hires (those hired using this
alternative selection strategy) with traditional hires (those hired on the
basis of the standard selection criteria). Specifically, the experimental
group (ASC HIRES) comprised individuals selected from an Airway Science
register into one of the five FAA occupations. The FAA Special Examining
Division in Oklahoma City determines which Airway Science applicants are
eligible for this register which dictates who may be placed in the
experimental group. ASC HIRES were further subdivided on the basis of
program eligibility inte a5 x 3 matrix which considers relevant college
education along one axis and relevant work experience atong the other. The
comparison group (TRAD HIRES) contained individuals hired by traditional
selection strategies since October 1983 who agreed to participate in the
project evaluation by signing a consent form. This study was designed to
investigate whether the Airway Science selection strategy results in: (a)
the hiring of a different type of employee; and (b) differences in
organizational measures of job satisfaction and success.

A second study was added to the demonstration project evaluation to examine
more closely the influences of college education on the demonstration project
objectives. Candidates with bachelor's degrees similar in some respects to
the Airway Science major (as well as Airway Science graduates themselves}
were eligible for the experimental group in this study. The method by which
they were selected (Airway Science or traditiona? strategy) was not a factor
in their placement into the experimental or comparison groups. The second
study and findings associated with analyzing the information in this manner
were described in detail in the original summative evaluation report (Clough,
1987). In general, few results differed from those found in comparing ASC
and TRAD HIRES. One reason for this was that several of the occupations had
few individuals who qualified for the experimental group on the basis of «
specific type of college education. Interested readers are referred to tue
complete agency report; the second study is not further discussed in this
paper.

This interim report summarizes the findings of the Airway Science Curriculum

2


NUMBER OF PROPOSAL SUBMISSIONS

Fi

g

4

10

ure Z.

ZZ

i

wm
CLURMEOARDOE

LORRI IR,

paste tees

AWS MGMT alt SYS COMP Sc?

YR OF SUBMISSION

1982
1983
1984
1985
1986
1987

aS NS
VOILE LZ ZL

AV MNI ELEC SYS

AREA OF CONCENTRATION
* 1987 FIGURES AS OF APRIL

Airway Science Proposal Submissions:

13

Areas of Concentration bv Year


Support the purchase, lease, or const tion of buildings or associated
facilities, or non-expendable instructional materials or equipment, and are
to be used in conjunction with an FAA recognized Airway Science curriculum.
During CY-84, the grant proposal solicitation was limited to Historically
Black Colleges and Universities (HDCUs}; interested HBCUs were required to
submit an Airway Science curriculum proposal to the FAA prior to submitting a
grant proposal. During CY-85, all institutions that had suomitted a
curriculum proposal prior to October 16, 1985, were eligible to compete for
grant monies. ‘the relatively high number of curriculum proposals submitted
in 1985 may Aave been due, in part, to the grant solicitation. The CY-87
competitive grant award was restricted to these institutions with recognized
Airway Science programs prior to December 31, 1986.

It is unlikely that the majority of institutions have become involved in the
Airway Science program solely because of the availability of federal funds.
As previously discussed, the curriculum review process alone is very thorough
and difficult, In addition, institutions must totally support human and
other resources to initiate, develop, and maintain their Airway Science
program. The Grant Program may have served as a catalyst to
colleges/universities with an interest in aviation, encouraging them to
institute their own Airway Science degree program. The Demonstration Project
and the Airway Science Grant Program both serve to complement the FAA's
objective in enhancing its ties with aviation higher education.

Project Objective 3: To Maintain the Technical Competence of the

FAA Workforce During a Period of Rapid Technological Change

Occupations were examined separately. At this point, technical competence
was assessec via training activities and performance. As might be expected,
the training and the manner in which performance was measured differ
substantially between occupations. It should be recognized that technical
training within the FAA is not all classroom instruction, but rather involves
extensive work (both on-the-job training and independent) in the fieid.
Although the Human Resources Research Branch maintains data on ATCS (T/E)
field training, this database has aot yet been fully integrated with the
Airway Science project evaluation database. Therefore, the focus of the
present discussion is on performance in the non-radar screening program which
is administered to new trainees at the FAA Academy upon entry on duty. ATCS
students mst pass this screen to remain in ATCS (T/E) training. For the
purposes of the Airway Science project evaluation, pass rates are determined
on the basis of the trainee's last entrance into the FAA Academy. For
example, the second pass/ fail/withdrawal outcome would be the one used to
compute the pass rate for trainees who recycle through the program. This
strategy for determining pass rates was selected because most of the interest
in the Airway Science program evatuation is focused on tracking the success
of individuals.

ATCS (1/E) Training Performance

The initial qualification training of ATCSs (T/E) was altered substantially
following the class which ended in December 1985. Prior to 1986, incumbents
were placed in the terminal or en route options before beginning the program
with a unique initial pass/fail screen for each option. With the class that
began training in October 1985 (and graduated in January 1986), a common

15


AIRWAY SCIENCE CURRICULUM DEMONSTRATION PROJECT:

SUMMARY OF INITIAL EVALUATION FINDINGS

INTRODUCTION

The Airway Science Curriculum Demonstration Project was designed io
investigate the effectiveness of an alternative approach to recruitment and
selection within five of the Federal Aviation Administration's (FAA) major
occupations. Traditionaliy, individuals with highly technical backgrounds
have beeri hired into the fields of Aviation Safety Inspector, Electronics
Technician, and Computer Specialist. In addition, prior to 1981, many of the
incoming Air Traffic Control Specialists had former military experience in
controiling air traffic. Since that time, however, proportionally fewer
entrants have been found to have air traffic control experience; their
training, once hired into the FAA, is technical.

The designers of the Airway Science program believed that the FAA's focus on
technical skills ana expertise might limit emplioyees' ability to smoothly
adjust to the substantial technological and other changes that they are
beginning to experience. Also, it was felt that the strong technical
orientation might not serve employees as well as a broader background as they
rise to supervisory and managerial positions. The Airway Science project
reflects the FAA's interest in the implications of compiementing the changing
technological and managerial] demands within the FAA with a professional
workforce whose background combines technical knowledge with exposure to
subjects ranging from aviation, the physical sciences, and computer
technology to management, the social sciences, and the humanities.

In coordination with the FAA, a baccalaureate curriculum was developed by the
University Aviation Association (UAA), a professional organization of
non-engineering collegiate aviation educators. The resulting Airway Science
degree includes core coursework in general studies, science/technotoay/
mathematics, management, computer science, and aviation. in add on, ive

areas of concentration (AOC) supplement this curriculum core: Ai y Science

Management, Aircraft Systems Management, Airway Computer Science, Aviation fe

Maintenance Management, and Airway Electronic Systems. Academic instit
interested in receiving official FAA recognition of their Airway Science
programs undergo a thorough review which includes a detailed examination of
the proposed curriculum as well as a visit to the school to determine whether
the program meets the Airway Science program spirit ard intent. The FAA has
contracted with the UAA for the services of the UAA Airway Science Curriculum
Committee to conduct this program review function and to make a4
recommendation to the FAA regarding the merit cf each Airway Sci
curriculum proposal.

Under the auspices of the Demonstration Project, the FAA was authorized to
create an Airway Science register to facilitate the hiring of individuals
with the previously described background. It should be noted that the
relevant rating guides permit comparable work experience or training to
substitute for educational requirements. Therefore, the Demonstration
Project includes individuals who qualify on the basis of education alone,
experience alone, or some education/experience combination. Airway Science
candidates are eligible for the foliowing FAA occupations: ‘ir Traffic

ed |
tion.

fonf
lity Code

‘1 and/or

“+ -vecial

1 i
|
|



by demonstrating “equivalent” college education or work experience).
Graduates of the unique Airway Science curriculum are the subgroup of Airway
Science hires in which the agency is most interested. Increasing numbers of
Airway Science graduates are expected to enter the FAA in the future.
Academic institutions first received FAA recognition for Airway Science
programs in 1983. Therefore, students who were freshmen when the programs
were first put intc place will be graduating in 1987 or 1985. Also, approved
refinements to the Ai- way Science curriculum review process should make the
Airway Science degree more salient to undergraduate students interested in an
aviation career, thereby increasing student enroilments (and the resulting
applicant pool).

The time required to establish the Airway Sc ce program, given its ambition
to link coliegiate aviation education with entry into FAA technical
occupations which themselves require several vears of additional training,
extends well beyond the initial 5 year demonstration project duration. It is
recommended that those with an interest in the Airway Science project
recognize and accept the limitations of this preliminary assestment. To
date, graduates of recognized Airway Science programs have not been examined
separately from all other "look-alike" Airway Science hires. Again, although
very few Airway Science graduates have been selected, this is expected to
change in the very near future as the first undergraduate students complete
the 4 to 5 year program. In addition, the project evaluation has focused
primarily on technical competence observed in the first year of training.
The Airway Science background was not expected to improve the technical
competence of the incumbent workforce; rather, a strong technical orientation
was rejected in favor of a relatively broad-based exposure to a variety of
subject areas which were thought to enhance other aspects of an employee's
contribution to the FAA over the years of his or her career. Although
adequate technicai competerice is required te continue within each of the
relevant occupations, the expected pay-offs of the Airway Science program are
several years in the future. It is onty at that time that an objective,
balanced appraisal of this program can be complete.

24


Technical Report Documentation Page 4

py Report No. 2. Government Accession No.

| DOT/FAA/AM-88/5 i

3. Recipient's Catalag No.

4. Title and Subtitle

|

5. Report Dare

OCTOBER 1988

12. Sponsoring Agency Name and Address
Office of Aviation Medicine
Federal Aviation Administration
800 Independence Avenue, S.W. 14
Washington, D.C. 20591

15. Supplementary Notes

This work was performed under tasks AM-C-87-PSY-110 and AM-C-88-HRR-101. ; E

\ ie atareer

i> the performance, perceptions, and characteristics of Airway Science hires
were compared with those of traditional hires. As of May 12, 1987, a total
of 197 Airway Science candidates had been selected into FAA cccupations. The
demographic characteristics of Airway Science and traditional hires tended to
differ, although these differences were more marked for Air Traffic Control
Specialists and much less so for Electronics Technicians.

| AIRWAY SCIENCE CURRICULUM DEMONSTRATION PROJECT: Performing Organ zonon Code 4
| SUMMARY OF INITIAL EVALUATION FINDINGS : ages |
i 18. Performing Organization Report No. 1
1 7) Auhors | |
i Debora L. Clough L
9. Performing Organization Nome and Address V100 Work Unser No. “TRAIS) |
FAA Civil Aeromedical Institute |
P. 0. Box 25082 PIT Contract or Grant No. |
Oklahoma City, OK 73125
f i=

sy

Findings for two of the eight project objectives were noteworthy. First,
there was considerable evidence of the academic community's interest in and
support of the Airway Science curriculum. Second, lower performance on the
Air Traffic Control Specialist Initial Qualification Screen program
(Terminal/ En route) was found for Airway Science hires. Few strong,
consistent findings were observed for the remaining Airway Science project
objectives for which data were available.

Two limitations associated with the project evaluation at this time were
described. First, greater numbers of both Airway Science hires (particularly
Airway Science graduates) and traditional hires are required for a full
examination of the project's components. in addition, more time is needed to
track the attitudes, perceptions, and performances of project participants,
most of whom are still in the early stages of training. Differences between
| the samples may be expected to emerge over time. Kecprcias! 3}

[A7~ Key Words

18. Distribution Statement

Document is available to the public
through the National Technical
Information Service, Springfield,
Virginia 22161.

~- rtSelection, Airway Science, Career
Progression ./°.. 7

DW. Security Classif. (of this pagel Wi. No. of Pages | 22, Price

Unclassified 26

19. Security Classif. (of this report)

Unclassified

Form DOT F 1700.7 8-72)

Reproduction of completed page authorized


frograms. One additional school previcusiy had an FAA recognized program.
That recognition was suspended, however, when it was discovered that the
program had not been implemented as proposed; the school was in the process
of working to reestablish its prior status. The majority of Airway Science
programs received FAA recognicion when the Nemonstration Project began in
1983 (see Figure 1). Twenty-four of the 30 institutions had Airway Science
Management degree orograms. In addition, 20 and 18 schools, respectively,
had recognized Airway Computer Science and Aircraft Systems Management
programs. There were 26 colleges and universities with recogniticn in more
than one area of concentration suggesting strong support for the Airway
Science program by participating schools. In fact, 10 institutions had
returned to the FAA to request reccegnition in areas of concentration in
addition to those originally proposed.

Perhaps a more comprehensive index of university support for the Airway
Science program is attained by examining the number of Airway Science
Curriculum proposal submissions. AS previousiy mentioned, institutions
interested in receiving FAA recognition of their Airway Science program must
successfully complete an extensive proposal review. The curricu?um proposal
review process was designed to ensure the academic integrity of the Airway
Science program. Submission of a curriculum proposal represents, from the
FAA's perspective, a valid measure of interest in the Airway Science program
since even curriculum proposal development requires an institutional
commitment of time and other resources.

Figure 2 illustrates the number of Airway Science proposals received in each
area of concentration (AOC) from 1982 through 1987. In these statistics, one
$chool applying for five AOCs counts five times {one per AOC). However, a
given curricuium proposal (regardiess of the mumber of times it may be
resubmitted) counts only once. The number of proposal submissions has varied
substantially from one year to the next with a high of 51 in 1983 and a low
of eight in 1986. During 1987 (as of April}, 19 proposals had been received
from nine schools. Considering that only 4 months had elapsed in 1987, the
proposal submission rates were up markedly (see also Figure 3). The number
of 1987 proposal submissions for four of the five AOCs was already higher
than that of the previous year; the number of i986 and 1987 submissions was
identical for the remaining AOC {see Figure 2). {[Note: Figure 3 differs
from the preceding two charts in that institutions rather than AOCs were
counted, and only initial proposal submission and recognition were recorded.]

Several hypotheses have been suggested to explain the proposal submission and
recognition ievels previously described. The majority of institutions
receiving Airway Science program recognition in the first 2 years had
longstanding aviation programs which required relatively minor modifications
to accommodate the Airway Science degree requirements (G. WwW. Kiteley,
personal communication, May 1987). More recentiy, more of the schools
submitting curricutum proposals (some of which eventually have achieved FAA
recognition) have developed their Airway Science programs without a strong
background in aviation. Their involvement in Airway Science represents an
attempt to initiate aviation education within their institution.

An FAA program which probably contributed to the fluctuations in proposal
submission rates was the Airway Science Grant Program which was designed to ,
provide selected institutions with partial funding for the enhancement of
their Airway Science program. Specifically, awarded funds can be used to

ll


arn INSTITUTIONAL PROGRAM STATUS
a
e * mamm RECOGNIZED
16 a s = = PROPOSED

NUMBER OF INSTITUTIONS

19862 1983 1984 1985 1986 1987

INITIAL YEAR OF ACTIVITY
* 1987 FIGURES AS OF APRIL

Figure 3. Institutional Activities Per Year: First Proposal Submission
Versus Initial Recognition

1a