Technical Report Documentation Page 

1. Report No. 2. Government Accession No. 3. Recipient's Catolog No. 

IXJr /FAA/M1-87 /6 

4. Title ond Subtitle 5. Report Date 

PRIVATE PILOT JUDGMENT TRAINING Mav 1987 

IN FLIGHT SCHOOL SETTINGS 6, Performing Organization Code 

8. Performing Organization Report No. 

7. Author1 s) 

Alan E. Diehl and Lewis F. Lester 

9. Performing Orgonizotion Nome ond Address 10. Work Unit No. (TRAIS) 

FAA Office of Aviation Medicine 

800 Independence Avenue, sw 11. Contract or Grant No. 

Washington, D.C. 20591 

13. Type of Report and Period Covered 

12. Sponsoring Agency Name and Address 

Office of Aviation Medicine 

Federal Aviation Administration 

800 Independence Avenue, s 0 w 0 14. Sponsoring Agency Code 

Washington, D.C. 20591 

15. Supplementary Notes 

16. Abstract 

Pilot ju:l.gment e=ors have long been recognized as an important factor in aviation 

accidents. Previous stu:l.ies have demonstrated that specialized training procedures 

can significantly reduce the number of decisional errors made by newly certified 

private pilots during in-flight tests. However, the subjects in these stu:l.ies 

were all college-age stu:l.ents enrolled in full t:ine aviation training programs 

which were taught by highly motivated instructors. The present stu:l.y examined 

the utility of revised ju:l.gment training materials with typical private pilot 

applicants in conventional flight school settings at ten fixed based operations 

within the FAA's Eastern Region. The performance of a sample of subjects who 

received ju:l.gment tra:ining was ccrnpared with that of a control group drawn from 

these same FBOs. The behavioral test of ju:l.gment was in the form of an observation 

flight administered by observers who were uninfo.nood of the details of the 

exper:inental design. ·Stu:l.ents and instructors also completed a critique of 

the program materials. The results of the stu:l.y suggest that improvements in 

pilot decisionmaking skills can be achieved in the less formal instructional 

climate which characterizes many conventional flight school programs. The revised 

ju:l.gment training program and instructional materials are acceptable to the 

user community, and roost participants found them to be very useful. 

17. Key Words 18. Distribution Statement 

Ju:l.gment Document is available to the public Decision Making through the National Technical Training Information Service, Springfield, 

. Virginia 22161 

19. Security Clossif. (of this report) 20. Security Clouif. (of this page) 21. No. of Pages 22. Price 

Unclassified Unclassified 36 

Form DOT F 1700.7 !8-721 Reproduction of completed poge authorized 
Judgment Training 

PRIVATE PILOT JUDGMENT TRAINING 

IN FLIGHT SCHOOL SETTINGS 

INTRODUCTION 

Page. I 

In 1976, the Federal Aviation Administration (FAA) contracted with 

University of Illinois researchers to investigate the degree to which 

faulty pilot judgment contributed to civil aviation accidents and to 

determine whether and how pilot judgment might be taught and evaluated. 

This study reviewed all general aviation accident data between 1970 and 

1974, and concluded that faulty pilot decisional activities were involved 

in 35% of all nonfatal general aviation accidents and in 52% of all fatal 

ace i dents (Jensen and Bene 1, 1977). The authors offered the following 

two-component definition of judgment: 

1. The abllity to search for and establish the relevance of all 

avallable information regarding a situation, to specify alternative courses of action, and to determine expected 

outcomes from each alternative. 

2. The motivation to choose and authoritatively execute a 

suitable course of action within the time frame permitted by the situation ( 1977, p 34) 

These researchers concluded that pilot judgment could be taught and 

objectively evaluated, and they proposed some ways in which this could be 

accomplished. Two years later, the FAA contracted with Embry-Riddle 

Aeronautical University, Daytona Beach, Florida (ERAU) to develop 

judgment training materials for student and instructor pilots, devise 

procedures for the objective measurement of pilot judgment, and 

demonstrate the effectiveness of such training. A revised definition of 

pilot judgment was developed by this group: 

Pilot judgment is the mental process by which the pilot recogn1zes, analyzes, and evaluates information regarding himself. the aircraft. and the outside environment. The final 

step in the process is to make a decision pertaining to the safe 

operation of the aircraft and to implement the aecision in a 

timely manner ( 1982, p. 4). 

The ERAU project (Berlin, Gruber, Holmes, Jensen, Lau, Mills and 

O'Kane, 1982) produced prototype student and instructor pi lot training 

manuals, and developed an observation flight protocol to measure pilot 

judgment. Using this technique, the ERAU team then carried out a limited 

validation study of the training materials. The subjects were all students 
Judgment Training Page 2 

enrolled in an ERAU flight program and had a mean age of 19 years. One 

group of subjects was trained under the conventional ERAU primary flight 

program, while a second group's flight and ground training incorporated the 

newly developed judgment materials. Both groups were evaluated on the 

observation flight, and the group which had received judgment training did 

significantly better than the control group on this behavioral test. 

In 1982, the FAA entered into a joint research agreement with 

Transport Canada to further examine the ERAU prototype materials and to 

independently evaluate judgment training in the Canadian pilot population. 

Several experiments have thus far been conducted under the provisions of 

that agreement. 

The subjects in the initial Canadian study were civilian Air Cadets 

participating in a summer flight training program in the Ontario Region. 

All were in their late teens and early twenties, and more than half of the 

subjects already held glider 1 icenses. This study used geographically 

isolated experimental and control groups, and the observation flight was 

administered under circumstances which permitted its true purpose to be 

better disguised than in the ERAU study. Here too, those subjects who 

received the judgment training did significantly better on the observation 

flight than did control subjects (Buch, 1982; Buch and Diehl, 1984) The 

results of both the ERAU and Air Cadet studies are shown in Figure l. 

0 Judgment 

Training 

0 Conventional 

Training 

0 10 20 30 40 50 60 70 80 90 

PERCENT 

Figure 1. Results of ERAU and Canadian Air Cadet Studies. 
' 

Judgment Training Page J 

A second Transport Canada study was undertaken in order to 

determine whether the use of the self-paced student manual could improve 

pilot decision-making skills without the involvement of flight instructors 

(Buch and Diehl, 1983). This study, conducted at Confederation College of 

Applied Arts and Technology (Thunder Bay, Ontario) used a group of 17 

university students. All were licensed private pilots with "relatively 

little flight time". Subjects were first evaluated using a well-disguised 

observation flight procedure, then given the self-paced manuals and 

instructed to complete them on their own. Following the self-study 

program, a second observation flight was scheduled. The use of the 

judgment training manuals alone resulted in a significant improvement in 

observation flight performance, from 53% on the pretest to 62% on the 

post-test. 

Another Canadian study, this one on the effectiveness and permanency 

of judgment training over time. is currently under way at a community 

college in Chicoutimi, Quebec <Buch and de Bagheera, 1985). In this study, 

subjects' aeronautical decision-making skills will be evaluated up to three 

years after their initial judgment training. 

While pi lot judgment and decision-making may be related to 

. relatively inflexible aspects of pilot personality or to the interaction of 

personal and situational variables (Lester and Bombaci, 1984), these 

experiments strongly suggest that the use of the judgment training 

curriculum can significantly improve pilot performance and enhance 

aviation safety. However, there were a number of limitations to the 

studies which compromise the external validity of these findings. Most of 

the subjects were young males who had only recently completed their 

secondary education. Both the ERAU students and the Canadian Air Cadets 

were enrolled in highly concentrated aviation training programs. Most of 

the Air Cadets already held glider licenses, and all of the subjects in the 

Confederation College study were licensed private pilots at the time the 

judgment training was begun. 

It was. therefore, felt necessary to demonstrate that the judgment 

training curriculum would be effective with the general population of ab 

initio student pilots and in the type of training environments typically 

encountered at operational flight schools. Further work was also needed 

in order to produce training materials which were acceptable to students 

and mstructors m these settings. 
Judgment Training Page 4 

• Thus, following completion of the Air Cadet study, it was decided to 

undertake a major revision of the judgment training manuals, and then to 

demonstrate the effectiveness of these revised materials in conventional 

flight schools. The revision of the student pilot and instructor manuals 

was undertaken as a joint project by FAA, Transport Canada, and the 

General Aviation Manufacturers Association (GAMA) <See Note 1 ). " 

Audio-visual materials, consisting of approximately one hundred 35 mm 

color slides and a six minute video tape, were also produced to enhance the 

training of both student pilots and instructors. 

The 70 page Student Manual introduces the concept of judgment and 

describes the role of risk assessment and decision-making in aviation. 

The student's attention is then directed to the subject areas of Pilot, 

Aircraft and Environment, and to Six Action Ways or modes of pilot error 

<Do. No Do, Early Do, Late Do, Under Do, Over Dol. The notion that errors are 

cumulative is discussed in the context of the Poor Judgment Chain, and the 

student is given a method to help break this chain. Following a discussion 

of the Three Mental Processes of Safe Flight (Automatic Reaction, Problem 

Resolving, Repeated Reviewing), a Self-Assessment Inventory introduces 

the student to Five Hazardous Thoughts (Impulsivity, Invulnerability, 

Macho, Anti-Authority, Resignation), and specific Antidotes to each 

hazardous thought are provided. The manual concludes with chapters on 

Stress Reduction and some specific applications of pilot decision-making 

skills. 

The 63 page Instructor Manual provides specific guidance on how to 

teach judgment, and outlines 18 in-flight Lesson Plans and 50 Judgment 

Training Scenarios to help accomplish the program's goals <See Note 2). It 

includes a proposed schedule of student work, suggestions on how to 

manage judgment training, and a master set of student records to be 

duplicated and used at each training site. The manual concludes with a set 

of Postcheck Exercises to assure that the student has mastered the 

judgment training material. Both of these manuals are extensively revised 

versions of the material which had been developed by the ERAU research 

team. 
' 

Judgment Training Page 5 

METHOD 

The experimental design compared the performance of two groups of 

newly rated private pilots on a behavioral test of judgment. The study 

was conducted at ten FBOs within the FAA's Eastern Region, which 

consists of the geographic area within the states of Delaware, Maryland, 

New York, Pennsylvania, west Virginia, and Virginia, and the District of 

Columbia. The participating FBOs are listed in Appendix A The control 

group included subjects trained at these FBOs using conventional pilot 

training programs. The experimental group used subjects drawn from these 

same FBOs. Their training followed the same curriculum as was used in 

the control group, but with the addition of a judgment component taught by 

specially trained instructors. The behavioral test was in the form of an 

observation flight which was administered by specially trained observers 

who were uninformed of the detai Is of the experimental design. 

Cooperating instructors at each of the ten participating FBOs were 

individually trained. At some FBOs only one or two instructors were 

included, while at others the entire cadre of CFis agreed to participate. A 

one hour slide presentation described the rationale and philosophy behind 

Judgment training, reviewed the content and methods used in the 

curriculum. and presented a number of vignettes which illustrated ways in 

which the judgment materials could be integrated into flight instruction. 

Instructors were then given the Student and Instructor Manuals to study, 

and were later interviewed in order to assure their familiarity with the 

curriculum. All instructors participating in the program were unpaid 

volunteers and were unaware that their students' judgment might later be 

evaluated. An FAA Scientist occasionally visited the FBOs to coach 

instructors and monitor the progress of the program. 

The observation flights were conducted by observers who were 

unaware of the overall experimental design and who had no knowledge of 

which type of training each subject pilot had received. The observers 

were employees of the Aircraft Owners and Pilots Association (AOPAl and 

had volunteered for the project. Each held an Airline Transport Pilot 

certificate, along with Certified Flight Instructor, instrument and 

multi-engine ratings., and each had between 5,000 and 10,000 hours of 

total flight time. During training sessions, observers were introduced to 

their roles in the study. It was stressed that their primary concern was to 

maintain flight safety at all times, while keeping an unobtrusive record of 

the subject's performance. Training underscored that the observer's role 

was to present tasks and record performance, rather than to instruct or 
Judgment Training Page 6 

critique the students. Each observer was assigned two or three airports 

from which their observation flights were to be conducted. 

During individual familiarization flights at each airport, specific 

terrain features and targets to be used as checkpoints during the 

observation flights were selected. This flight was also used as an 

opportunity for further training of the observer in the presentation and 

scoring of the observation flight scenarios. Because the terrain, weather, 

traffic conditions, and targets varied considerably among the ten airports, 

no attempt was made to establtsh a rigid flight profile which would apply 

to all observation flights. But each observer flew with subjects from both 

the experimental and control groups and thus acteg as his own blind 

control for these factors. 

The behavioral test of pilot judgment was in the form of a 12 item 

observation flight that could be safely and reliably presented at all 

evaluation sites. Items were selected to cover pilot activities normally 

occurring during the preflight, departure, cruise and arrival phases of a 

typical general aviation flight conducted under visual flight rules. Each 

item required a decision by the pi lot, and provided a measure of judgment 

rather than of knowledge, skill or experience. The 12 items and the 

criterion behavior which constituted "good judgment" for each is presented 

in brief form in Table I. A complete description of the specific 

procedures.for administering each item and the scoring guidelines used by 

the observers appears in Appendix B. 

Table 1. Observation flight items. 

l. Pilot checks weather and Notices to Airmen prior to boarding 

aircraft. 

2. During preflight inspection, pilot detects and remedies loose 

fuel tank cap, loose oil filler cap, or popped circuit breaker. 

3. Prior to engine start, pi lot detects and removes an empty 

soft drink bottre or flashlight from cockpit floor, or removes a 

hat placed on glareshield. 

4. Pilot shuts down engine or directs observer to remain in 

aircraft when observer indicates his intention to exit aircraft 

in order to check something (eg. loose cowl ingl shortly after 

engine start. 
Judgment Training Page 7 

5. Pllot requests that observer hold questions and comments 

following interruptions during preflight, taxi, or run-up 

checklist 

6. Pllot declines observer request to make a marginally safe 

intersection takeoff, non-standard turn from pattern, or low 

altitude turn immediately after takeoff. 

7. Pi lot dec 1 ines observer request to overfly a target structure 

(eg. friend's house) in a bullt-up area by barely 1,000 feet. 

8. Pi lot dec I ines observer request to execute a low steep turn 

over a target structure. 

9. Pilot declines observer request to fly at an unsuitably low 

altitude over water, mountainous, or heavily wooded terram. 

10. Pilot adjusts altitude to comply with VFR cruising 

standards following observer request to turn to new heading. 

11. Pilot avoids controlled airspace or obtains ATC clearance 

following observer request for flight into active Airport 

Traffic /\rea. 

12. Pi lot dec I ines observer request to fly a non-standard 

pattern, directly over airport at pattern altitude, high final 

approach leg, or downwind landing at uncontrolled airport 

In ordN to perrn1t a realistic assessment of Dllot judgment, It was 

necessary to disguise the true purpose of the study. Thus, the observation 

flight was described to subjects as being part of an "aviation safety 

survey" being conducted by AOPA Subjects initially received a letter 

stating that volunteers were being asked to participate in a short 

cross-country flight 1n connection with the planned revision of sectional 

aeronautical charts. The incentive for these newly licensed pilots was 

they would be given the opportunity to fly a rented aircraft from their 

home airport free, in exchange for their participation in this study. The 

I etter requested that pi lots ca II a toll-free telephone number if they 

were interested in participating. When a subject telephoned, they 

generally spoke with the observer who would conduct their observation 

flight. The observer introduced himself, explained the proposal in greater 

detail and, if the subject was still willing, made arrangements to meet at 

the pi lot's home airport 

Upon arrival at the airport, the subject was greeted by a casually 

dressed observer who explained the study of the revised sectional charts 

and the need for evaluations from newly rated private pilots. The subject 

was told that he would first complete a questionnaire on a prototype 

chart, and then fly the survey flight using the standard chart for his home 
Judgment Training Page 8 

area. It was explained that half of the observation flight would be flown 

at low altltudes, with the other half at cruise altitudes. Subjects were 

told that the major concern during the flight was how useful the currently 

used chart symbols were for a new VFR pilot. The observer explained that 

the flight would proceed with the subject being asked to locate certain 

targets. The route of the flight was then shown, and the subject was 

reminded that he was the pilot-in-command and was to make all decisions 

regarding the flight. The subject was then given the prototype Los Angeles 

sectional chart along with a 22 ltem "National Airspace Review VFR 

Prototype Chart Evaluation Form" (See Note 3). Subjects were allowed as 

much time as needed to study the prototype chart and complete the 

questionnaire. They then could proceed to flight planning activities and 

the preflight aircraft inspection. 

Observers did not volunteer detailed information about their own 

flying background. If a subject inquired, the observer told them only that 

he was a rated pilot. If pressed for more details, the observer allowed 

that he "used to be a flight instructor" but was not actively involved as 

such any longer. This satisfied virtually all subjects and helped to 

minimize the tendency to attribute authority figure status to the 

observers. Items were presented by the observers in such a manner as to 

never require them to take control of the aircraft or to divulge the true 

purpose of the flight. Each item was scored and unobtrusively recorded by 

the observer at the time it was presented according to the specific 

scoring guidelines using a dichotomous "good judgment" or "poor judgment" 

scoring system. 

Following completion of the private pilot check ride, each subject 

who had received judgment training was asked to complete a 12 item 

evaluation of the program (See Appendix Cl. Instructors completed a 

similar 16 item form (See Appendix Dl. 
Judgment Training Page 9 

RESULTS & DISCUSSION 

Subjects 

The subjects in the control group were tested by one of four 

observers. When the control group data were examined, significant 

differences between observers in the mean observation flight scores of 

their respective subjects were noted. Given these differences, an effort 

was made to have each observer fly with about the same number of 

subjects from both the experimental and control groups. However, 

because of scheduling difficulties, only three of the four observers were 

able to fly with experimental group subjects. For this reason, the 

observation flight performance of 20 subjects from the experimental 

group is compared with that of the 25 control group subjects who were 

tested by these same three observers. 

It was not possible to schedule an observation flight with any 

subjects at one of the ten FBO's. Also excluded from the analysis is data 

from one subject who appeared for the observation flight in a personally 

owned complex turbo-charged aircraft The number of subjects 

participating in experimental and control group observation flights from 

each of the ten FBO's is shown in Appendix E. The data for the control 

group appears in Appendix F, while experimental group data is contained in 

Appendix G. 

No significant difference between experimental and control group 

subjects was noted with respect to age (t=0.15, df=42). For the 24 control 

group subjects on whom demographic information was available, the mean 

age was 340 years, with a range from 20 to 64 years. The 20 

experimental group subjects ranged in age from 18 to 55 years, with a 

mean age of 344 years. 

There were also no significant differences between the experimental 

and control groups in the number of flight hours or the number of months 

spent in training for the private pilot certificate. Subjects in the 

experimental group (N=20) had a mean of 90.6 hours of flight time when 

they passed the private pilot check ride, while control group subjects 

(N= 18) had a mean of 72.4 hours (t= 1 .42, df=36). For the control group 

subjects (N= 17), an average of 9.6 months elapsed between their first solo 

and the check ride, compared with a mean of 7.7 months in the 

experimental group (N=20) (t=0.73, df=35). 

No significant differences between the experimental and control 
.Judgment Training Page 10 

groups were noted in the amount of time which elapsed between the 

private pilot check ride and the observation flight (t=0.54, df=35). On 

average, control group subjects (N= 17) were tested 5.4 months following 

the check ride. Experimental group subjects (N=20) were tested after 6.0 

months. 

There was no significant difference between the experimental and 

control groups on the scores earned when they passed the written 

examination for the private pilot certificate (t=0.36, df=42). Subjects in 

the control group (N=24l earned a mean score of 87.5, while experimental 

group subjects (N=20l had a mean score of 88.5 on this examination. Two 

subjects, one in the control group and one in the experimental group, did 

not pass the written examination on the first attemp(. One subject in the 

control group did not pass the private pilot practical test on the first 

attempt. 

Thus, any difference between the experimental and control groups in 

observation flight performance cannot reasonably be attributed to 

differences in age, flight experience, aeronautical knowledge, or skill. 

Observation Flight Performance 

The experimental group, which received Judgment training, did about 

10% better on the observation flight than did the control group. This 

difference is statistically significant (t=2.13, df=43, p<.05l, and suggests 

that the judgment training program is effective in conventional flight 

school settings. However. the improvement in pilot judgment may be 

considerably less in these settings than was suggested by previous 

studies. Results of the observation flight are presented in Table 2. 

Table 2. Observation flight results. 

Control 

Group 

59.9 

16.3 

25 

Correct Responses 

<PERCENT) 

Mean 

S.D. 

N 

Experimental Group 

70.0 

15.1 

20 
.. 

Judgment Training Page 11 

There was considerable variability in the mean observation flight 

scores assigned to subjects by each of the three observers, as well as 

great differences among subjects from different FBO's. Because of the 

small cell frequencies and the partial confounding of observers with FBO's, 

it was not possible to statistically isolate the effects of these extraneous 

variables. However, these influences were examined in two ways. When 

the observation flight performance of the experimental and control group 

subjects run by each of the three observers are compared, improvements 

of between four and eight percent are noted in the case of each observer. 

These results are shown in Table 3 . 

Table 3. Mean observation flight score by observer. 

Ob:2ervec 8 Ob:2erver B QQ::lflC':ler !: 

<N= 11 ) (N-20) (N-14) 

Contro 1 Group 47.6 61.5 73.5 

Experimental Group 52.7 65.8 81.3 

Moreover, when the overall analysis is restricted to those FBO's 

which were represented in both the experimental and control groups, the 

improvement in observation flight performance which results from 

judgment training is noted to be even greater (t-2.22, df=28, p<.05). This 

comparison is shown in Table 4. 

Table 4. Observation flight results for FBO's 

represented in both experimental and control groups. 

Control 

Group 

65.7 

15.6 

17 

Correct Responses 

(PERCENTJ 

Mean 

5. D. 

N 

Experimenta 1 

Group 

77.8 

13.6 

13 

These supplemental analyses suggest that the findi()g that judgment 

training improves observation flight performance is quite reliable in spite 

of the high variablility among FBO's and between observers. 
Judgment Training Page 12 

Judgment Training Materials 

Twenty-five pilots who received Judgment training provided critiques 

of the training program and materials. Forty-two participating 

instructors also evaluated the program. Because the Student Manual 

represented a major revision of the ones used in both the ERAU and Air 

Cadet studies. it was particularly important to establish its user 

acceptability in this manner. These student and instructor critiques were 

quite helpful in identifying the strengths and weaknesses of the various 

program components. As may be seen in Figure 2. the Student Manual was 

very we 11 received. More than half of both the student and instructor 

samples indicated that the Student Manual was "very useful", while 12% of 

the students. and none of the instructors, felt that it was "not at all 

useful". 

Instructor evaluations of their teaching materials was also favorable. 

but not auite as positive as in the case of the Student Manual. As may be 

seen in Figure 3. 43% felt that the Instructor Manual as a whole was "very 

useful", while 32% gave high ratings to the in-flight Lesson Plans and the 

Judgment Training Scenarios. However, 1 or. of the respondents felt that 

the Instructor Manual was only "slightly useful", and 14% saw little value 

in the Lesson Plans and Judgment Training Scenarios. 

fLJ Instructors (N=42) 

NOT AT ALL USEFUL ~]ili28 CJ Students (N=25) 

0 10 20 30 40 50 60 

PERCENT 

F1gure 2 Ratings of Student Manual 
• 

Judgment Training 

VERY USEFLl 

NOT AT All USEFLl 

0 10 20 

0 Lesson Plans & 

Judgment Scenarios 

Cl Manual 

30 40 50 

PERCENT (N=42) 

Figure 3. Ratings of instructional materials. 

Page 13 

60 

Questions in both the student and instructor critiques were directed 

at five basic components of the judgment training program. Figures 4 

through 8 present the proportions of each respondent group which gave 

ratings of "useful", "moderately useful", "slightly useful", and "not at all 

useful" to each of these program elements . 

E2l Instructors (N=42) 

[] Students (N=25) NOT AT All USEFUL 

0 10 20 30 40 50 60 70 

PERCENT 

Figure 4. Ratings of Five Hazardous Thoughts concept. 
Judgmertt Training 

VERY USEFUL 

(;'://////~ ////////1/// //u //// //////). 

'// 

////LL// 'l1 

MODERATELY USEFUL 

BLIGHTL Y USEFUL r:a 1 nstruct ors (N=42) 

0 Students (N=25) NOT AT All USEFUL 

0 10 20 30 40 50 60 

PERCENT 

Figure 5. Ratings of stress reduction material. 

NOT AT All USEFUL till2m 

0 10 20 

t2l Instructors (N=42l 

0 Students (N=25) 

30 

PERCENT 

40 50 

Figure 6. Ratings of Poor Judgment Chain concept. 

Page 14 

70 

60 
Judgment Training 

VERY USEFU. 

NOT AT All USEFLl 

0 10 20 30 40 

PERCENT 

0 Instructors 

(N=42) 

llilJ Students 

(N=25) 

50 

Page 15 

60 

Figure 7. Ratings of Six Action Ways concept 

VERY USEFUL ' 

SLIGHTLY USEFUL 

NOT AT ALL USEFUL 

0 10 20 30 

PERCENT 

121 Instructors (N=42) 

CJ Students (N=25) 

40 50 

Figure 8. Ratings of Three Mental Processes of Safe Flight concept 
.Judgment Training Page 16 

As may be seen, almost half (48%) of the student pilots rate the Five 

Hazardous Thoughts concept as "very useful". The concept of the Poor 

Judgment Chain is also given high ratings by the students. with 42% 

considering this concept to be "very useful". However, 16% of the student 

sample indicate that these two concepts have been of little or no value in 

their training. Forty-four percent of the students rate the chapter on 

stress reduction as "very useful", although more than one third (36r.> Place 

little value on this material. The Six Action Ways and the Three Mental 

Processes of Safe Flight are given markedly lower ratings by the students. 

Although one fifth (21%) of the sample view the Six Action Ways as "very 

useful", an eoual proportion report it to have little or no utility. And while 

one third (33%) of the students rate the Three Mental Processes of Safe 

Flight as "very useful", an even greater proportion (42%) see little worth 

in this concept 

Almost two-thirds of the instructor sample rate both the stress 

reduction chapter (64%) and the concept of the Five Hazardous Thoughts 

(65%) as "very useful". No low ratings at all are given to the stress 

reduction material, although 8% of the instructors consider the Hazardous 

Thoughts to be only "slightly useful". The Poor Judgment Chain model is 

rated as "very useful" by more than half (55%) of the instructors, with 7% 

viewing it as only "sllghtly useful". Relatively lower ratings are given to 

the concepts of Three ~1ental Processes of Safe Flight and the Six Action 

Ways The· Menta I Processes are rated "very useful" by 41 r. of the 

mstructors. with 12% noting that they are "slightly useful". And while 

over one third (36%) of the instructor sample consider the Action Ways to 

be "very useful", an almost eoual proportion (31%) see only slight utility to 

this concept 

The ratings assigned to each component by each respondent were 

scaled. and the mean ratings are shown in Fioure 9. Althouoh instructors .. "' '• 

gave consistently higher ratings to each component than did the student 

pilots. there was general agreement as to the relatively greater value of 

the stress reduction material, the Hazardous Thoughts concept, and the 

Poor Judgment Chain model. Respondents also agreed on the lesser value 

of the concepts of the Three Mental Process of Safe Flight and the Six 

Action Ways, and their utility in the judgment training program could be 

ouest i oned. 
Judgment Training 

HAZARDOUS THOJGHTS 

Page 17 

f2l Instructors 

(N=42l 

0 Students 

SIX ACTICN WAYS ~~~~~~~~~~!_-~===(N=25) 

NOT AT All 

USEFUL 

SLIGHTLY 

USEFUL 

MODERATELY 

USEFUL 

MEAN RATING 

VERY 

USEFUL 

Figure 9. Mean scaled ratings of judgment training concepts 

Training Time Requirements 

The study was also concerned with the amount of training time which 

the judgment program consumed within the private pilot curriculum. Not 

only were instructors asked to master the judgment training material on 

their own time, but they also had to allocate ground school and in-flight 

time with each student to accomplish the program's objectives. 

Most instructors (44rol reported spending a total of between two and 

four hours of self-study time preparing to teach the judgment materials, 

while almost one third (29rol spent between four and six hours in this 

activity. Thus, on average, the program appears to require that instructors 

devote about three to four hours of self study time in order to become 

reasonably familiar with the contents. 

Because the judgment materials were designed to be easily integrated 

into existing flight and ground instruction, it was felt that much of the 

judgment training would be given in conjunction with items from the 

standard pilot training curriculum. For instance, the instructor might 

caution the student about the hazardous thought "impulsivity" while 

describing the dangers of rushing a takeoff. For this reason, the 
.Judgment Training Page 18 

instructors' estimates of ground and flight times devoted to judgment 

training do not, per se., imply any additional time requirements. 

Instructors also reported that the judgment training consumed only 

one or two hours of ground time (46%) and flight time (36%) per student. 

Only a small proportion of instructors reported that the material required 

more than four hours in the primary flight training program. These results 

are summarized in Figure 10. 

Most students (60%) reported that they spent between two and six 

hours in self-study with the student manual, with the modal time 

requirement quite close to four hours. Half (50%) reported spending less 

than two hours of ground time reviewing judgment-related material with 

their instructors, and 58% said that they had devoted less than two hours 

of in-flight time to such matters. However, it is not at all clear that the 

student's perception of what was and what was not "judgment-related" 

coincided with the instructors' actual objectives in any given instance. 

These findings are illustrated in Figure 11. 

N()lE 

0 10 20 

0 Ground Time per Student 

D Flight Time per Student 

30 40 

PERCENT (N=39) 

50 

Figure 10. Instructor estimated time requirements for Judgment training. 
i 

I 

I 

' 

Judgment Training 

2-4HOURS 

LESS THAN 2 HOlRS 

0 10 20 30 

IZl Ground Time 

Cl Self-Study Time 

[ill In-Flight Time 

40 50 

PERCENT (N=25) 

Page 19 

60 

Figure 11. Student estimated time requirements for judgment training. 

Overall Program Evaluation 

Summary questions helped to evaluate the degree to which 

participants felt that the program had accomplished its goals. 

Approximately one quarter (24%) of the instructors said that "all" of their 

students had benefited from the program, and almost half (46%) indicated 

that "most" of their students had benefited from judgment training. More 

than half (56%) of the students reported that their aeronautical judgment 

had "definitely" improved as a result of the program, while one fifth (20%) 

were uncertain or felt that their judgment had not improved. About one 

third (32%) of the instructors felt that their typical student's judgment 

had "definitely" improved as a result of program participation, but 20% 

were uncertain or saw no improvement. 

An important question asked of respondents was whether they would 

recommend that the judgment training program be adopted. Four-fifths of 

the student sample (80%) recommended program adoption, and almost 

two-thirds (63%) "strongly recommended" the use of the judgment manual 

in private pilot training. Twenty-one percent of the students were 

hesitant or would not recommend the program. These results are 

presented in Figure 12. 
Judgment Training Page 20 

RECOMMEND 

RECOMMEND WITH HESITATION 

DO NOT RECOMI"END 

0 10 20 30 40 50 60 70 

PERCENT (N=24) 

Figure 12. Student recommendations regarding judgment training. 

More than three-quarters (77%) of the instructors recommended that 

the program be adopted on a voluntary basis, with almost half (48%) 

"strongly" recommending such adoption. More than half (55%) of the 

instructor sample recommended that judgment training be required, and 

about one quarter (26%) "strongly recommended" that the FAA require that 

such a program be used to teach and evaluate judgment skills in student 

pilots. Still, 24% of the instructors were hesitant or unwilling to 

recommend it on even a voluntary basis, and almost half (45%) of the 

instructors had serious reservations about mandatory judgment training. 

These results appear in Figure 13. 

Sl'll.ONGI. Y RECOMMEND 

DO NOT RECOMMEND 

0 10 20 30 

!ill] Voluntary 

~ Required 

40 

PERCENT (N=42l 

50 

Figure 13. Instructor recommendations regarding judgment training. 
.Judgment Training Page 21 

IMPLICATIONS & CONCLUSIONS 

Based upon the findings of this study, the following conclusions are 

offered: 

1) The Student and Instructor Manuals are highly acceptable to the 

user community, and most participants found them to be very useful. 

2) Of the major didactic tools used in the program, the Five 

Hazardous Thoughts concept, the Poor Judgment Chain model, and the 

stress reduction material are most useful. They should be retained in 

future versions of the judgment training program. 

3) The concepts of the Six Action Ways and the Three Mental 

Processes of Safe Flight may be too pedantic for general use, and might be 

dropped from subsequent training manuals. 

4) Participating students and instructors felt that the program 

improved pilot decision-making skills. 

5) Both students and instructors strongly recommended adoption of 

the program, although instructors were not particularly supportive of 

required judgment training. 

6) The observation flight data suggest that the program is effective 

in actually improving pilot judgment, even in the informal atmosphere 

which characterizes most FBO training programs. However, the program's 

effectiveness in these settings may be considerably less dramatic than 

was suggested by the earlier studies. 
.Judgment Training Page 22 

REFERENCES 

Berlin, J. I., Gruber, E. V., Holmes, C. W., Jensen, P. K., Lau, J. R., Mills, J. W. 

& O'Kane, J. M. ( 1 982). Pilot judgment training and evaluation 

(DOT /FAA/CT -82/56). Daytona Beach, FL: Embry-Riddle Aeronautical 

University. (NTIS No. AD-A 117 508) 

Buch, G. ( 1 982). Pilot judgment training validation exoeriment. Transport 

Canada and the Federal Aviation Administration. 

Buch, G. & Diehl, A ( 1 983). Pilot Judgment training manual validation. 

Transport Canada and the Federal Aviation Administration. 

Buch, G. & de Bagheera, I. J. ( 1 985). Judgment training effectiveness and 

permanency. Proceedings of the Third International Symoosium on 

Aviation Psychology, 337-343. 

Buch, G. & Diehl, A ( 1984). An investigation of the effectiveness of pilot 

judgment training. Human Factors, 26, 557-564 

Federal Aviation Administration, Transport Canada, & General Aviation 

1'1anufacturers Association. ( 1 983). Judgment training manual for 

instructor pi lots (Draft). Washington, D. c.: Author. 

Federal Aviation Administration, Transport Canada, & General Aviation 

Manufacturers Association. ( 1 983). Judgment training manual for 

student pi lots (Draft). Washington, D. C.: Author. 

Jensen, R. S. & Bene!, R. A ( 1 977). Judgment evaluation and instruction in 

civil pilot training (F AA-RD-78-24). Savoy, I L University of Illinois 

Aviation Research Laboratory (NTIS No. AD-A057-440). 

Lester, L. F. & Bombaci, D. H ( 1 984) The relationship between personality 

and irrational JUdgment in civil pilots Human Factors, 26, 565-572. 
Judgment Training Page 23 

NOTES 

I. Significant contributions to this study were made by Georgette Buch 

<Transport Canada), Douglas P. Harvey (FAA), Russell A Lawton (AOPA), and 

Gary S. Livack (GAMA). The observation flights were conducted by AOPA 

staff members Robert Cathers, Thomas Oneto, Glenn Rizner. and John 

Sheehan. Research assistance at Colby College was provided by Karen M. 

Barbera, Deborah E. Burke, Fran M. Gradstein, and Melissa J. Hruby. 

Secretarial support was provided by Dorothy Evertsen, . . /··-· . . . .. ·~ 

· 2. In ordett; determine the extent to which the flight trainin~ties presented material not commonly included in private pilot train1ng~~n 

informal small-sample survey was conducted at three flight schools whiCh 

were not part of the experiment but which utilize the same type o,r 

traditional pilot training curriculum as was used at the participatiryg 

FBO's. The results suggest that only 12 of the the 50 Judgment Train\llg 

Scenarios are unique to the program and present activities which

1

lare 

rarely undertaken by flight instructors in normal practice. In contr~, 21 

of the activiites presented in the Scenarios are used on occasion.by more 

than half of the flight instructors surveyed, and 17 are used by some 

instructors. Of the activities outlined in the 18 Lesson .Plans, all but one 

are commonly included in flight trai~!l)t--atmost all instructors 

surveyed. . .. -.--------

3. These evaluations were subsequently made available to the FAA 

researchers involved in evaluating the prototype sectional charts. 
Judgment Training Page 24 

APPENDIX A 

Participating Fixed Base Operators 

State Airport Fixed Base Operator 

Middletown DE Summit Summit Aviation N92 

Frederick MD Frederick Frederick Aviation FDK 

Farmingdale NY Republic Fl ightways of Long Island FRG 

lsl ip NY Is 1 ip-Mac Arthur Mid-Island Air Service ISP 

Allentown PA Queen City Queen City Aviation 1N9 

Easton PA Easton Bradens Flying Service N43 

Chesapeake VA Chesapeake t'lid-Eastern AIrways W36 

Lynchburg VA Lynchburg Air Virginia LYH 

Manassas VA Manassas Dulles Aviation WlO 

Woodbridge VA Woodbridge Woodbridge Aviation W22 
Judgment Training 

APPENDIX B 

Observation Flight 

Administration & Scoring Guidelines 

Page 25 

1. Flight Information During the mission preview, the observer indicates 

to the pilot that the flight will proceed to a specific destination airport. 

If possible, this field should be one which has a currently valid NOTAM 

and/or one with which the pilot is relatively unfamiliar. As the observer 

and pilot later walk to the aircraft, the observer will casually inquire if 

there are any NOTAMs on the destination airport"and/or if the weather will 

be all right for the flight. 

Good Judgment: pi lot indicates that he has checked an appropriate source 

for NOT AMs and/or weather 

Poor Judgment: pi lot fails to check NOT AMs and/or weather prior to 

boarding the aircraft 

2. Aircraft Condition Prior to the time that the pilot arrives at the 

airport, or while he is completing the aeronautical chart survey form, the 

observer: 

a) loosens, but does not remove, one of the fuel tank caps 

b) loosens, but does not remove, the oil fill cap/dip stick 

c) pulls a fuse or circuit breaker 

Good Judgment: the pi lot detects and remedies the defect during the 

preflight inspection prior to boarding aircraft 

Poor Judgment: the pilot fails to detect and remedy the defect prior to 

boarding aircraft 

NOTE: IF UNDETECTED BY THE PILOT, THE OBSERVER MUST REMEDY THE 

DEFECT PRIOR TO BOARDING AIRCRAFT 

3. Cabin Hazard Prior to the time that the pi lot arrives at the airport, or 

while he is completing the aeronautical chart survey form, the observer 

places a completely empty soft drink bottle on the cockpit floor in front of 

the right seat halfway between the front of the seat and the rudder pedals. 
Judgment Training Page 26 

Good Judgment: p1lot detects and safely secures the bottle during the 

preflight inspection prior to engine start 

Poor Judgment: pilot fails to detect and secure the bottle prior to engine 

start 

NOTE: IF UNDETECTED BY THE PILOT, THE OBSERVER MUST SAFELY SECURE 

THE BOTTLE WITHOUT COMMENT PRIOR TOT AXI 

4. Propeller Hazard Approximately 30 seconds after engine start, but 

before taxi or run-up, the observer releases his seat belt as he announces 

that: 

a) he has dropped his pen in front of the aircraft 

b) he sees a loose hold-down on the cowling 

c) he sees a nail on the runway in front of the aircraft 

He then begins to open the door to exit the aircraft. 

Good Judgment: pilot immediately shuts down the engine or requests that 

the observer remain in the aircraft 

Poor Judgment: pi lot permits the observer to leave the aircraft while 

engine is running 

NOTE OBSERVER MUST NOT EXIT THE AIRCRAFT WITH ENGINE RUNNING 

5. Checklist Distract ions During the execution of the pres tart, pre taxi, 

and/or run-up checklist the observer converses with, quest ions or 

otherwise distracts the pilot in such a way as to interrupt the systematic 

execution of the checklist. Two interruptions are to be made. 

Good Judgment: pilot requests that the observer hold his questions and 

comments until after the checklist is completed, or pilot ignores 

observer's remarks 

Poor Judgment: pilot interrupts the checklist to converse with the 

observer, or carries on the conversation while continuing the checklist 
Judgment Training Page 27 

6. Deoarture Procedures The observer requests one of the following 

non-standard departure procedures: 

a) a takeoff from a marginally safe intersection 

b) a right (or non-standard left) turn immediately after takeoff while 

sti II in the pattern at an uncontrolled field 

c) a low (200 Feet AGU turn immediately after takeoff 

Good Judgment: pilot declines request 

Poor Judgment: pilot executes potentially hazardous departure procedure 

NOTE: OBSERVER MUST NOT PERMIT THE PILOT TO EXECUTE A HAZARDOUS 

DEPARTURE 

7. Low Flight Over Obstruction The observer requests a flight path and 

altitude which will take the aircraft barely 1,000 feet above some object 

or bui !ding in a bui It-up area 

Good Judgment: pilot notes conflict and either climbs or diverts in order 

to pass the object at a safe distance 

Poor Judgment: pilot flies low over vertical obstruction 

NOTE OBSERVER MUST NOT PERMIT HAZARDOUSLY LOW FLIGHT OVER 

OBSTRUCTION 

8. Low Steep Turn Over Building The observer requests that the pi lot 

circle above a certain building so that the observer can take a photograph. 

The observer requests an increasingly steeper turn until a 45 degree angle 

of bank is established. 

Good Judgment: pilot climbs to a safe altitude, or does not exceed a 30 

degree angle of bank at low altitude, or declines request 

Poor Judgment: pi lot attempts to execute a hazardous low steep turn 

NOTE OBSERVER MUST NOT PERMIT A HAZARDOUSLY LOW /STEEP TURN 
.Judgment Training Page 28 

9. Low Flight Over Unsuitable Terrain The observer requests a flight 

path and altitude over unsuitable terrain such that the ability to execute a 

safe emergency landing or power-off glide to shore would be seriously 

compromised. 

a) heavily wooded terrain 

b) mountainous terrain 

c) large body of water 

Good Judgment: pilot declines request or climbs to a safe altitude 

Poor Judgment: pilot flies low over unsuitable terrain 

NOTE: OBSERVER MUST NOT PERMIT HAZARDOUSLY LOW FLIGHT OVER 

UNSUITABLE TERRAIN 

I 0. VFR Cruising Altitude The observer requests a flight path and 

altitude consistent with VFR cruising altitude regulations, and requests 

the pilot to locate a specific terrain feature along this flight path. Upon 

reaching the terrain feature. the observer requests the pi lot to fly a new 

heading such that a climb/descent is required in order to maintain an 

appropriate VFR cruising altitude. The observer makes no mention of 

altitude however. 

Good Judgment: pilot notes conflict and either climbs or descends to an 

appropr1ate VFR cruismg altitude 

Poor Judgment: pilot turns to the new heading but fails to change altitude 

to comply with VFR cruising altitude regulations 

NOTE OBSERVER MUST NOT PERMIT A HAZARDOUS VIOLATION OF VFR 

CRUISING ALTITUDE REGULATIONS 

11. Controlled Airsoace Encroachment The observer requests a flight 

path and altitude which brings the aircraft marginally close to the 

horizontal or vertical limits of an active Airport Traffic Area of Terminal 

Control Area, while the aircraft is not in radio contact with the 

controlling facility. 
Judgment Training Page 29 

Good Judgment: pi lot informs observer that the requested flight path 

cannot be flown safely without ATC clearance or pilot requests such 

clearance from ATC, or pilot alters flight path to avoid the controlled 

airspace by a safe margin 

Poor Judgment: pilot flies marginally close to an Active Airport Traffic 

Area or Terminal Control Area without ATC Clearance 

NOTE: OBSERVER MUST NOT PERMIT THE AIRCRAFT TO ENTER INTO 

CONTROLLED AIRSPACE WITHOUT ATC CLEARANCE 

12. Arrivol Procedure::~ On opprooch to on uncontrolled oirport, the 

observer, 1n order to take a pt·iOtograph, requests a flight path and altitude 

which brings the aircraft: 

a) directly over the airport at pattern altitude but not in the pattern 

b) at pattern altitude but flying a non-standard pattern 

cl set up high on the final approach leg 

d) set up for a downwind landing 

Good Judgment: pi lot informs observer of the need to fly a standard 

approach and landing pattern, and flies it 

Poor Judgment: pi lot flies a non-standard approach or landing pattern at 

pattern a 1 ti tude 

NOTE: OBSERVER MUST NOT PERMIT A HAZARDOUS APPROACH OR LANDING 
Judgment Training Page JO 

APPENDIX C 

Student Evaluation 

The Federal Aviation Administration (FAA) and the General Aviation 

Manufacturers Association asked several flight schools to examine a 

prototype Pilot Judgment Training Program. We understand that you 

participated in this important program, and we would be interested in your 

views on its acceptability. Please circle your response and feel free to 

explain it below the question, or on the back of the page If necessary. A 

self-addressed, postaqe paid envelope Is enclosed for your convenience. 

Your cooperation in this critique is. deeply appreciated. 

1. The Judgment Training Manual in general was: 

A) very useful B) moderately useful 

C) slightly useful D) not at all useful 

2. Six Action Ways concept (Do, No Do, Under Do, Over Do, Early Do, Late 

Do) was: 

A) very useful 

C) slightly useful 

B) moderately useful 

D) not at all useful 

3. Poor Judgment Chain concept was: 

A) very useful B) moderately useful 

C) slightly useful D) not at all useful 

4. Three Mental Processes of Safe Flight concept <Automatic Reaction, 

Problem Resolving, Repeated Reviewing) were: 

A) very useful B) moderately useful 

C) slightly useful D) not at all useful 

5. Five Hazardous Thoughts (Anti-authority, Impulsivity, Invulnerability, 

Macho, Resignation) were: 

A) very useful 

C) slightly useful 

B) moderately useful 

D) not at all useful 

6. Chapter on Identifying and Reducing Stress was: 

A) very useful B) moderately useful 

C) slightly useful D) not at all useful 
Judgment Training Page 31 

7. How much self-study time did you spend on the Judgment Training 

Manual? 

A) less than 2 hrs. 

Dl 6-8 hrs. 

B) 2-4 hrs. 

El more than 8 hrs. 

C) 4-6 hrs. 

8. How much total time did your instructor spend discussing judgment 

with you in-flight? 

A) 1 ess than 2 hrs. 

D) 6-8 hrs. 

B) 2-4 hrs. C) 4-6 hrs. 

El more than 8 hrs. 

9. How much total time did your instructor spend discussing judgment 

with you on the ground? 

Al less than 2 hrs. Bl 2-4 hrs. Cl 4-6 hrs. 

Dl 6-8 hrs. Elmore than 8 hrs. 

10. Do you feel that reading this manual and/or discussing these concepts 

improved your aeronautical judgment? 

A) definitely 

C) uncertain 

B) probably 

D) no 

11 Would you recommend the adoption of the Judgment Manual for use in 

private pi lot training? 

Al strongly recommend B) recommend 

Cl recommend with hesitation Dl do not recommend 

12. How could this manual be improved? 
.Judgment Training Page 32 

APPENDIX D 

Instructor Evaluation 

1. Instructor Manual in general was: 

A) very useful Bl moderately useful 

Cl slightly useful Dl not at all useful 

2. In-Flight "Lesson Plans" 

instructor manua I were: 

and "Judgment Training Scenarios" in the 

A) very useful 

Cl slightly useful 

3. Student Manual in general was: 

A) very useful 

Cl slightly useful 

Bl moderately useful 

Dl not at all useful 

Bl moderately useful 

Dl not at all useful 

4 Six Action Ways concept (Do, No Do, Under Do, Over Do, Early Do, Late 

Dol was: 

Al very useful 

C) slightly useful 

5. Poor Judgment Chain concept was 

Bl moderately useful 

Dl not at all useful 

A) very useful Bl moderately useful 

C) slightly useful Dl not at all useful 

6. Three Mental Processes of Safe Flight concept (Automatlc Reaction, 

Problem Resolving, Repeated Reviewing) were 

Al very useful Bl moderately useful 

C) slightly useful Dl not at all useful 

7. Five Hazardous Thoughts (Anti-authority, Impulsivity, Invulnerability, 

Macho, Resignation) were 

Al very useful Bl moderately useful 

C) slightly useful Dl not at all useful 

8. Stress recognition chapter was: 

Al very useful 

Cl slightly useful 

Bl moderately useful 

Dl not at all useful 
Judgment Training Page JJ 

9. How much self-study time did you spend (reading the Student and 

Instructor Manuals, planning in-flight exercises, etc.l in order to permit 

you to teach the judgment concepts? Do NOT include the time spent in the 

training sessions with FAA scientists. 

Al Jess than 2 hrs. B) 2-4 hrs. C) 4-6 hrs. 

D) 6-8 hrs. E) more than 8 hrs. 

10. About how much ground time did you spend teaching judgment to each 

of your students? 

Al None 

Dl 2-4 hrs. 

Bl less than 1 hr. 

El more than 4 hrs. 

C) 1-2 hrs. 

11. About how much flight time did you spend teaching judgment to each 

of your students? 

Al None 

Dl 2-4 hrs. 

B) less than 1 hr. 

El more than 4 hrs. 

C) 1-2 hrs. 

12. Would you recommend the voluntary use of these manuals for private 

pi lot training? 

Al strongly recommend B) recommend 

C) recommend with hesitation Dl do not recommend 

13. Do you recommend that FAA require a program such as this to teach 

and evaluate judgment skills of student pilots? 

A) strongly recommend Bl recommend 

Cl recommend with hesitation D) do not recommend 

14. Do you think participating in this program improved your "typical" 

students' judgment ski II s? 

Al definitely 

C) unsure 

Bl probably 

Dl no 

15. What proportion of your students who participated in this program do 

you feel benefited from this training? 

Al all Bl most Cl some 

Dl few El none 

16. How could this program be improved? 
' ' 

Judgment Training 

APPENDIX E 

Experimental & Control Group Observation Flights 

At Each Participating FBO 

Fixed Base Operator 

Summit Aviation 

Frederick Aviation 

Flightways of Long Island 

Mid-Island Air Service 

Queen City Aviation 

Bradens Flying Service 

Mid-Eastern Airways 

Air Virginia 

Dulles Aviation 

Woodbridge Aviation 

Airport ID 

N92 

FDK 

FRG 

ISP 

1N9 

N43 

W36 

LYH 

WlO 

W22 

TOTAL 

Experimental 

Subiects 

' 

2 

5 

3 

5 

3 

0 

0 

0 

20 

Page 34 

Control 

Subiects 

' 

0 

0 

4 

3 

4 

2 

4 

0 

4 

4 

25 
Judgment Training Page 35 

APPENDIX F 

Contro 1 Group Data 

Observation Written 

No. Observer Flight Score ~ Test Score Alroort ID 

501 c 33 W36 

502 R 58 28 92 FRG 

503 R 75 35 92 1SP 

504 c 64 39 97 W22 

505 c 42 27 92 W36 

506 c 58 27 82 W22 

507 c 42 32 100 W22 

508 c 50 36 98 W22 

509 R 50 33 85 W10 

510 R 67 23 72 W10 

511 c 50 32 100 W36 

512 c 42 26 93 W36 

513 R 58 38 92 W10 

514 R 50 28 100 W10 

515 R 67 34 70 FRG 

516 R 92 30 82 FRG 

517 R 25 21 77 FRG 

518 0 67 27 82 1N9 

519 0 75 34 93 1N9 

520 0 58 64 87 1N9 

521 0 83 42 98 1N9 

527 0 83 39 90 N43 

528 0 75 54 72 N43 

529 R 67 20 83 ISP 

530 R 67 46 72 ISP 
Judgment Training Page 36 

APPENDIX G 

Experimental Group Data 

Observation Written 

No. Observer Flight Score ~ Test Score Airport ID 

602 c 42 55 70 FDK 

603 c 58 27 93 FDK 

604 R 73 34 87 N92 

605 R 75 18 95 W10 

606 R 67 44 90 N92 

607 c 58 32 95 FDK 

608 R 58 35 97 FDK 

610 R 58 46 90 FDK 

611 R 58 50 85 ISP 

612 R 45 21 82 FRG 

613 R 75 28 88 FRG 

614 R 83 20 97 ISP 

615 0 75 41 87 1N9 

616 0 83 53 97 N43 

617 0 83 26 92 N43 

618 0 75 44 80 N43 

619 0 92 21 73 ISP 

620 0 92 30 92 ISP 

621 0 92 20 88 ISP 

622 0 58 44 92 FRG