Doc 

### FAA AM 

### 97 11

### 1'hisdoeu1J1~1'1t**i~arailail).~ tO:~** **lY~~········.** 

## through**th~ Natto1'1aiT~l~** **Servi.~.** 

## Sj>rii'ISfi;eld,**Vi~gini;t '1,~~1>'**  

# '

.·'• ··

. '"

. - -- ,_- - ----

-**~-** ,(_ I• 1

I

# I

!Technical Report Documentation Page

1. Report No. 2. GovernmentAccession No. 3. Recipient's Catalog No. DOT/FAA/AM-97/11

4. 1111e and Subtitle 5. Report Date Transfer ofT raining Effectiveness ofPersonal May 1997 Computer-BasedAviation Training Devices 6. Performing Organization Code

7. Author(s) 8. Pertorming Organization Report No. Taylor, H.L., Lintern, G., Hulin, C.L., Talleur, D.,Emanuel, T., and Phillips, S.
9. Perfoonlng Organization Name andAddress 1 

### o. Work unn No. (lRAIS) lnscirute ofAviation University oflliinois at Urbana-Champaign Savoy, illinois 61874 11. COntractorGrant No.
DTFA 94-G-044
12. Sponsoring fi>Qercy name andAddress 13. Type of Report and Period Covered FAA Office ofAviation Medicine Federal Aviation Administration 800 Independence Avenue, SW. Washington, DC 20591 14. Sponsoring Agercy Code

15. Supplemental Notes Drs. Robert Blanchard and Kevin W. Williams served jointly as COTR for FAA-CAMI
16. Abstract The training effectiveness ofPersonal Computer-Based Aviation Training Devices (PCATDs) has received only limited testing. In the experiment reported here, a commercially available PCA TO was evaluated in a transfer oftraining experiment for its effectiveness in supporting instrument flight training. The data show levels ofsavings in airplane flight time that varied from 15o/o to over 40o/o for certain training exercises. However, there were also cases in which savings were essentially zero or even showed decrements as high as 25o/o. In general, transfer savings were positive and substantial when new tasks were introduced. The data indicate that a PCA TO can provide training benefit for certain tasks but, in addition, use ofthe PCA TO in

some areas is not expected to result in savings and will erode the overall potential to reduce costs.

17. KeyWords 18. Distribution Statement Personal Computer-Based Aviation Training Devices, Document is available to the public Flight Training, Instrument Flight, through the National Technical Psychology, Applied Psychology Informacion Service, Springfield, Virginia 22161

19. Securtty Ciassn. (ofthisreport) 

# J
20. Securtty Ciassn. (ofthis poge) 121. No. of Pages 

## 122. Price Unclassified Unclassified 24 Form DOT F1700.7 (8-72) Reproducfton of completed page authonzed

### FOREWORD

This study tested 

## the effectiveness ofPC-based aviation training devices (PCA TDs) in instrument training for the purpose ofguiding regulation and certification considerations by the FederalAviation Administration (FAA). Heretofore, only data of a subjectivenaturehadbeenavailable. The study was supported under FAA Grant No. 94-G-044withtheUniversityoflllinois at UrbanaChampaign duringfiscal years 1995 and 1996. Ms. KarenAyers,Ms.DianaChristenson,andMs. Sharon Allen assisted with the manuscript and Mr. Greg Morrisonprovidedtechnicalassistanceduringequipment set-up; Mr. RickyWeinberg, chiefpilot, and Mr. RichardKeir, instrumentcoursesupervisor,providedinvaluableassistancewithflightoperationsand with student management. Mr. Steve Owen, Mr.

iii

### David Petersen, and Mr. Phillip Martin served as stage checkpilots for the study.Wethanktheflight instructors and the students ofAviation 130 and Aviation 140 for participatingin the study. Sponsorship ofthe study was provided by Mr. Thomas

### C.Accardi, Director, FlightStandards Service, AFS-1. Project oversight was 

### provided by Mr. Louis 

### C. Cusimano, AFS-800, and Mr. Lauren Basham,AFS-840. Humanfactors programcoordination was provided by Dr. Thomas McCloy and Mr. Ronald Simmons, Office ofthe ChiefScientist andTechnicalAdvisor for Human Factors, AAR100. Drs. Robert E. Blanchard and Kevin W. Williams servedjointlyin the role ofcontracting officer's technical representative for FAA Civil AeromedicalInstitute. 

## ,.....

# ..

TABLE OF CONTENTS

Page INTRODUCTION

.......................................................................................................................................... I

METHOD

......................................................................................................................................................... 3 Subjects

................................................................................................................................................. 3 Apparatus

.............................................................................................................................................. 3 Procedure

.............................................................................................................................................. 3 Analyses

................................................................................................................................................ 6

RESULTS

.......................................................................................................................................................... 7 Trials to Criterion

................................................................................................................................7 Time to Complete Flight Lesson

....................................................................................................... II Course Completion Time

.................................................................................................................. 14

DISCUSSION

................................................................................................................................................. 15 Aviation 130

....................................................................................................................................... 15 Aviation 140

....................................................................................................................................... 16 Substitution ofthe PCATD for Instrument Flight Hours

............................................................... 17

SUMMAR¥

..................................................................................................................................................... 18

REFERENCES

................................................................................................................................................ I8

v

## .• TRANSFER OF TRAINING EFFECTIVENESS OF PERSONAL COMPUTER-BASED AVIATION TRAINING DEVICES



## INTRODUCTION

The resources required for flight training impose a substantial burden on the aviation community. The potential to reduce costs through the use ofinexpensive buteffective trainingdevices has added an incentive for flight training departments to conduct more training with ground-based flight training devices. The cost ofcurrentlycertified, generic flight training devices is out of reach of many flight .students and flight schools, but personal computer-based aviation training devices (PCATDs) offer a low-cost alternative for instruction offlight tasks. Recently, there has been increased demandfrom developers, from industry, and from training organizations to use PCATDs in flight training. A summary of a joint industryFederal Aviation Administration (FAA) conference concernedwith the development anduse ofPCATDs documents this emphasis (Williams, 1994).An expense ofless than $10,000 will provide a PCATD including software, computer hardware, and a flight-control system. This is within reach ofmost flight schools. PCATDs have generated considerable enthusiasm within the aviationindustry(Falun, 1992;Lert, 1990; Peterson, 1993). Hampton, Moroney, Kirton, and Biers (1994) reportedthatstudentstrainedinaPCA TD performed as well on instrument procedures in the airplane as those students trained in a Frasca 141. Nevertheless, there is no clear evidence that PCA TD training will transfer effectively to an airplane across
a broad range offlight tasks. The fidelity ofPCATDs is low in areas normally thought to be important: representation ofdisplays, switches, and out-of-cockpit scenes; with control loading and flight dynamics being the most obvious. In addition, PCATDsaccept control inputs from low-fidelity devices that range from computer keyboards, single joysticks, and yoke/ pedal combinationsof varyingquality (Peterson, 1993).

I Thereis evidenceofpositive transferofPCATDsto the airplane (Phillips, Hulin, & Lamermayer, 1993; Ortiz, 1993, 1994; Dennis, 1994), although the empirical evaluationshave beenlimitedin scope. Experimental data bearing on issues of skill transfer are equivocal, and there is no means ofascertaining the trainingeffectiveness ofa PCA TDwithoutan empirical test of its impact on transfer of skills to flight performance. Specifically, itis necessary to determine the types offlight tasks that can be trained effectively in the PCATD and the level ofproficiency obtained by practice with a PCATD. Analytical evaluation of PCA TDs indicates that they provide many of the features required for standard instrument flight maneuvers. Instrument trainingin a PCATD may result in substantial transfer for instrument flight tasks. Toevaluate transferoftraining, theperformanceof

a group ofsubjects trained in a flight training device, and later trained to criterion in an aircraft, is compared with the performance of a control group of subjectswho have been trained onlyinthe airplane. A conventional measure oftransfer oftraininghas been percent transfer, which is known to be a negativelyaccelerated function. Roscoe (1971) and Roscoe and Williges (1980) correctly pointed out that percent transfer fails to consider the amountofpractice in the flight training device in determining the training effectiveness ofthe device. The transfer effectiveness ratio (TER), a negatively-decelerated function, has been used to determine the ratio ofthe trials/rime in the airplane saved by the training device group, compared with the airplane group, as a function of the numberoftrials/time in the training device (Williams & Flexman, 1949; Roscoe, 1971). Many of the applied research studies on flight training effectiveness have examined the effect of specific simulator features on specific tasks (e.g., Lintern, Roscoe, Koonce & Segal, 1990; Taylor, 

# .,..Lintern, Koonce, Kaiser, & Morrison, 1991). The first transfer of training evaluations of PCATDs (Phillips et al., 1993; Ortiz, 1993, 1994; Dennis, 1994) have followed that pattern. Onlya few experiments have examined the impact ofa single device on
a whole course (e.g., Povenmire & Roscoe, 1971, 1973). Transfer studies conducted in 1949/50, but first reported by Flexman, Roscoe, Williams, and Williges (1972), used percent transfer and transfer effectiveness ratios in terms of errors, time to critep rion, and trials to criterion to assess a comprehensive setof contactandinstrumentflighttasks. TheFlexman et al. (1972) study remains the most definitive study oftheeffectiveness offlight trainingdevices for a total flight course. It described a research plan that was implemented in the current evaluation ofPCATDs. Williams and Blanchard (1995), proposed that task analysis be used as a basis for predicting transfer effectiveness. The appropriate task analysis would identify the learning requirements for a particular flight task, and these requirements would then be organizedin terms ofcommonfeatures. An analysis of thePCATDwoulddeterminewhich learningrequirements could be supported by that device and, consequently, which flight tasks could be taught in the PCATD. They proposed that the initial validation of the guidelines would be accomplished through the collection of performance data from FAR Part 141 schools using PCA TDs. The purpose of the present study is to determine the extent to which a PCATD

can be used to develop specific instrument skills that

are taught in instrument flight training and to measure transfer of these skills to the aircraft. The approach taken in this study is to use a representative PCA TDto teach instrumenttasks in thetwo-semester instrument training course at the University of Illinois Institute ofAviation. Our training course outline for the PCATD emphasized instruction ofspecific maneuvers in orderto determine if the PCATD was an effective training device for avarietyofinstrumenttasks. Foreach flight lesson, whichwas programmed to be accomplished in

one week, one or more instrument tasks were introduced and/or reviewed. Progression to the next flight lesson, however, required that the student meet the

2 objectives in that flight lesson. Advancement to the next flight lesson, therefore, was based on demonstratedproficiencyontherespective instrumenttasks. Forthe PCA TDgroup, proficiencyhadto be demonstrated inthePCATD prior to flying the flight lesson in the airplane. The PCA TD group was required to demonstrate their proficiency in the airplane on a respective flight lesson prior to attempting the next flight lesson. TheAirplane group received all oftheir instrument training in the airplane but also had to demonstrate proficiency in the airplane on each flight lesson priorto advancingtothe next flight lesson. The effectiveness ofthe PCATD in teaching specific instrument tasks in a flight lesson was measured by comparing the trials required for the PCA TD group and the Airplane group to reach proficiency in the airplane on specific instrument tasks. Subsequently, thepercenttransferandTERfortrials were computed for each instrument task. The effectiveness of using thePCATD toreach proficiencyfor aparticularflight lesson was measured by comparing the time required for the PCA TO group and the Airplane group to complete the objectives of the flight lesson in the airplane. Subsequently, thepercenttransfer andTER for time to complete the lesson were computed. Earlyintraining, the emphasis was onbasiccontrol skills with reference to instruments. Instruction then progressedthroughincreasingly challenging exercises with emphasis on instrument holding patterns, instrumentapproaches, and preparation for cross-country flights. In each flight lesson, transfer on specific instrument tasks was evaluated; transfer relative to logged airplane time to satisfy the completion standards for that lesson was also assessed. In many lessons, the logged time to satisfy completion standards represents studentprogress onmorethanjustthetasks tested inthelesson. Anydiscrepancybetween trials to proficiencyand time to proficiencymayreflect differential progress with relation to the rated instrument tasks and those other lesson objectives. An earlier interim report 

### by Taylor, Lintern, and Hulin(1995) providedtheresultsofthefirstyear'swork. This was also reported by Taylor, Lintern, Hulin, Emanuel, Phillips, and Talleur (1995), and Taylor, Lintern, Hulin, Talleur, Phillips, and Emanuel (1996).

### METHOD

Subjects One-hundred forty-four subjects were tested in a transfer of training design. These subjects were enrolled in instrument flight instruction at the Universityofillinois.Atotal of 117subjects were enrolledin basic instrumentinstruction (Aviation 130);40in the Fall. 1994 semester; 17 in the Spring, 1995 semester; 12 in theSummer, 1995 semester; 37in the Fall, 1995 semester; and II inthe Spring, 1996 semester. Tenof the subjects enrolled inAviation 130 in theFall 1995 received training in an integrated curriculum, which used botha PCATDand a Frasca 140 trainingdevice. The results ofthis preliminaty study are reported by Taylor, Lintern, Hulin,Talleur, Emanuel,andPhillips (1996a). The remaining 27 students were enrolled in the advanced instruments course (Aviation 140) in the Fall, 1994 semester. Seventy-seven ofthe students who completed Aviation 130 completed the experiment as Aviation 140 students. The subjects were restricted tostudentsbetween 18 and30years oldwho had taken their private pilottrainingatthe University ofIllinois or had completed a familiarization course and were recommended for Aviation 130. Subjects were randomly assigned to the PCATD group and the Airplane group, with the constraint thatmale andfemale studentswere distributed evenly

over the two groups. Students in the PCATD group
were taught specific instrument tasks on the PCATD to a specified criterion and were then transferred to theaircraft, wheretheywererequiredtoperformthese tasks again to the same criterion. Comparisons were made with a Control group that had received similar training only in an airplane. Students from both groups were required to achieve criterion on those tasks in the airplane priorto moving to the next flight lesson. The Fall 1994 Aviation 140 data set will be presented separately, since these students had no previous experience with PCATDs, while the remaining Aviation 140 students in the PCATD group had received basic instrumenttraining in the PCATD. Of the 92 subjects who completed Aviation 130, 45 subjects were assigned to the Airplane group, and 47

3subjectswere assigned to the PCATD group. Ofthese 92 subjects, 74 subjects completed Aviation 140; 36 subjects were assigned the Airplane group, and 38

were assigned to the PCA TD group.
Apparatus A representative PCA TD system that would be affordablefor themajorityofFARPart 141- approved schools 

### was assembled from commercially available, state-of-the-art but proven PCATD technology. The choice ofPCATD components was established in a discussion with the Civil Aeromedical Institute's Human Factors Research group on July 22, 1994. Figure I shows the system, based on the criteria of representativeness, capability, and cost, which includedMDMFSI00software, modifiedfor Beechcraft Sundownerperformancecharacteristics,an IBM-compatible Pentium, 60MHz computer, an instructor station mapdisplay, flight controlsby Precision Flight Controls (see Figure 2), and a 20-inch monitor and hood. The 20-inch monitor permitted display of 6 standard 3.0-inch instruments. The system is described inmore detail in Appendix I ofthe University of Illinois contract report (Taylor, Lintern, Hulin, Talleur, Emanuel & Phillips, 1996b). The training aircraft used were Beechcraft Sports/Sundowners, whichhave a single-engine, fixed-pitch propeller, and fixed Ianding gear.

Procedure Instrument training at the University ofIllinois is conducted in a two-semester sequence. Basic instrumentproceduresemphasizingaircraftcontrol, instrumentdeparture, enroute andapproach procedures are taught during the first semester in Aviation 130; remaining instrument skills are taught during the second semesterinAviation 140 (Table 1). Theobjective ofthe courses is to develop the basic skills necessary for the control and accurate maneuvering ofan airplane solely by reference to flight instruments; specifically, the skills necessatyto perform IFRflight, including departure, enroute, and arrival procedures. During each semester, the students are scheduled for 45 hoursoflecture (groundschool) and 15 flightlessons, each programmed for one week. An experimental___,;.,..· ._.

## -----·

Figure 1. 

### Student flight console on left with 20" screen. Avionics console to right of student console. Tower computer controls FS100 software for simulation. Computer on right of desk controls instructor's station.

Figure 2. 

### More detailed view of student flight console with rudder pedal assembly. Also shows avionics console to right.

4

### Table 1. Summary of Training Course Outlines for Aviation 130, the First Semester Instrument Rating Course, and Aviation 140, the Second Semester Instrument Rating Course.

Lesson'

31 32 33 34 35 

### 36 37



### 38 39 

### 40 41 42 

### 43 44 45 46 

### 47 48 49 50 51 52 53 

## 54 55 56 57 

### 58 

### 59 60 Content

Review private pilot maneuvers VFR cross-country flight review Two solo cross-countryflights Introduce basic instrument flight skills Review basic instrument flight skills, Introduce attitude instrument flight wtth distractions Introduce VOR use and orientation, VOR radial intersection holding patterns Introduce ILS, VOR, and LOC BC instrument approach procedures, procedure turns, radar vectoring, review VOR tracking Introduce OME Arcs, review holding patterns and instrument approaches Introduce instrument cross-country flight procedures Review holding patterns and instrument approach procedures Review holding patterns and instrument approach procedures Review instrument cross-country flight procedures Review holding patterns and instrument approach procedures Review all content areas, mock stage check with a different instructor Review and stage check Review private pilot maneuvers, review basic instrument flying skills, solo cross-countryflight Review basic instrument flying skills Review OME Arcs, ILS, VOR, and LOC BC approaches Introduce NOB tracking, NOB holding patterns, NOB instrument approaches Introduce localizer and localizer back course holding patterns, review NOB skills Introduce partial panel basic attitude instrument flying skills, review instrument approaches Introduce partial panel instrument approaches and holding patterns Review instrument cross-country flying skills and procedures Review instrument cross-countryflying skills and procedures Review instrument cross-countryflying skills and procedures, introduce night procedures 250 nm IFR cross-country flight, also a review flight Review instrument cross-country flying skills and procedures Mock stage check with a different instructor Review for instrument stage check Stage check
'Flight Lessons 1-30 are allocated to Aviation 101 and 120, which lead to Private Pilot Certification.



### 5**~-·**-··

curriculum for both the PCA TD group and theAirplane group (control) was developed (Emanuel, Taylor, Hulin, Lintern, Phillips, & Talleur, 1995). For the PCA TD group, all new maneuvers and procedures were introduced and trained to proficiency in a PCATD prior to training and skill validation in the aircraft. For the Airplane group, all new
maneuvers were introduced and trained to proficiency in the airplane. Tasks within a flight lesson, trained in the PCATD, were trained or tested for proficiency in the airplane before proceeding to the next flight lesson. Subjects in theAirplane groupwere trained on the same tasks but in the airplane only. Instructors rated student performances on designated flight tasks in boththePCATDandtheaircraft. For performance assessment in the aircraft, each instructor used a rating scale developed for each flight lesson to recordwhetherthe student metthe completion standards duringthe execution ofthe designated flight tasks. Instructors also recorded trials to criterion for specific tasks and flight time to compl.:!te flight lessons. The rating scales used in the project

were modeled on those that have been used in previous flight research attheUniversityoflllinois, and are described in detail in an earlier paper (Phillips, Taylor, Lintern, Hulin, Emanuel, & Talleur, 1995). Inter-raterreliabilities of0.80onthesetypes ofscales are typically achieved (Koonce, 1979). The rating scales

were used for both the PCA TD group and for the Airplane group. Forthe PCA TD group, two identical rating scales were developed for each flight lesson (or

a series ofrelatedflightlessons): onefor trainingin the PCATD and one for training in the aircraft after reaching criterionin the PCATD.An identicalrating scale was used for each flight lesson for the Airplane group. The rating scales can be found inAppendix 2 ofTaylor et al. (1996b). The guiding principles behind the development and scoring ofthese maneuver-assessment scales were to attempt to digitize the observing and scoring tasks ofthe instructor/observers. Maneuvers were divided into time segments, and the instructors were asked to record specificflight parameters, say, within±100feet oftheassigned altitudeorexceedingthis range; within ±5• of assigned heading or outside this range. By

6 breaking up the flight maneuvers into many parameterswhose values define thequalityofthemaneuver, taking digitized assessments at specified times, and summing these digitized assessments across many flight parameters and times, the overall quality ofthe total maneuver and flight can be captured very accurately. Inaddition, theworkload on the flight instructors was significantly reduced. They only had to glance at one or two instruments at specified times and note whether the student was within the range printed on their scoring sheets. By reducing the implicit analogue taSk from integrating a.student's performance on all parameters throughout a maneuver and attempting to summarize the overall quality of their performance to recording a small number of digitized scores, wewere able to maintain flight safety standards while recording performance levels. Experienced flight instructors at the University of Illinois served as both flight instructors and experimenters in the project. The instructors were trained

ondelivetyofthe instructional curriculum andonthe

use ofthe rating scales. Three check pilots, who were uninformedas totheallocation ofstudentsto training conditions, were used for the stage checks.
Analyses Percent transfer and transfer effectiveness ratios (TERs) were computedfor each flight lesson using the following equations: 

### Yc-Yx ..::..:.__:..;;. x 

### 100 = Percent Transfer 

### Yc
Yc-Yx 

### = Transfer Effectiveness Ratio

X
Where: Y, = Time/Trials in airplane byAirplane group 

### Yx = Time/Trials in airplane byPCATD group 

### Y, = Time/Trials in airplane by PCA TD group

Percent Transfer measures the difference between theairplane andthePCA TD groups in terms oftrials/ time to reach criterion in the airplane, expressed as a percent. Apositive percent transferfavors the PCATDTable 2. Instrument tasks in Aviation 130 and Aviation 140for which percent transfer and TERs for mean trials to criterion and mean time to complete the flight lesson were calculated.

Aviation 130 SteepTums Intersection Holds Instrument Landing System (JLS) Approach Localizer Back Course (LOC BC) Approach Visual Omni Range (VOR) Approach* Distance Measuring Equipment Arc Approach (DME Arc) Aviation 140 Intersection Holds ILS Approaches VOR Approaches NOB Holds NDB Approaches LOC BC Holds JLS Holds LOC BC Approaches Holding Patterns Flight Lesson••

34/35 36, 40-43, 42 37,38,39,40-43,42 37, 39, 40-43 38, 40-43, 42 38

48 48,51,52,53,56,57 48,51,52,53,56,57 49,50 49,50,51,52,53,54,55,56 50 50 52 52,53,54,55,56,57

*VOR approaches were first introduced in Flight Lesson 37, but instructions for scoring this exercise were inadvertently omitted from the score sheet. **See Table 1 forthe lesson descriptions for the respective flight lessons.

group, and a negative percent transfer favors the Airplane group. Percent Transfer does not consider the amount of prior training in the PCATD by the PCATD group. Transfer effectiveness ratio (TER)

measures the difference between the airplane and the PCATD groups in terms oftrials/time to reach criterion in the airplane, as a function ofthe amount of prior training in the PCATD by the PCA TD group, i.e., TERmeasures theeffectiveness ofthe PCA TD in training specific instrument tasks. In all instrument tasks in Aviation 130 and Aviation 140, students were trained to proficiency in the PCATD and then trained and tested in the airplane. Table 2 shows the instrument tasks trained in Aviation 130 and Aviation 140, for which mean percent transfer andmean transfer effectiveness ratios (TERs)

were computedfrom mean trials tocriterionandfrom

mean time to complete the flight lesson.

7 

### RESULTS

Mean percent transfer and mean transfer**effective~** ness ratios (TERs) are reported for two dependent

measures: number of trials to criterion for selected instrumenttasks (trials), and airplane time required in

a flight lesson for students to reach the completion standards (time). In addition, means and variances of Trials to Criterion The mean trials used to compute percent transfer and TERs for trials on instrument tasks trained in Aviation 130 are shown in Table 3. It is evident from this table that the PCA TD was more effective for introduction oftasks than for their review. For most tasks, positive transfer was found in the flight lesson during which the tasks were introduced. With steep turns, for example, the Airplane group (N=54) requiredameano£3.83 airplanetrialstoreachcriterion,while the PCATD group (N=53) required a mean of 3.40 airplane trials after a mean of3.28 priortrials in thePCATD.Thepercenttransferwas 11.2%,andthe transfer effectiveness ratio (TER) was 0.13. Therefore for this task, within the limit ofthe number oftrials administered, each trial in the PCATD saved 0.13 trials in the airplane. The difference between the

means of0.43 trials as tested by a t-test (two tailed)

was nonsignificant. For two tasks, ILS approach and VORapproach, the percent transfer was 33.3% (ILS, Flight Lesson 37) and 20.4% (VOR, Flight Lesson 38). The TERs for these two tasks were 0.28 (ILS, Flight Lesson 37) and 0.25 (VOR, Flight Lesson 38). For each ofthe tasks, each prior training trial in the PCATD saved one fourth or more trials in the airplane. FortheILS task (FlightLesson 37),thePCATD group required 1.50 mean trials to reach criterion in the airplane, compared with 2.25 mean trials for the Airplane group. Thedifference between themeans of 0.75 trials was significant: 1(86) = 4.18, 12 < 0.001 (two-tailed). For the VORtask (Flight Lesson 38), the PCATD group required 1.33 mean trials to reach criterion, compared with 1.67 for the Airplane group. The difference between the means of0.34 trials was significant: 1(93) = 2.37, 12 < 0.02 (two-tailed). For localizer, back course (LOC BC) (Flight Lesson 37) and the DME/Arc (Flight Lesson 38) approaches, the transfers were 17.5% and 18.1

### o/o, respectively. The TERs for these 2 tasks were 0.17 (LOC BC) and 0.20 (DME/Arc). For theLOCBC (Flight Lesson 37), the PCA TD group required 1.46 mean trials to reach criterion compared with 1.77 for theAirplane group. Thedifference of0.31 trials was not significant:t(97)

= 1.73, 12 < 0.09 (two-tailed). Similarly, the mean trials to criterionontheDME/ Arc task (FlightLesson 38) was 1.58 for the PCA TD group compared with 1.93 for the Airplane group. The difference between the means of0.35 trials was not significant: 1(98) = 1.84,12 < 0.07 (two-tailed).

Additional trainingwas given oneach taskafterthe flight lesson in which a task was introduced. For example, theILS approachandtheLOCBCapproach

were introducedin FlightLesson 37. Additional trainingwas conducted on the ILS in Flight Lesson 38 and

ontheLOCBC approach in FlightLesson 39. As seen

8 in Table 3, the mean number ofuials to reach proficiency in the airplane for theAirplane group and the PCATD group was substantially smaller for Flight Lessons 38 and 39compared with FlightLesson 37,as
was the mean number oftrials to reach proficiency in the PCATD. For example, for the ILS task in Flight Lesson 37, the Airplane and the PCATD groups required 2.25 and 1.5 trials, respectively, to reach proficiencyin the airplane after 2.70 prior trials in the PCATD by the PCA TD group. Review ofthe ILS in FlightLesson 38,however, indicates thattheAirplane and PCATD groups required only 1.36 and 1.20 trials, respectively, toreachproficiencyintheairplane after 1.33 prior trials in the PCATD by the PCATD group. The difference between the means was not significant. For Flight Lessons 38 and 39, the effectiveness ofthe prior training in the PCATD, as defined by percent transfer and the TER, was also reduced. For example, for the ILS task, the percent transfer was 33.3% in Flight lesson 37 and 11.8% in Flight Lesson 38; the TERwas 0.28 in Flight Lesson 37 and 0.12 in Flight Lesson 38. The means used to compute percent transfer and TERs for instrument tasks trained in Aviation 140 after the students had completed the experimental program in Aviation 130 are shown in Table4. Flight Lesson 48 was a review ofiLS andVORtasks thathad been introduced and reviewed extensively during the previous semester inAviation 130. The percent transfer for ILS was 18.7% and the TER was 0.23. The

mean trials toreach criterion for theILS taskinFlight Lesson 48 was 1.26mean trials for the PCATD group and 1.55 mean trials for the Airplane group. The difference between the means of 0.29 trials was not significant:1(57) = 1.60,12 <0.12 (two-tailed). Forthe VOR, the percent transfer was 14.6% and the TER

was 0.17. For theVOR task in Flight Lesson 48, the PCATD group required 1.05 mean trials to reach criterion compared with 1.23 trials for the Airplane group. The difference between the means of0.18 trials

was significant: t(50) = 2.20, 12 < 0.03 (two-tailed). Non Directional Beacon (NDB) holds and approaches were introduced in Flight Lesson 49. Positive transferwas found for themeantrials to proficiency for the NDB holds and the NDB approaches in that lesson. The percent transfer for NOB holds andTable 3. Mean trials in the airplane for the Airplane group 

### (Yc) and the PCATD group 

### (Yx). and the
mean trials in the PCATD (X), percent transfer and training effectiveness ratios (TEAs) for instrument tasks trained in Aviation 130. Levels of statistical significances of differences between 

### Yc and 

### Yx are shown.

Percent Task Grou11 Transfer TER 

### Yc Yx X Steep Turns (Fit Lessons 34/35) 3.83 3.40 3.28 ns 11.2% 0.13 lnt Hold (Fit Lesson 36)** ILS (Fit Lesson 37) 2.25 1.50 2.70 <0.001 33.3% 0.28 LOC BC (Fit Lesson 37) 1.77 1.46 1.81 <0.09 17.5% 0.17 VOR (Fit Lesson 38) 1.67 1.33 1.38 <0.02 20.4% 0.25 ILS (Fit Lesson 38)* 1.36 1.20 1.33 ns 11.8% 0.12 DME/Arc (Fit Lesson 38) 1.93 1.58 1.76 <0.07 18.1% 0.20 ILS (Fit Lesson 39)* 1.40 1.50 1.29 ns 7.9% 0.10 LOC BC (Fit Lesson 39)" 1.51 1.47 1.49 ns 2.7% 0.03 ILS (Fit Lessons 40,41 ,43)* 2.94 3.10 2.69 ns 5.4% 0.06 LOC BC (Fit Lessons 40,41 ,43)* 1.87 1.70 1.62 ns 4.9% 0.10 VOR (Fit Lessons 40,41 ,43)* 3.33 3.30 2.47 ns 1.0% 

### O.ot ILS (Fit Lesson 42)* 1.39 1.59 1.44 ns -3.6% -0.03 VOR (Fit Lesson 42)* 1.13 1.06 1.16 ns -2.3% -0.03

*Review tasks.
••An incorrect definition of a hold trial precluded meaningful data collection.

approachwas 19.7% and 21.0%, respectively, andthe TERs were 0.16 and 0.17, respectively. For the NOB holds task, the PCATO group required 1.18 mean trials; theAirplane group required 1.47 trials to reach criterion. Thedifference of0.29 trials was notsignificant:!(57) =**1.93,~** < 0.06 (two-tailed). ThePCATO group required 1.54 NOB approach trials to reach criterion compared with 1.95 mean approach trials for the Airplane group. The difference between the

means was significant: !(67) = 2.29,**~** < 0.03 (twotailed). The review ofNOB holds and approaches in the PCATO during Flight Lesson 50 also produced positive transfer. The percent transfer for NOB holds

was 15.8%, and the TERwas 0.16. These scores were about the same as the percent transfer and TER for holds when previously introduced in Flight Lesson

9 49. In Flight Lesson 50, the PCATO group required 1.12holdtrials to reach criterion compared with 1.33 trials for the Airplane group. The difference between the means was significanq(59) = 2.05,12 < 0.05 (twotailed). For review of NOB Approaches in Flight Lesson 50, the percent transfer was 28.1 o/o and the TER was 0.39, both of which were larger than the values found for Flight Lesson 49 when NOB approaches were introduced. For the NOB approach task in Flight Lesson 50, the PCATO group required 1.18 mean trials to reach criterion compared with 1.64 mean trials for the Airplane group. The difference between themeans of0.46was significant, !(54)

= 2.70,**~** < 0.01 (two-tailed). The LOC BC holds in Flight Lesson 50 also showed a significant amountof transfer. Percenttransferwas 26.5%andtheTERwas

### Table 4. Mean trials in the airplane for the Airplane group 

### (Yc) and the PCATD group 

### (Yx). mean trials in the PCATD (X), percent transfer and training effectiveness ratios (TEAs), for selected instrument tasks trained in Aviation 140 after the students completed experimental training in Aviation 130. Levels of statistical significances of differences between 

### Y c and 

### Y xare shown.

Percent Task Groug I! Transfer TEA 

### Yc Yx X

ILS (Fit Lesson 48) 1.55 1.26 1.26 ns 18.7% 0.23 VOR (Fit Lesson 48) 1.23 1.05 1.05 <0.03 14.6% 0.17 NOB Holds (Fit Lesson 49) 1.47 1.18 1.87 <0.06 19.7% 0.16 NOB Approach (Fit Lesson 49) 1.95 1.54 2.36 <0.03 21.0% 0.17 NOB Holds (Fit Lesson 50) 1.33 1.12 1.33 <0.05 15.8% 0.16 NOB Approach (Fit Lesson 50) 1.64 1.18 1.18 <0.01 28.1% 0.39 LOC BC Holds (Fit Lesson 50) 1.47 1.08 1.29 <0.03 26.5% 0.30 ILS Holds (Fit Lesson 50) 1.14 1.11 1.14 ns 2.6% 0.03 ILS (Fit Lesson 51)* 1.10 1.03 1.20 ns 6.4% 0.06 NOB (Fit Lesson 51)* 1.15 1.13 1.21 ns 1.7% 0.02 VOR (Fit Lesson 51)* 1.10 1.00 1.10 ns 9.1% 0.09 ILS (Fit Lesson 52)* 1.09 1.22 1.16 ·ns ·11.9% -0.11 VOR (Fit Lesson 52)* 1.17 1.29 1.51 ns -13.2% -0.10 NOB (Fit Lesson 52)* 1.14 1.19 1.73 ns -1.7% -0.01 LOC BC (Fit Lesson 52)* 1.15 1.10 1.19 ns 4.4% 0.04 ILS (Fit Lesson 53)* 1.29 1.14 1.18 ns 11.6% 0.13 VOR (Fit Lesson 53)* 1.06 1.00 1.12 ns 5.1% 0.05 NOB (Fit Lesson 53)* 1.06 1.06 1.07 ns 0.0% 0.00 Holds (Fit Lesson 53)* 1.07 1.14 1.04 ns -6.5% -0.07 NOB (Fit Lesson 54)* 1.45 1.34 1.21 ns 7.6% 0.09 Holds (Fit Lesson 54)* 1.23 1.03 1.12 ns 16.3% 0.18 NOB (Fit Lesson 55)* 1.62 1.56 1.61 ns 3.7% 0.04 Holds (Fit Lesson 55)* 1.11 1.24 1.09 ns -11.7% -0.12 ILS (Fit Lesson 56)* 1.14 1.11 1.17 ns 2.6% 0.03 VOR (Fit Lesson 56)* 1.22 1.06 1.16 ns 4.9% 0.06 Holds (Fit Lesson 56)* 1.22 1.11 1.20 ns 9.0% 0.09 ILS (Fit Lesson 57)* 1.14 1.14 1.03 ns 0.0% 0.00 VOR (Fit Lesson 57)* 1.14 1.14 1.03 ns 0.0% 0.00 Holds (Fit Lesson 57)* 1.04 1.18 1.12 ns -13.5% -0.13

•Review tasks.

100.30, which indicates that almost one-third ofa trial

was saved in the Airplane for the PCATD group for each prior trial in the PCATD. For the LOC BC task inFlightLesson 50, the PCATD group required 1.08

mean trials to reach criterion, compared with 1.47

mean trials for the Airplane group. The difference between themeans of0.39 trials was significant:l{40)

= 2.19, I! < 0.03 (two-tailed). None of the tasks reviewed in the remaining fight lessons indicated significant transfer. The means used to compure percent transfer and TERs for instrument tasks trained in Aviation !40 in theFall1994 are shown inTable S.As inAviation 130 and the combined Aviation !40 data in Table 4, the introduction oftasks generallyresultedinbettertransfer than review oftasks. Forexample, non directional beacon {NDB) holds andapproacheswereintroduced in FlightLesson 49. High positive transfer was found for the mean trials to proficiency for NDB holds and NDB approaches in thatlesson. The percent transfer

was 32.5% and 42.8%, respectively, and the TERs

were 0.35 (NDB holds) and 0.40 (NDB approach). Each NDB hold trial andeach NDB approach trial in the PCATD saved over one-third of a hold or approach trial in the airplane. For the students inAviation 140 trained in the Fall 1996, the transfer effectiveness forbothNDBholdsandNDBapproaches for the PCATD group was lower in Flight Lesson 50 than for Flight Lesson 49, when NDB holds and approaches were introduced. An NDB hold trial in the PCA TD saved approximately one-tenth ofa trial in the airplane, while an NDB approach trial in the PCATD group produced no savings.



### Time to Complete Flight Lesson The mean times to complete the flight lessons in Aviation 130 are shown in Table 6, together with associated values of percent transfer and the TERs. Transfer oftraining was substantial for manylessons. Positivetransferobservedrangedfrom 22.7%transfer for Flight Lesson 38 {review of VOR and ILS and introduction of DME Arc) to 37.5% transfer for Flight Lessons 34/35 {introduction of steep turns). Positive TERs ranged from 0.23 for Flight Lesson 36 {intersection holds) to 0.50 for Flight Lessons 34/35 (introduction to steep turns). For Flight Lessons 34/

II35, the PCATD group saved one-half hour in the airplanefor eachpriorhourinthePCATD. ForFlight Lessons 34/35 {introductionofsteep turns), themean time for the flight lesson for the PCATD group was 0.95 mean hours compared with 1.52 mean hours for the Airplane group; the difference between the mean of0.57 hours was significant: !(78) = 5.0I,I!< 0.001 {two-tailed). ForFlight Lesson 37 (ILS and LOC BC approaches), the percent transfer was 27.5 and the TER was 0.32, which indicated that the PCATD group saved almost one-third ofan hour in the airplane for each prior hour in the PCATD. For Flight Lesson 37 (ILS and LOC BC approaches), the mean time for the PCATD group to complete the flight lesson was 1.74mean hours comparedwith 2.40 mean hours for theAirplane group. The difference between the means of0.68 hours was significant: !(94) = 3.32, I!< 0.00 I (two-tailed). For FlightLesson 36, intersection holds, the PCATD group saved almost onefourth hour in the airplane for each prior hour in the PCATD. The PCATD group required 1.34 mean hours to complete Flight Lesson 36 (intersection holds), compared with 1.81 hours for the Airplane group. The difference between the means of 0.47 hours was significant: 1(99) = 2.89, I!< 0.001 (tworailed). ForFlightLesson 38 (reviewofVORandILS, and Introduction to DME Arc) the percent transfer

was 22.7 and the TERwas 0.30. The PCATD group required 1.53 mean hours to complete this flight lesson, compared with 1.98 mean hours for the Airplane group. Thedifference between the means of0.45

was significant: 1(85) = 2.82, jl < O.Ql (two-tailed). The times to complete Aviation 140 flight lessons for subjects who had completed the experimental curriculum in Aviation 130 are shown in Table 7, together with the associated percent-transfers and TERs. Transferoftraining, as measured bythe time to complete the flight lesson, was substantial for the introduction of NDB holds and approaches (Flight Lesson 49) and for their first review (Flight Lesson 50). The percent transfer was 26.4% and 19.2%, respectively, for Flight Lesson 49 and 50. The TERs

were 0.22 and 0.24 respectively for Flight Lesson 49 and50, which indicatedfor bothofthese lessonsprior trainingonNDBholds andapproaches inthePCATD saved almost one-fourth ofan hour in the airplane.

# -r---------------------------------._a.

Table 5. Mean trials in the airplane for the Airplane only group (Y

### c) and the PCATD group (Y

### xl and the mean trials in the PCATD (X), and percent transfer and training effectiveness ratios (TEAs), for instrument tasks trained in Aviation 140 (Fall 1994 data). Percent Task Groug Transfer TEA 

### Yc Yx X

Holds (Fit Lesson 48)* 1.69 1.93 2.43 -14.2% -0.10 ILS (Fit Lesson 48)* 2.23 2.00 2.64 10.3% 0.09 VOR (Fit Lesson 48)* 2.31 1.79 1.79 22.5% 0.29 NDB Holds (Fit Lesson 49) 1.69 1.14 1.57 32.5% 0.35 NDB Approach (Fit Lesson 49) 2.62 1.50 2.79 42.8% 0.40 NDB Holds (Fit Lesson 50)* 1.38 1.27 1.00 8.0% 0.11 NDB Approach (Fit Lesson 50)* 1.25 1.33 1.08 -6.4% -0.07 LOC BC Holds (Fit Lesson 50) 1.38 1.31 1.07 5.1% 0.07 ILS Holds (Fit Lesson 50) 1.15 1.08 1.08 6.1% 0.06 ILS (Fit Lesson 51)* 1.00 1.22 1.79 -22.0% -0.12 VOR (Fit Lesson 51)* 1.09 1.08 1.15 0.9% 0.01 NDB (Fit Lesson 51)* 1.50 1.00 2.00 33.3% 0.25 ILS (Fit Lesson 52)* 1.25 1.27 1.43 -1.6% -0.01 VOR (Fit Lesson 52)* 1.17 1.14 1.42 2.6% 0.02 NDB (Fit Lesson 52)* 1.62 1.08 2.10 33.3% 0.26 LOC BC (Fit Lesson 52)* 1.25 1.00 1.29 20.0% 0.19 Holds (Fit Lesson 52)* 1.08 1.00 1.50 7.4% 0.05 ILS (Fit Lesson 53)* 1.31 1.21 1.50 7.6% 0.07 VOR (Fit Lesson 53)* 1.10 1.15 1.00 -4.6% -0.05 NDB (Fit Lesson 53)* 1.00 1.14 1.20 -14.0% -0.12 Holds (Fit Lesson 53)* 1.67 1.00 1.33 40.1% 0.50 NDB (Fit Lesson 54)* 1.17 1.29 1.50 -10.3% -0.08 Holds (Fit Lesson 54)* 1.10 1.08 1.00 1.8% 0.02 NDB (Fit Lesson 55)* 1.75 1.50 1.93 14.3% 0.13 Holds (Fit Lesson 55)* 1.44 1.27 1.15 11.8% 0.15 ILS (Fit Lesson 56)* 1.00 1.14 1.07 -14.0% -0.13 VOR (Fit Lesson 56)* 1.17 1.29 1.14 -12.0% -0.11 NDB (Fit Lesson 56)* 1.07 Holds (Fit Lesson 56)* 1.00 1.25 1.29 -25.0% -0.19 ILS (Fit Lesson 57)* 1.00 1.07 1.14 -7.0% -0.06 VOR (Fit Lesson 57)* 1.25 1.29 1.17 -3.2% -0.03 Holds (Fit Lesson 57)* 1.17 1.13 1.22 3.4% -0.03

*Review tasks.

12Table 6. Mean time (in hours} to complete the flight lesson for V0 , 

### Y., and X, and percent transfer and TERs for Aviation 130 (Combined data for study). Levels of statistical significances of differences between 

### Yc and 

### Yx are shown.

Percent Flight Lesson Groug Transfer TEA •, 

### Yc Yx X

34/35 (Steep Turns, N=54 and 53) 1.52 0.95 1.13 <0.001 37.5% 0.50 36 (Intersection Holds, N=54 and 53) 1.81 1.34 2.07 <0.001 26.0% 0.23 37 (ILS, LOC BC, N=54 and 53) 2.40 1.74 2.05 <0.001 27.5% 0.32 38 (VOA, ILS, OME Arc,

N=54 and 52) 1.98 1.53 1.51 <0.01 22.7% 0.30 39 (ILS, LOC BC, N=52 and 52) 2.12 2.24 1.20 ns -5.7% -0.10 40,41,43 (ILS, LOC BC, VOA,

Holds, N=48 and 48) 4.75 4.72 3.49 ns 0.6% 0.01 42 (ILS, VOA, Holds, N=49 and 52) 2.25 2.38 1.19 ns -5.8% -0.11

Table 7. Mean time (in hours} to complete flight lesson for

### Yc. 

# v •. and X and percent transfer and TERs for Aviation 140 (After completing training in Aviation 130}. Levels of statistical significances of differences between 

### Yc and 

### Yx are shown.

Percent Flight Lesson Groug Q Transfer TEA 

### Yc 

# v. X

49 (NOB Holds, Approach, N=38

and 39) 1.63 1.20 1.98 <0.001 26.4% 0.22 50 (NOB Holds, Approach, ILS

and LOC BC Holds, N=38 and 39) 1.88 1.52 1.51 <0.04 19.2% 0.24 51 (ILS, NOB, VOA, N=38 and 38) 1.60 1.58 1.47 ns 1.3% 0.01 52 (ILS, NOB, VOA, LOC BC,

Holds, N=38 and 38) 2.05 2.02 1.92 ns 1.5% 0.02 53 (ILS, NOB, VOA, Holds, N=38

and 38) 2.13 2.15 1.32 ns -0.9% -0.02 54 (NOB, Holds, N=38 and 37) 2.02 2.02 0.83 ns 0.0% 0.00 55 (NOB, Holds, N=38 and 37) 2.47 2.52 1.00 ns -2.0% -0.05 56 (ILS, VOA, Holds, N=36 and 37) 3.94 3.99 1.49 ns -1.3% -0.03 57 (ILS, VOA, Holds, N=37 and 37) 2.14 2.49 1.12 ns -16.4% -0.31

13

# F1

ForFlightLesson 49 (introduction ofNDBholds and approaches), the PCATD group required 1.20 mean hours to complete the flight lesson compared with 1.63 mean hours for the Airplane group. The difference between the means was significant: !(57) = 2.96, ll < 0.001 (two-tailed). NDB holds and approaches, ILS and LOC BC holds were reviewed in Flight Lesson 50. The mean time to complete the flight lesson for the PCATD group was 1.52 mean hours compared with 1.88 mean hours for the Airplane group. The difference between the means of 0.36 hours was significant: t(65) = 2.06, 

### 12 < 0.04 (twotailed). Therewas no significant transfer, as measured by time, for theremaining flight lessons. The percent transfer was -16.4%, and the TER was -0.31 which

means that for each hour in the PCATD it took the PCA TD group almost one-third ofan hour longer to complete the lesson in the airplane than the Airplane group. This difference was not significant. The mean times to complete selected flight lessons in Aviation 140 are shown for the Fall1994 data in Table 8, together with associated values of percent transfer and the TERs. As indicated in Table 8, for FlightLesson 49 (the introductionofNDB holds and approaches) and for their review (Flight Lesson 50), the transfer effectiveness for the time to complete the flight lesson variable was positive and substantial. Transfer for Flight Lesson 49 was 31.6% and 23.2% for Flight Lesson 50. TERs for lessons 49 and 50 were 0.31 and 0.35, respectively. PCATD students saved about a third ofan hour in the airplane for each hour in the PCATD.

Course Completion Time The means andvariances for total airplane time for theAirplane and the PCATD groups to complete the two courses are shown in Table 9. For Aviation 130, the PCATD group required a mean of21.0 hours to complete the course, compared with 23.1 hours for theAirplanegroup.Thedifferencebetween themeans of2.1 hours was significant: t(90) = 3.53,11 < 0.001 (two tailed). The data for thoseAviation 140 subjects who first completed the Aviation 130 experimental program indicated that the PCATD group required a
mean of 26.37 hours to complete the course, compared with 28.18 hours for the Airplane group. The difference between the means of 1.81 hours was significant: !(72) = 2.61,11 < 0.01 (two tailed).



### Table 8. Mean time (in hours) to complete flight lesson for

## v •. Yx. and X, and percent transfer and TEAs for Aviation 140 (Fall 1994 Data).

Percent Flight Lesson Grou11 Transfer TER 

# v. Yx X

49 (NOB Holds, Approaches, N=13

and 14) 1.90 1.30 1.92 31.6% 0.31 50 (NOB Holds, Approaches, ILS

and LOC BC, N=13 and 14) 1.81 1.39 1.19 23.2% 0.35 51 (ILS, NOB, VOR, N=13 and 14) 1.54 1.61 2.20 -7.0% -0.03 52 (ILS, NOB, VOR, LOC BC,

Holds, N=13 and 14) 2.52 1.73 1.96 31.4% 0.40 53 (ILS, NOB, VOR, Holds, N=13

and 14) 2.18 2.08 1.06 4.6% 0.09 54 (NOB, Holds, N=12 and 14) 1.99 1.85 0.87 7.0% 0.07 55 (NOB, Holds, N=13 and 14) 2.74 2.50 1.05 8.8% 0.23 56 (ILS, VOR, Holds, N=12 and 14) 4.02 4.15 1.07 -3.2% -0.12 57 (ILS, VOR, Holds, N=12 and 14) 2.13 2.16 1.02 -1.4% -0.03

14Table 9. Mean Airplane times (in hours) and variances to complete courses for Airplane and PCATD groups for Aviation 130 and Aviation 140.

Mean (Yc-YJ Variance 

### !::! ! Q

Aviation 130 (Yc) Airplane 23.1 (YJ PCATD 21.0 2.1

Aviation 140 Airplane 28.2 PCATD 26.4 1.8

Aviation 140 (Fa111994) Airplane 29.8 PCATD 26.7 3.1

For the Aviation 140 Fall1994 data, the PCATD group required a mean of26.7 hours to complete the
course, compared with 29.8 hours for the Airplane group. Thedifference between themeans of3.1 hours

was significant: I(25) = 2.91, I!< 0.05 {two tailed).


### DISCUSSION

The data from the studyindicate that the PCATD is an effective training device for teaching instrument tasks to private pilots. These data show a significant level of transfer savings for course hours overall. Approximately4.0 hours were saved for Aviation 130 andAviation 140 combined. This represents aworthwhile benefitbecauserental costs are $69/hourfor the airplane used in this experiment and $5/hour for the PCATD. The savings may seem marginal in light of the fact that time spent in the PCA TD to generate those savings exceeded 12.5 hours for Aviation 130 and 14 hours for Aviation 140. It should be noted, however, that the procedures developed to satisfy the goal of the experiment generated inefficiencies. In thattheprimarygoal was to explore all possibilities of contributions ofthe PCATD, a considerable amount ofthe PCA TDtrainingwas undertaken toreviewtask

areas that resulted in little or no transfer. Training PCATD flight lessons to proficiencyalso reduced the efficient use ofthe PCATD.

15 8.3 45 8.3 47 3.5 < 0.001

8.41 36 9.47 38 2.61 < 0.01

9.85 13 5.67 14 2.91 < 0.05

Some signs of negative transfer are also evident. None ofthese effects was statistically significant, and they probably reflect random fluctuations. The data that have been generated in this experiment offer a solid guide to development ofan instrument syllabus that could maximize the efficiency of the PCATD. Indeed, further savings should be possible from targetingfor PCATDtrainingonlythose tasks thatshow worthwhile positive transfer. Specifically, the data suggest that the same savings ofapproximately four hours could be achieved by use ofapproximately 6.75 PCATD hours in Flight Lessons 34 to 38 and 4.5 PCATD hours in Flight Lessons 45 to 50.

Aviation 130 Flight Lessons 34 to 38 were used to develop proficiencyin a number ofbasic instrument tasks and to introduce and review various instrument hold and approach procedures. Total lesson times for each of these lessons showed significant levels of transfe; ranging from approximately 20 to 40%, with TERs rangingfrom approximately 0.20 to 0.50. Analyses of individual tasks revealed that, in almost all cases, this transfer resulted, at least in part, from the learning of specific instrument maneuvers when they were first introduced.The single exception to this trend was in Flight Lessons 34 and 35,when the task ofsteep turns failed toreveal significant transfer, buttherewas significant transfer for the overall flight lesson. In that Flight Lessons 34 and 35 included a considerable amountof nonspecific flying beyond steep turns that were also practicedin thePCATD,we assumetheenhancement

was generated in these nonspecific flying activities. No single instrument procedure trained in Aviation 130 showed statistically significant transfer in any lesson in which it was reviewed. A few ofthe transfer indexes for reviews of instrument procedures were negative, bur none approachedstatisticalsignificance. Flight Lessons 39 to 42 prepared students for InstrumentFlightRules (IFR) cross-countryflight. Ir

was difficult to develop aPCATD scenario thatwould assist thestudentswithvery much ofthis preparation. Theexercise thatwas developed had studentspractice ILS, VOR, and LOC BC that would be flown in the cross-country flight, which provided an opportunity to rehearse these tasks in the PCATD prior to the cross-country flight. The percent transfer and TER

wereinsensitive measures interms ofdetermining the potential benefit ofapproach rehearsal during those flight lessons. The specific objectives of the flight lessons prevented substantial transfer, as measured by the percent transfer and TER. Flight Lesson 43 was a review of IFR procedures and cross-country skills with the completion standards to the Instrument Rating Practical Test Standards for the lesson content with no assistance from the instructor.lrwasrequired thatthe PCATD group fly the PCATD a minimum ofone hour per lesson in Flight Lessons 40 and 41. The group could use the PCATD up to a maximum of five hours in Flight Lessons 40-43 combined. Additional PCATD time

was available with prior permission. Since the means of airplane trials to completion standards show that most students required only one trial to achieve proficiency on each of these exercises, high levels of positive transfer were not expected. One approach trial would be expected in cross-country flight, since the students would typically perform only one ILS, VOR, and/orLOCBC approachin the airplane atany airport.

16 The final two lessons in Aviation 130 are devoted to a practice check flight and a·check flight. These lessons may appear at first to offer a critical point for measuring training effectiveness. However, 

### it was assumed at the outset that these measures would be insensitive to differences between the Airplane and the PCA TD groups. By design of the experiment, students from both groups had been brought to the

same proficiency standards lesson by lesson in the airplane. The only possibility that real differences would show up in the practice or the real check flight is that one or the other ofthe training conditions had provided some sort oflearning experience that contributed substantially to overall flight competence, as evaluated in the check flight, bur had nor been taught explicitly. There is no evidence here that this occurred.

Aviation 140 As noted earlier, Aviation 140 students were not treated identicallyin this experiment. Thefirst group ofAviation 140 subjects (Fall 1994) had not been involved in the experiment duringtheirAviation 130 training. The remaining Aviation 140 subjects had been in the experiment as Aviation 130 subjects prior to taking the experimental curriculum in Aviation 140.Theirdataare combinedinto asingledatasetand treated separately from the Aviation 140 data collected during in the Fall 1994. Some differences between the two sets of results are evident. Most noticeably, therewas adifference in transfer, based on rime required ro complete Flight Lesson 52, for transfer based ontrials to reach criterion for review ofNDB and LOC BC approaches in Flight Lesson 52, and for transfer based on trials to reach criterion for review of holds in Flight Lesson 53. There are a number of possiblereasons for changesofthis typefrom semester to semester, two being variations in weather and a changingpoolofinstructors.Themost likelyexplanation is that the small number ofstudents in the Fall 1994, coupled with the fact that these review lessons required very few trials to criterion in the airplane, induced some instability in the data. Because ofthe limited resources available for this project, we could not investigate the cause ofthese differences. Giventhepossibilityofunstabledatafor the Fall1994group ofAviation 140 students, we emphasize the Aviation 140 combined data set in this discussion.



### As in Aviation 130, task introduction generally resulted in bettertransfer than task review but several exceptions were observed. Flight Lesson 48 was a review lesson for the ILS and VOR approaches. The subjects had received substantial practice on these tasks in Aviation 130 during the previous semester. The data indicate that, compared with the Airplane group, the PCATD group saved in the airplane approximately one-fifth of the trials on the VOR approaches, due to prior review in the PCATD. This is strong evidence that the PCA TD is an effective tool for reviewing previously learned instrument tasks if

some time has elapsed since the tasks were last practiced. Indeed, these dataare inmarked contrasttothe review data obtained in Aviadon 130 that indicated review of instrument tasks learned to proficiency during theprevious lesson was not aproductiveuse of the PCATD. In other respects, the observations to be drawn on the data from the Aviation 140 closely follow those drawn on the data from Aviation 130. Again, it was the early lessons in which new material was introduced that there was significant positive transfer. Flight Lessons 49 and 50 showed transfer values of approximately 20 and 25%, with TERs ofapproximately 0.20 and 0.25. All tasks in these two lessons

were introduced here for the first time; all, with the exception ofNDB holds in Flight Lesson 49 and ILS holds inFlightLesson 50, showed statisticallysignificant positive transfer. These values ranged from approximately 15 to 30%, with TERs in the range of approximately 0.15 to 0.40. With the exception of VOR in Flight Lesson 48 and NDB holds and approachesinFlightLesson 50, noreviewedtasks showed statistically significant positive transfer, and none of the negative transfer values was statistically significant. No times to complete the flight lessons in Aviation 140 showed statistically significant positive transfer, withtheexception ofFlightLessons 49 and 50. As in Aviation 130, the practice checkflight for Aviation 140 (Flight Lesson 58) and the checkflight (FlightLesson 60) mayappearatfirst to offera critical pointfor measurementoftrainingeffectiveness. How17ever, these two lessonswere again unlikelyto discriminate between the training conditions because in**pre~** ceding lessons, instructors were required ro bring students from both groups to the same standards of proficiency. Flight Lesson 59 was used to review instrument procedures for which individual students had not met instrument rating practical test standards. There were relatively few students providing data on any specific exercise. It is likely that instructors usedthislesson to selectivelyreinforce a student's weakest areas. Someofthetransferratiosinthislesson

are high and there is also one large negative value. Nevertheless, these ratios are difficult to interpret because ofthe selective nature ofthe data.



### Substitution ofthe 

### PCATD for Instrument 

### Flight Hours CurrentFAAregulations,FARPart 141.41,permit the substitution of 15 hours of time in a certified ground trainer for aircraft time required for instrumentcettification (FAA, 1992). TheFAAFlightStandards Office has drafted an Advisory Circular, which provides that some ofthose hours ofthe instrument time required by FAR Part 141 can be flown in a PCA TD. The draft advisory circular, if approved, would permitaPCATD, that meets the requirements ofFAR Part 141.41 (a) (1), to be used in lieu ofsome ofthe time afforded an authorized ground training device under FAR Part 141, Appendix C. 

### As mentioned earlier, the PCATD group required a mean 47.4hours intheairplaneto completetheinstrument rating, compared with 51.3 hours for the Airplane group. This represents a saving of approximately 4 hours. This finding supports the proposal to permit thesubstitutionofPCATDtimefor instrumenttime. Indeed, the data presented here suggest that the PCA TD is effective in teaching all instrument tasks. This experiment is specific to evaluation of a PCATD. It is possible that a generic training device would be moreeffective andmight notsuffer from the occasional problems ofnegative transfer we have observed here. However, the available evidence does not support this belief (Hampton et al., 1994) and there is little evidence that a generic training device would offer any benefit in addition to that gained from the PCATD.

### SUMMARY

The data show that trainingin a PCATD resulted in positive transfer for instrument flight tasks when measured in reference to trials to criterion or time to completea flight lesson intheairplane. Thebenefitof training in the PCA TD varied substantially among the instrument tasks tested (from 15% to over 40%). Ingeneral, savings werepositiveand substantial when

new tasks were introduced. Thedataindicated that a PCATD can save apartoftheaircraft time thatwould otherwisebe needed. Acomparison ofcoursecompletion times for the rwo courses resulted in savings of about four hours for the PCATD group, compared with the Airplane group. Time constraints and other organizational limitationsprevented the full use ofthe PCA TD in all flight lessons. One substantial constraint on savings is an FAA-mandated requirementoffourhours ofpilot-incommand cross-country time in Aviation 130 and 12.5 hours in Aviation 140. In the present experiment, the PCATD was not used to teach cross-country skills. It would, however, be possible to offer practice on cross-country scenarios. With development of an appropriate assessment tool, it may be possible to demonstrate that PCATD-trained students could achieve full competence with cross-country flight well before they had completed their mandated exercises. Adjustments in the way the PCATD has been used may extend its usefulness in relation to these tasks. The data also reveal that some tasks do not benefit to a large extent from training in the PCATD, and that use ofthe PCATD is more beneficial for training ofnewtasks than itis for reviewoftasks trainedearlier in the course. These results suggest principles that might be used to target a PCATD for maximum effectiveness within a flight training program. This

was notdone in the present study. Our objective was to investigate the instrument tasks that could be trained using a PCA TD. In thatthe overall savings in each ofthe courses, although statistically significant, are relatively small, it may be desirable to ensure that

aPCATDis focused specificallyonareas thatresultin substantial transfer to the airplane. 

### REFERENCES

Dennis, 

### KA (1994). Computer-based simulation as adjunct to the initial stages 

### ofprivate pilot device 

### training. Cranlield, England: Cranfield University. Emanuel, T.W.,Jr.,Taylor,H.L.,Hulin,C.L.,Lintern, G., Phillips, S.I., & 

### Talleur, D.A. (1995). Developmentofan experimental flight syllabus for testing PC computer-based aviation trainingdevices. In R.S. Jensen & L.A. 

### Rakovan 

### (Eds.),Proceedings of the Eighth InternationalSymposium onAviation Psychology (pp. 1198-203). Columbus, OH: The Ohio State University. Falun, N. (1992, September). Azuresoft elite. 

### Flight Training, 50-2. FAA. (1992). 

### Flightschools (14 CFR Part 141). Federal Aviation Administration, Department ofTransportation. Flexman, R.E., Roscoe, S.N., Williams, A.C., 

### Jr., & 

### Wtlliges, B.H. (1972,June). 

### Studies inpilottraining: Tht anatomy 

## of transftr (Aviation Research Monographs, vol. 2, no. 1). Savoy, IL: University of Illinois, Institute of Aviation, Aviation Research Laboratory. Hampton, S., Moroney, W., Kirton, T., & 

### Biers, D.W. (1994). 

### The useofpmonalcomputer-basedtraining tkvices in teachinginstrument flying:A comparative study (Final Report). Daytona Beach, FL: EmbryRiddle Aeronautical University.

18

### Koonce, J.M. (1979). Predictive validity offlight simulators as a function ofsimulator motion. 

### Human Factors, 21(2), 215-23. Lert, P. (1990, April). Flight simulator programs. 

### Air Progress, 20-68. Lintern, G., Roscoe, S.N., Koonce,}.M., & 

### Segal, L.D. (1990). Transfer of landing skill in beginning flight training. 

### Human Factors, 32, 319-27. Ortiz, G.A. (1993). PC-based flight simulation effectiveness. In R.S. Jensen & 

### D. Neumeister (Eds.), 

### Proctedings of the Seventh InternationalSymposium
onAviation Psychology (pp. 698-701). Columbus, OH: TheOhio State University. Ortiz, G.A (1994). Effectiveness of PC-based flight simulation. 

### The Inttrnational journal of Aviation Psychology, 4, 285-91.Peterson, C.,Jr. (1993,April). Controllingyour simulator software. 

### Private Pilot, 72-7. Phillips, S.I., Hulin, C.L., & Lamermayer, P.J. (1993). Usesofpart-tasktrainers in instrumentflight training. In R.S. Jensen & D. Neumeister (Eds.), 

### Proceedings of the Seventh InternationalSymposium on AviationPsychology (pp. 743-46).Columbus, OH: The Ohio State University. Phillips, S.I., Taylor, H.L., Lintern, G., Hulin, C.L., Emanuel, T.W.,Jr., & Talleur, D.A. (1995). Developingperformancemeasures for evaluatingpersonal computer-based aviation training devices within an FAR part 141 pilot training school. In RS.Jensen & L.A. Rakovan (Eds.), 

### Proceedings of the Eighth International Symposium on Aviation Psychology (pp. 1204-9). Columbus, OH: The Ohio State University. Povenmire, H.K., & Roscoe, S.N. (1971). An evaluation of ground-based flight trainers in routine primaryflight training. 

### Human Factors, 15, 109-16. Povenmire, H.K., & Roscoe, S.N. (1973). Incremental transfer effectiveness of a ground-based general aviation trainer. 

### Human Factors, 15, 534-42. Roscoe, S.N. (1971). Incremental transfer effectiveness. 

### Human Factors, 13, 561-7. Roscoe, S.N., & Williges, B.H. (1980). Measurement oftransfer oftraining. In S.N. Roscoe 

### (Ed.),Aviationpsychology (pp. 182-93). Ames, IA: The Iowa State University Press. Taylor, H.L., Lintern, G., & Hulin, C.L. (1995). 

### Transfir training effectiveness of personal computer-based aviation training devices (Interim Report). Savoy, IL: University ofIllinois, Institute ofAviation. Taylor, H.L.,Lintern, G., Hulin, C.L.,Emanuel,T.W., Jr., Phillips, S.I., &Talleur,D.A. (1995). Transfer of training effectiveness of personal computerbased aviation training devices. In R.S. Jensen & L.A. Rakovan (Eds.), 

### Proceedings ofthe Eighth International Symposium on Aviation Psychology (pp. 1210-5). Columbus, OH: The Ohio State University.

19Taylor, H.L., Lintern, G., Hulin, C.L., Talleur, D.A., Emanuel, T.W., & Phillips, S.I. (1996a). 

### Apreliminary study ofan integrated syllabus using a personal computer aviation training device and a generic training device. Savoy, IL: University of Illinois, Institute ofAviation. Taylor, H.L., Lintern, G., Hulin, C.L., Talleur, D.A., Emanuel,T.W., & Phillips, S.I. (199Gb). Transfer of training effectiveness of personal computerbased aviation training devices. Final Technical Report ARL-96-3/FAA-96-2. Savoy, IL: University ofIllinois, Institute ofAviation. Taylor, H.L., Lintern, G., Hulin, C.L., Talleur, D.A., Phillips, S.I., & Emanuel,T.W. (1996). Effectiveness ofpersonal computer-based aviation training devices. 

### Poster session presentedat the Eighth AnnualConvention of theAmericanPsychologicalSociety, San Francisco, CA. Taylor, H.L., Lintern, G., Koonce, J.M., Kaiser, R.H., & Morrison, G.A. (1991). Transfer of training and quasi-transfer ofscene detail and visual augmentation guidance in landing training. 

### Proceedings of trainingtramfir- Can we trust flightsimulation (pp. 6.1-6.14). London: Royal Aeronautical Society. Williams, A.C., Jr., & Flexman, R.E. (1949). Evaluation ofthe school link as an aid in primary flight instruction. 

### University of IllinoisAeronauticsBulletin, 

### 46(5), 71. Williams, K.W. (Ed.). (1994). 

### Summary Proceedings of thejointIndustry-FAA Conference on the Development and Use of PC-BasedAviation Training Devices. Washington,DC: DepartmentofTransportation/Federal Aviation Administration. (DOT/ 

### FAAIAM-94125. NTIS #N95-14917). Williams, K.W., & Blanchard, RE. (1995). 

### Development of qualification guidelinesforpersonal computer-basedaviation trainingdevices. Washington, DC:DepartmentofTransportation/FederalAviationAdministration. (DOT/FAA/AM-95/6.NTIS #ADA-292961).

*U.S. GOVERNMENT PRINnNG omCE: 1997 •**5~/400t'f**