July 2014 

NASA/TP–2014-218279 

General Aviation Flight Test of Advanced 

Operations Enabled by Synthetic Vision 

Louis J. Glaab and Monica F. Hughes 

Langley Research Center, Hampton, Virginia 

Russell V. Parrish 

Raytheon Company, Hampton, Virginia 

Mohammad A. Takallu 

Lockheed Martin Corporation, Hampton, Virginia 
NASA STI Program . . . in Profile

Since its founding, NASA has been dedicated to the 

advancement of aeronautics and space science. The 

NASA scientific and technical information (STI) 

program plays a key part in helping NASA maintain 

this important role. 

The NASA STI program operates under the 

auspices of the Agency Chief Information Officer. 

It collects, organizes, provides for archiving, and 

disseminates NASA’s STI. The NASA STI 

program provides access to the NASA Aeronautics 

and Space Database and its public interface, the 

NASA Technical Report Server, thus providing one 

of the largest collections of aeronautical and space 

science STI in the world. Results are published in 

both non-NASA channels and by NASA in the 

NASA STI Report Series, which includes the 

following report types: 

• TECHNICAL PUBLICATION. Reports of 

completed research or a major significant phase 

of research that present the results of NASA 

Programs and include extensive data or 

theoretical analysis. Includes compilations of 

significant scientific and technical data and 

information deemed to be of continuing 

reference value. NASA counterpart of peerreviewed formal professional papers, but 

having less stringent limitations on manuscript 

length and extent of graphic presentations. 

• TECHNICAL MEMORANDUM. Scientific 

and technical findings that are preliminary or of 

specialized interest, e.g., quick release reports, 

working papers, and bibliographies that contain 

minimal annotation. Does not contain extensive 

analysis. 

• CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees. 

• CONFERENCE PUBLICATION. 

Collected papers from scientific and 

technical conferences, symposia, seminars, 

or other meetings sponsored or cosponsored by NASA. 

• SPECIAL PUBLICATION. Scientific, 

technical, or historical information from 

NASA programs, projects, and missions, 

often concerned with subjects having 

substantial public interest. 

• TECHNICAL TRANSLATION.

English-language translations of foreign 

scientific and technical material pertinent to 

NASA’s mission. 

Specialized services also include organizing 

and publishing research results, distributing 

specialized research announcements and feeds, 

providing information desk and personal search 

support, and enabling data exchange services. 

For more information about the NASA STI 

program, see the following: 

• Access the NASA STI program home page 

at http://www.sti.nasa.gov

• E-mail your question to help@sti.nasa.gov

• Fax your question to the NASA STI 

Information Desk at 443-757-5803 

• Phone the NASA STI Information Desk at 

443-757-5802 

• Write to: 

 STI Information Desk 

 NASA Center for AeroSpace Information 

 7115 Standard Drive 

 Hanover, MD 21076-1320
National Aeronautics and 

Space Administration 

Langley Research Center 

Hampton, Virginia 23681-2199 

July 2014 

NASA/TP–2014-218279 

General Aviation Flight Test of Advanced 

Operations Enabled by Synthetic Vision 

Louis J. Glaab and Monica F. Hughes 

Langley Research Center, Hampton, Virginia 

Russell V. Parrish 

Raytheon Company, Hampton, Virginia 

Mohammad A. Takallu 

Lockheed Martin Corporation, Hampton, Virginia 
Available from: 

NASA Center for AeroSpace Information 

7115 Standard Drive 

Hanover, MD 21076-1320 

443-757-5802 
1

Table of Contents

1.0 ABSTRACT ....................................................................................................................................11

2.0 INTRODUCTION ............................................................................................................................11

3.0 METHOD ......................................................................................................................................13

3.1 FLIGHT TEST AIRCRAFT AND APPARATUS .....................................................................................................13

3.2 DISPLAY CONCEPTS.................................................................................................................................14

3.3 SVS TERRAIN PORTRAYAL ........................................................................................................................21

3.4 EVALUATION PILOTS ...............................................................................................................................21

3.5 SCENARIOS, TRAINING, AND OPERATIONS ...................................................................................................22

3.5.1 VFR Approach................................................................................................................................24

3.5.2 ILS Approach..................................................................................................................................24

3.5.3 VMC-Like Approach.......................................................................................................................24

3.5.4 Training .........................................................................................................................................25

3.5.5 Operations.....................................................................................................................................25

3.6 INDEPENDENT VARIABLES.........................................................................................................................26

3.7 TEST MATRIX.........................................................................................................................................26

3.8 DEPENDENT MEASURES...........................................................................................................................28

3.8.1 Qualitative Measures....................................................................................................................28

3.8.2 Quantitative Measures..................................................................................................................29

4.0 DATA ANALYSES AND RESULTS .....................................................................................................31

4.1 SITUATION AWARENESS...........................................................................................................................32

4.1.1 VFR approach................................................................................................................................33

4.1.2 ILS approach, all pilots ..................................................................................................................33

4.1.3 VMC-Like approach.......................................................................................................................35

4.1.4 ILS approach, without VFR pilot data............................................................................................35

4.1.5 ILS and VMC-Like approach analysis, all pilots, advanced displays ..............................................36

4.1.6 Test condition analysis..................................................................................................................36

4.2 SUBJECTIVE WORKLOAD ..........................................................................................................................38

4.2.1 VFR approach................................................................................................................................39

4.2.2 ILS approach, all pilots ..................................................................................................................39

4.2.3 VMC-Like approach.......................................................................................................................40

4.2.4 ILS approach without VFR pilot data.............................................................................................41

4.2.5 ILS and VMC-Like approach analysis, all pilots, advanced displays ..............................................41

4.2.6 Test condition analysis..................................................................................................................42

4.3 DISPLAY READABILITY RATING...................................................................................................................43

4.3.1 VFR approach, all pilots.................................................................................................................46

4.3.2 ILS approach, all pilots ..................................................................................................................46

4.3.3 VMC-Like approach, all pilots........................................................................................................46

4.3.4 ILS approach without low-time VFR pilot data..............................................................................47

4.3.5 ILS and VMC-Like approach analysis, all pilots, advanced displays ..............................................48

4.3.6 Test condition analysis..................................................................................................................48

4.4 TERRAIN AWARENESS..............................................................................................................................49

4.4.1 VFR approach discussion...............................................................................................................51

4.4.2 ILS approach, all pilots ..................................................................................................................52

4.4.3 VMC-Like approach, all pilots........................................................................................................53

4.4.4 ILS approach without low-time VFR pilot data..............................................................................53

4.4.5 ILS and VMC-Like approach analysis, all pilots, advanced displays ..............................................54
2

4.4.6 Test condition analysis..................................................................................................................55

4.5 PTS RESULTS.........................................................................................................................................57

4.5.1 ILS approach, all pilots ..................................................................................................................58

4.5.2 VMC-Like approach, all pilots, advanced displays.........................................................................60

4.5.3 ILS approach Segment 3 without VFR pilot data...........................................................................61

4.5.4 ILS and VMC-Like approaches, Segment 3, all pilots, advanced displays......................................61

4.5.5 Test condition analysis, Segment 3 ...............................................................................................64

4.6 LOCALIZER ERROR...................................................................................................................................65

4.6.1 VFR approach, all pilots.................................................................................................................65

4.6.2 ILS approach, all pilots ..................................................................................................................68

4.6.3 VMC-Like approach, all pilots........................................................................................................70

4.6.4 ILS approach, Segment 3, no VFR pilots........................................................................................72

4.6.5 ILS and VMC-Like approach analysis, Segment 3, all pilots...........................................................73

4.6.6 Test condition analysis, Segment 3 ...............................................................................................75

4.7 GLIDESLOPE ERROR.................................................................................................................................77

4.7.1 VFR approach, all pilots.................................................................................................................77

4.7.2 ILS approach, all pilots ..................................................................................................................80

4.7.3 VMC-Like approach.......................................................................................................................82

4.7.4 ILS approach, Segment 3, no VFR pilots........................................................................................85

4.7.5 ILS and VMC-Like approach analysis, Segment 3, all pilots...........................................................86

4.7.6 Test condition analysis, Segment 3 ...............................................................................................89

4.8 INDICATED AIRSPEED ERROR.....................................................................................................................91

4.8.1 VFR approach all pilots, S2 and 3..................................................................................................92

4.8.2 ILS approach, all pilots, Segments 2 and 3....................................................................................95

4.8.3 ILS approach, Segment 3, no VFR pilots........................................................................................97

4.8.4 VMC-Like approach, all pilots, Segments 2 and 3 .........................................................................97

4.8.5 ILS and VMC-Like approach analysis, Segment 3, all pilots...........................................................98

4.8.6 Test condition analysis, Segment 3 .............................................................................................101

4.9 LATERAL FLIGHT-PATH ERROR.................................................................................................................102

4.9.1 VFR approach..............................................................................................................................103

4.9.2 ILS approach, all pilots ................................................................................................................105

4.9.3 VMC-Like approach, all pilots......................................................................................................108

4.9.4 ILS approach, Segment 3, without VFR pilot data.......................................................................111

4.9.5 ILS and VMC-Like approach analysis, Segment 3, all pilots.........................................................112

4.9.6 Test condition analysis, Segment 3 .............................................................................................113

4.10 VERTICAL FLIGHT PATH ERROR................................................................................................................115

4.10.1 VFR approach..........................................................................................................................115

4.10.2 ILS approach, all pilots............................................................................................................118

4.10.3 VMC-Like approach, all pilots.................................................................................................120

4.10.4 ILS approach, Segment 3, without VFR pilot data ..................................................................122

4.10.5 ILS and VMC-Like approach analysis, Segment 3, all pilots....................................................122

4.10.6 Test condition analysis, Segment 3.........................................................................................123

5.0 SUMMARY OF RESULTS ..............................................................................................................125

6.0 LIMITATIONS OF CURRENT STUDY ..............................................................................................125

7.0 CONCLUSIONS ............................................................................................................................126

8.0 REFERENCES ...............................................................................................................................129

APPENDIX A: SINGLE-CUE FLIGHT DIRECTOR DESCRIPTION ..................................................................132

APPENDIX B: HIGHWAY-IN-THE-SKY GUIDANCE SYSTEM......................................................................135
3

APPENDIX B REFERENCES: ...................................................................................................................................135

APPENDIX C: QUESTIONNAIRES...........................................................................................................136

C.1 GENERAL ............................................................................................................................................136

C.2 EVALUATION PILOT QUESTIONNAIRE (EPQ) TOOL ......................................................................................136

C.3 POST-RUN QUESTIONNAIRES, BASIC BLOCK: ..............................................................................................137

APPENDIX D: INDIVIDUAL SUBJECTIVE DATA PLOTS.............................................................................141
4

List of Figures

FIGURE 1: NASA LANGLEY CESSNA 206. .............................................................................................................................14

FIGURE 2: BASELINE ROUND-DIAL DISPLAY CONCEPT .............................................................................................................15

FIGURE 3: SINGLE-CUE FLIGHT DIRECTOR DISPLAY CONCEPT....................................................................................................16

FIGURE 4: PATHWAY-BASED GUIDANCE DISPLAY CONCEPT......................................................................................................17

FIGURE 5: SYNTHETIC VISION SYSTEM DISPLAY CONCEPT.........................................................................................................17

FIGURE 6: CLOSE-UP OF DISPLAY CONCEPTS .........................................................................................................................19

FIGURE 7A: EXPANDED VIEW OF SVS DISPLAY CONCEPT PFD...................................................................................................20

FIGURE 8A: VFR SECTIONAL MAP VIEW WITH APPROACHES OVERLAID......................................................................................23

FIGURE 9: SART RESULTS, ALL DISPLAY CONCEPTS, ALL PILOTS AND ALL APPROACHES ................................................................33

FIGURE 10: TLX RESULTS, ALL DISPLAY CONCEPTS, ALL PILOTS, AND ALL APPROACHES ................................................................39

FIGURE 11: CUMULATIVE PERCENTAGE OF DRR, ALL DISPLAY CONCEPTS AND ALL PILOTS ............................................................45

FIGURE 12: TERRAIN AWARENESS RESULTS FOR ALL DISPLAY CONCEPTS, ALL PILOTS AND ALL APPROACHES.....................................51

FIGURE 13: PERCENT TIME WITHIN IFR PTS PERFORMANCE METRICS, ALL PILOTS, SEGMENTS 2 AND 3..........................................58

FIGURE 14: PERCENT TIME WITHIN IFR PTS PERFORMANCE METRIC, ALL PILOTS, SEGMENT 3 ONLY ..............................................62

FIGURE 15: VFR APPROACH, LOCALIZER ERROR, BY DISPLAY CONCEPT AND PILOT GROUP............................................................67

FIGURE 16: ILS APPROACH, LOCALIZER ERROR, BY DISPLAY CONCEPT AND PILOT GROUP .............................................................69

FIGURE 17: VMC-LIKE APPROACH, LOCALIZER ERROR, BY DISPLAY CONCEPT AND PILOT GROUP ...................................................71

FIGURE 18: LOCALIZER ERROR, SEGMENT 3 FOR ILS AND VMC-LIKE APPROACHES, ALL PILOTS......................................................73

FIGURE 19: VFR APPROACH, GLIDESLOPE ERROR, BY DISPLAY CONCEPT AND PILOT GROUP..........................................................79

FIGURE 20: ILS APPROACH, GLIDESLOPE ERROR, BY DISPLAY CONCEPT AND PILOT GROUP ...........................................................81

FIGURE 21: VMC-LIKE APPROACH, GLIDESLOPE ERROR, BY DISPLAY CONCEPT AND PILOT GROUP .................................................84

FIGURE 22: GLIDESLOPE ERROR FOR SEGMENT 3, FOR APPROACHES 2 AND 3, ALL PILOTS ............................................................86

FIGURE 23: VFR APPROACH, IAS PERFORMANCE FOR ALL PILOTS AND DISPLAY CONCEPTS ...........................................................94

FIGURE 24: ILS APPROACH, IAS PERFORMANCE FOR ALL PILOTS AND DISPLAY CONCEPTS.............................................................96

FIGURE 25: VMC-LIKE APPROACH, IAS PERFORMANCE FOR ALL PILOTS AND DISPLAY CONCEPTS .................................................100

FIGURE 26: IAS ERROR FOR SEGMENT 3, ALL PILOTS, ALL APPROACHES ..................................................................................101

FIGURE 27: VFR APPROACH LATERAL ERROR FOR EACH DISPLAY, EACH PILOT GROUP ................................................................104

FIGURE 28: IFR APPROACH LATERAL ERROR, BY DISPLAY CONDITION AND PILOT GROUP............................................................107

FIGURE 29: VMC-LIKE APPROACH LATERAL ERROR, BY DISPLAY CONCEPT AND PILOT GROUP .....................................................110

FIGURE 30: LATERAL ERROR FOR SEGMENT 3 FOR THE ILS AND VMC-LIKE APPROACH, ALL PILOTS ..............................................111

FIGURE 31: VFR APPROACH, VERTICAL ERROR, BY DISPLAY CONCEPT AND PILOT GROUP ...........................................................117

FIGURE 32: ILS APPROACH, VERTICAL ERROR, BY DISPLAY CONCEPT AND PILOT GROUP.............................................................119

FIGURE 33: VMC-LIKE APPROACH, VERTICAL ERROR, BY DISPLAY CONCEPT AND PILOT GROUP...................................................121

FIGURE 34: VERTICAL ERROR, SEGMENT 3, FOR APPROACHES 2 AND 3, ALL PILOTS ...................................................................124

FIGURE A- 1: SINGLE CUE FLIGHT DIRECTOR........................................................................................................................132

FIGURE A- 2: SINGLE-CUE FLIGHT DIRECTOR PITCH AND ROLL ATTITUDE COMMAND BLOCK DIAGRAM. ............................................133

FIGURE B- 1: SVS PRIMARY FLIGHT DISPLAY SHOWING HITS GUIDANCE SYMBOLOGY.................................................................135

FIGURE C- 1: EXAMPLE IMAGE OF THE EVALUATION PILOT QUESTIONNAIRE TOOL. .....................................................................136

FIGURE C- 2: HAYWORTH-NEWMAN MODIFIED COOPER HARPER (DISPLAY READABILITY)...........................................................140

FIGURE D- 1:SART AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE FOR EACH EVALUATION, VFR APPROACH. .........................142

FIGURE D- 2: SART AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE FOR EACH EVALUATION, ILS APPROACH...........................143

FIGURE D- 3: SART AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE, VMC-LIKE APPROACH. ................................................144

FIGURE D- 4 TLX AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE, VFR APPROACH.............................................................145

FIGURE D- 5: TLX AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE, ILS APPROACH. ............................................................146

FIGURE D- 6: TLX AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE, VMC-LIKE APPROACH....................................................147
5

FIGURE D- 7: DRR AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE, VFR APPROACH..........................................................148

FIGURE D- 8: DRR AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE, ILS APPROACH. ...........................................................149

FIGURE D- 9: DRR AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE, VMC-LIKE APPROACH..................................................150

FIGURE D- 10: TERRAIN AWARENESS AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE, VFR APPROACH..................................151

FIGURE D- 11: TERRAIN AWARENESS AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE, ILS APPROACH....................................152

FIGURE D- 12: TERRAIN AWARENESS AS A FUNCTION OF DISPLAY CONCEPT AND PILOT TYPE, VMC-LIKE APPROACH..........................153
6

List of Tables 

TABLE 1 EVALUATION PILOT DATA ......................................................................................................................................22

TABLE 2: INDEPENDENT VARIABLES USED FOR THIS EXPERIMENT .............................................................................................26

TABLE 3: TEST MATRIX ....................................................................................................................................................27

TABLE 4: DEFINITION OF TEST CONDITION VARIABLE...............................................................................................................28

TABLE 5: SART DESCRIPTIVE STATISTICS (ALL PILOTS).............................................................................................................32

TABLE 6: MEAN SART SNK RESULTS FOR ALL PILOTS, ILS APPROACH .......................................................................................34

TABLE 7: MEAN SART SNK RESULTS FOR ALL PILOTS FOR ALL DISPLAY CONCEPTS, ILS APPROACH ................................................34

TABLE 8: MEAN SART SNK RESULTS FOR ALL PILOTS FOR ALL DISPLAY CONCEPTS, VMC-LIKE APPROACH ......................................35

TABLE 9: MEAN SART RESULTS FOR IFR AND H-IFR PILOTS FOR THE ILS APPROACH ...................................................................35

TABLE 10: MEAN SART SNK RESULTS FOR DISPLAY CONCEPT FOR IFR AND H-IFR PILOTS FOR THE ILS APPROACH...........................36

TABLE 11: MEAN SART SNK RESULTS FOR ADVANCED DISPLAY CONCEPTS FOR ALL PILOTS FOR ILS AND VMC-LIKE APPROACH .........36

TABLE 12: MEAN SART SNK RESULTS FOR TEST CONDITION...................................................................................................37

TABLE 13: TLX DESCRIPTIVE STATISTICS (ALL PILOTS).............................................................................................................38

TABLE 14: MEAN TLX SNK RESULTS FOR DISPLAY CONCEPT FOR ALL PILOTS AND DISPLAY CONCEPTS, ILS APPROACH.......................40

TABLE 15: MEAN TLX SNK RESULTS FOR THE ADVANCED DISPLAY CONCEPTS FOR THE VMC-LIKE APPROACH ..................................40

TABLE 16: MEAN TLX SNK RESULTS FOR PILOT TYPE FOR THE VMC-LIKE APPROACH..................................................................41

TABLE 17: MEAN TLX SNK RESULTS FOR DISPLAY CONCEPT FOR IFR AND H-IFR PILOTS FOR ALL DISPLAY CONCEPTS FOR THE ILS

APPROACH..............................................................................................................................................................41

TABLE 18: MEAN SNK TLX RESULTS FOR DISPLAY CONCEPT FOR ALL PILOTS FOR THE ILS AND VMC-LIKE APPROACH (ADVANCED 

DISPLAY CONCEPTS).................................................................................................................................................42

TABLE 19: MEAN TLX SNK RESULTS FOR TEST CONDITION .....................................................................................................43

TABLE 20: DISPLAY READABILITY PERCENTAGE OF TIME SELECTED RESULTS (ALL PILOTS)...............................................................44

TABLE 21: DISPLAY READABILITY RESULTS FOR THE ILS APPROACH, IFR AND H-IFR PILOTS ...........................................................47

TABLE 22: DRR RESULTS FOR TEST CONDITION.....................................................................................................................49

TABLE 23: TA DESCRIPTIVE STATISTICS (ALL PILOTS)...............................................................................................................50

TABLE 24: MEAN TA SNK RESULTS FOR DISPLAY CONCEPT, FOR ALL PILOTS FOR THE VFR APPROACH ............................................52

TABLE 25: MEAN TA SNK RESULTS FOR DISPLAY CONCEPT, FOR ALL PILOTS FOR THE ILS APPROACH .............................................52

TABLE 26: MEAN TA SNK RESULTS FOR DISPLAY CONCEPT FOR ALL PILOTS FOR THE VMC-LIKE APPROACH ....................................53

TABLE 27: MEAN TA SNK RESULTS FOR THE ILS APPROACH, IFR AND H-IFR PILOTS .................................................................54

TABLE 28: MEAN TA SNK RESULTS FOR PILOT TYPE FOR THE ILS AND VMC-LIKE APPROACH FOR ALL PILOTS ..................................54

TABLE 29: MEAN TA SNK RESULTS FOR DISPLAY CONCEPT FOR THE ILS AND VMC-LIKE APPROACH, ALL PILOTS .............................55

TABLE 30: MEAN TA SNK RESULTS FOR TEST CONDITION.......................................................................................................56

TABLE 31: PTS DESCRIPTIVE STATISTICS (% TIME), SEGMENTS 2 AND 3 FOR ALL PILOTS ...............................................................57

TABLE 32: MEAN PTS (% OF TIME) SNK RESULTS FOR DISPLAY CONCEPT FOR ALL PILOTS, ILS APPROACH. .....................................59

TABLE 33: MEAN PTS (% OF TIME) SNK RESULTS FOR PILOT TYPE FOR ALL DISPLAY CONCEPTS, ILS APPROACH. .............................59

TABLE 34: MEAN PTS (% OF TIME) RESULTS FOR PILOT TYPE AND DISPLAY CONCEPT, ILS APPROACH. ...........................................60

TABLE 35: MEAN PTS (% OF TIME) SNK RESULTS FOR DISPLAY CONCEPT FOR ALL PILOTS, VMC-LIKE APPROACH ............................60

TABLE 36: MEAN PTS (% OF TIME) SNK RESULTS FOR PILOT TYPE FOR ALL DISPLAY CONCEPTS, VMC-LIKE APPROACH ....................61

TABLE 37: PTS (% OF TIME) RESULTS FOR DISPLAY CONCEPT FOR SEGMENT 3 OF THE ILS APPROACH, IFR AND H-IFR PILOTS ONLY....61

TABLE 38: PTS (% OF TIME) CRITERIA FOR DISPLAY CONCEPT AND APPROACH INTERACTIONS, ILS AND VMC-LIKE APPROACHES,

SEGMENT 3 ............................................................................................................................................................63

TABLE 39: PTS (% OF TIME) CRITERIA RESULTS FOR DISPLAY CONCEPT AND PILOT TYPE INTERACTIONS, ILS AND VMC-LIKE 

APPROACHES, SEGMENT 3.........................................................................................................................................63

TABLE 40: PTS (% OF TIME) CRITERIA RESULTS FOR APPROACH AND PILOT TYPE INTERACTIONS, ILS AND VMC-LIKE APPROACHES,

SEGMENT 3 ............................................................................................................................................................63

TABLE 41: MEAN PTS (% OF TIME) SNK CRITERIA RESULTS FOR TC, ALL PILOTS, SEGMENT 3 .......................................................64

TABLE 42: LOCALIZER ERROR (DOTS) DESCRIPTIVE STATISTICS FOR ALL PILOTS.............................................................................65

TABLE43: MEAN LOCALIZER ERROR (DOTS) SNK RESULTS FOR DISPLAY CONCEPT VFR APPROACH FOR ALL PILOTS ............................66

TABLE 44: MEAN LOCALIZER ERROR (DOTS) SNK RESULTS FOR DISPLAY CONCEPT FOR ILS APPROACH FOR ALL PILOTS .......................68
7

TABLE 45: LOCALIZER ERROR RESULTS (DOTS) FOR DISPLAY CONCEPT AND PILOT TYPE INTERACTIONS FOR THE ILS APPROACH ............70

TABLE 46: MEAN LOCALIZER ERROR (DOTS) SNK RESULTS FOR DISPLAY CONCEPT FOR THE VMC-LIKE APPROACH FOR ALL PILOTS .......70

TABLE 47: MEAN LOCALIZER ERROR (DOTS) SNK RESULTS FOR THE ILS APPROACH FOR THE IFR AND H-IFR PILOTS ..........................72

TABLE 48: MEAN AND STANDARD DEVIATION FOR LOCALIZER ERROR (DOTS) FOR THE ILS AND VMC-LIKE APPROACH, ALL PILOTS.......74

TABLE 49: MEAN LOCALIZER ERROR (DOTS) SNK RESULTS, ILS AND VMC-LIKE APPROACH FOR ALL PILOTS .....................................74

TABLE 50: MEAN AND STANDARD DEVIATION RESULTS FOR LOCALIZER ERROR (DOTS) FOR THE ILS AND VMC-LIKE APPROACH FOR ALL 

PILOTS ...................................................................................................................................................................75

TABLE 51: MEAN LOCALIZER ERROR (DOTS) SNK RESULTS FOR TEST CONDITION .........................................................................76

TABLE 52: GLIDESLOPE ERROR (DOTS) DESCRIPTIVE STATISTICS FOR ALL PILOTS...........................................................................77

TABLE 53: MEAN GLIDESLOPE ERROR (DOTS) SNK RESULTS FOR DISPLAY CONCEPT FOR THE VFR APPROACH FOR ALL PILOTS..............78

TABLE 54: MEAN GLIDESLOPE ERROR (DOTS) SNK RESULTS FOR DISPLAY CONCEPT, ILS APPROACH FOR ALL PILOTS ..........................80

TABLE 55: MEAN GLIDESLOPE ERROR (DOTS) SNK RESULTS FOR DISPLAY CONCEPT, VFR APPROACH FOR ALL PILOTS ........................80

TABLE 56: MEAN GLIDESLOPE ERROR (DOTS) SNK RESULTS FOR DISPLAY CONCEPT, VMC-LIKE APPROACH FOR ALL PILOTS ................83

TABLE 57: MEAN GLIDESLOPE ERROR (DOTS) SNK RESULTS FOR THE ILS APPROACH, IFR AND H-IFR PILOTS ...................................85

TABLE 58: MEAN GLIDESLOPE ERROR (DOTS) SNK RESULTS FOR THE ILS AND VMC-LIKE APPROACHES FOR ALL PILOTS......................87

TABLE 59: STATISTICAL INTERACTION RESULTS FOR GLIDESLOPE ERROR (DOTS) FOR DISPLAY CONCEPT AND PILOT TYPE FOR THE ILS AND 

VMC-LIKE APPROACHES ...........................................................................................................................................88

TABLE 60: STATISTICAL INTERACTION GLIDESLOPE ERROR (DOTS) DATA FOR DISPLAY CONCEPT AND APPROACH FOR THE ILS AND VMCLIKE APPROACHES ....................................................................................................................................................89

TABLE 61: MEAN GLIDESLOPE ERROR (DOTS) SNK RESULTS FOR TEST CONDITION.......................................................................90

TABLE 62: IAS ERROR (KTS) DESCRIPTIVE STATISTICS FOR SEGMENT 2 FOR ALL PILOTS..................................................................91

TABLE 63: IAS ERROR (KTS) DESCRIPTIVE STATISTICS FOR SEGMENT 3 FOR ALL PILOTS..................................................................92

TABLE 64: MEAN IAS ERROR (KTS) SNK RESULTS FOR THE VFR APPROACH, SEGMENT 3 FOR ALL PILOTS.........................................93

TABLE 65: MEAN IAS ERROR (KTS) SNK RESULTS FOR THE ILS APPROACH, SEGMENT 2 FOR ALL PILOTS ..........................................95

TABLE 66: MEAN IAS ERROR (KTS) SNK RESULTS FOR THE ILS APPROACH, SEGMENT 3 FOR ALL PILOTS ..........................................95

TABLE 67: INCREASE IN MEAN IAS ERROR (KTS) FOR THE VFR PILOTS COMPARED TO THE AVERAGE OF THE IFR AND H-IFR PILOTS FOR 

THE ILS APPROACH, SEGMENTS 2 AND 3......................................................................................................................97

TABLE 68: MEAN IAS ERROR (KTS) FOR SEGMENT 2 AND SEGMENT 3 OF THE ILS APPROACH, IFR AND H-IFR PILOTS.......................97

TABLE 69: IAS ERROR (KTS) FOR SEGMENT 2 AND SEGMENT 3 OF THE VMC-LIKE APPROACH FOR ALL PILOTS..................................98

TABLE 70: PERCENT IAS ERROR INCREASE COMPARED TO ILS EVALUATION OF BRD DISPLAY CONCEPT (IFR AND H-IFR PILOTS) FOR 

SEGMENT 3 FOR ILS AND VMC-LIKE APPROACHES (ALL PILOTS).......................................................................................99

TABLE 71: MEAN IAS ERROR (KTS) SNK RESULTS FOR TC .....................................................................................................102

TABLE 72: LATERAL FLIGHT-PATH ERROR (FT) DESCRIPTIVE STATISTICS FOR ALL PILOTS...............................................................103

TABLE 73: MEAN LATERAL FLIGHT-PATH ERROR (FT) SNK RESULTS FOR DISPLAY CONCEPT FOR ALL PILOTS FOR THE VFR APPROACH 105

TABLE 74: MEAN LATERAL FLIGHT-PATH ERROR (FT) SNK RESULTS FOR DISPLAY CONCEPT FOR ALL PILOTS FOR THE ILS APPROACH ..108

TABLE 75: MEAN LATERAL FLIGHT-PATH ERROR (FT) SNK RESULTS FOR DISPLAY CONCEPT FOR ALL PILOTS FOR THE VMC-LIKE 

APPROACH ...........................................................................................................................................................109

TABLE 76: MEAN LATERAL FLIGHT-PATH ERROR (FT) SNK RESULTS, FOR THE ILS APPROACH, IFR AND H-IFR PILOTS ......................112

TABLE 77: MEAN LATERAL FLIGHT-PATH ERROR (FT) SNK RESULTS, SEGMENT 3 OF THE ILS AND VMC-LIKE APPROACHES FOR ALL PILOTS

..........................................................................................................................................................................113

TABLE 78: LATERAL FLIGHT-PATH ERROR STATISTICAL RESULTS (FT), SEGMENT 3 OF THE ILS AND VMC-LIKE APPROACHES FOR ALL 

PILOTS .................................................................................................................................................................113

TABLE 79: MEAN LATERAL FLIGHT-PATH ERROR (FT) SNK RESULTS FOR TC..............................................................................114

TABLE 80: VERTICAL PATH ERROR (FT) DESCRIPTIVE STATISTICS FOR ALL PILOTS ........................................................................115

TABLE 81: MEAN VERTICAL FLIGHT-PATH ERROR (FT) SNK RESULTS FOR DISPLAY CONCEPT FOR ALL PILOTS FOR THE VFR APPROACH116

TABLE 82: MEAN VERTICAL FLIGHT PATH ERROR (FT) SNK RESULTS FOR DISPLAY CONCEPT FOR ALL PILOTS,ILS APPROACH..............118

TABLE 83: MEAN VERTICAL FLIGHT-PATH ERROR (FT) SNK RESULTS FOR THE ILS APPROACH, IFR AND H-IFR PILOTS ONLY..............122

TABLE 84: MEAN VERTICAL FLIGHT-PATH ERROR (FT) SNK RESULTS FOR PILOT TYPE, ILS AND VMC-LIKE APPROACHES FOR ALL PILOTS

..........................................................................................................................................................................123

TABLE 85: MEAN VERTICAL FLIGHT-PATH ERROR (FT) SNK RESULTS FOR THE ADVANCED DISPLAY CONCEPTS, ILS AND VMC-LIKE 

APPROACHES FOR ALL PILOTS ...................................................................................................................................123

TABLE 86: MEAN VERTICAL FLIGHT-PATH ERROR (FT) SNK RESULTS FOR TC ............................................................................124
8
9

Abbreviations

2-D Two Dimensional

3-D Three Dimensional

AC Advisory Circular

ADAHRS Air Data, Attitude, and Heading Reference System

AGL Above Ground Level

ANOVA Analysis of Variance

ATC Air Traffic Control

BRD Baseline Round Dials

C206 Cessna 206 aircraft

CDI Course Deviation Indicator

CFIT Controlled Flight Into Terrain

DEM Digital Elevation Model

DH Decision Height

DRR Display Readability Rating

EBG Elevation-Based Generic terrain texturing

EP Evaluation Pilot

EPQ Evaluation Pilot Questionnaire tool

F F-ratio: a test statistic with a known probability distribution (the Fdistribution) used to test for overall differences between group means in 

experiments

FAA Federal Aviation Administration

ft Feet

GA General Aviation

GAMA General Aviation Manufacturers Association

GPS Global Positioning System

H-IFR High-time Instrument Rated Pilots

HF Human Factors

HITS Highway-In-The-Sky guidance

IAS Indicated Airspeed

IFD Integration Flight Deck Simulator

IFR Instrument Flight Rules

ILS Instrument Landing System

IMC Instrument Meteorological Conditions

kts Knots

KROA FAA 4-letter identifier for Roanoke Woodrum Airport

LCD Liquid Crystal Display

MA Missed Approach

MSL Mean Sea Level

NAS National Airspace System

ND Navigation Display

nm Nautical Mile

p Degree of significance of the results from the analysis of variance
10

PBG Pathway-Based Guidance

PFD Primary Flight Display

PTS Practical Test Standards

Q Questionnaire

RMS Root Mean Square

RNP Required Navigation Performance

SA Situation Awareness

SART Situation Awareness Rating Technique

SCFD Single-Cue Flight Director

SNK Student-Newman-Keuls post-hoc analysis technique

SP Safety Pilot

SVS Synthetic Vision System

TA Terrain Awareness

TAWS Terrain Awareness and Warning System

TC Test Condition

TE Test Engineer

TLX Task Load Index

VFR Visual Flight Rules

VMC Visual Meteorological Conditions

WAAS Wide-Area Augmentation System
11

1.0 Abstract

A flight test was performed to compare the use of three advanced primary flight and navigation 

display concepts to a baseline, round-dial concept to assess the potential for advanced operations. The 

displays were evaluated during visual and instrument approach procedures including an advanced 

instrument approach resembling a visual airport traffic pattern. Nineteen pilots from three pilot groups,

reflecting the diverse piloting skills of the General Aviation pilot population, served as evaluation 

subjects. The experiment had two thrusts:. 1) an examination of the capabilities of low-time (i.e., <400 

hours), non-instrument-rated pilots to perform nominal instrument approaches, and 2) an exploration of 

potential advanced Visual Meteorological Conditions (VMC)-like approaches in Instrument 

Meteorological Conditions (IMC). Within this context, advanced display concepts are considered to 

include integrated navigation and primary flight displays with either aircraft attitude flight directors or 

Highway In The Sky (HITS) guidance with and without a synthetic depiction of the external visuals (i.e.,

synthetic vision). Relative to the first thrust, the results indicate that using an advanced display concept, 

as tested herein, low-time, non-instrument-rated pilots can exhibit flight-technical performance, 

subjective workload and situation awareness ratings as good as or better than high-time Instrument Flight 

Rules (IFR)-rated pilots using Baseline Round Dials for a nominal IMC approach. For the second thrust, 

the results indicate advanced VMC-like approaches are feasible in IMC, for all pilot groups tested for

only the Synthetic Vision System (SVS) advanced display concept. These findings are valid for the 

conditions tested and that this test only examined nominal operations and did not include traffic 

separation, hazardous weather, equipment failures, etc. Other related factors not addressed were pilots’ 

decision-making ability or possible overconfidence.

2.0 Introduction

Synthetic Vision Systems (SVS) primary flight displays present a continuous computer-generated 

image of the external scene topography from the perspective of the flight deck, albeit possibly minified, 

creating an electronic means of continuous visual-like information for the pilot/flight crew. When 

combined with integrated guidance symbology, SVS have been shown to significantly increase situation 

awareness (SA) and pilot performance while decreasing workload during operations in Instrument 

Meteorological Conditions (IMC) (Ref. 1-12) compared to traditional flight instrument display concepts. 

Throughout this paper, for brevity, this combination of flight path guidance with SVS terrain information 

will be referred to as SVS displays or the SVS display concept. 

A critical component of SVS displays is the appropriate presentation of terrain to the pilot. 

References 1-4, 6, 9, and 12 examined the effects of two primary elements of terrain portrayal on the 

primary flight display (PFD): 1) variations of digital elevation model (DEM) resolution; and, 2) terrain 

texturing, along with other aspects of SVS displays. Higher resolution DEMs can enable more terrain 

features to be represented in the SVS terrain especially in mountainous areas. Different texturing 

concepts can be employed to convey various types of information to the pilot. One type of texturing, 

referred to as photo-realistic, applies ortho-rectified imagery to produce SVS terrain that appears very 

similar to the outside world, thereby conveying cultural feature information. Other terrain texturing 

concepts employ elevation-based terrain texturing concepts which can enhance the terrain height cues. 

References 1-4, 6, 9, and 12 reported on several generic as well as photo-realistic concepts. References 4 

and 12 expanded the analysis to include interactions between different symbology, both 2-dimensional (2-

D) and 3-dimensional (3-D), along with various SVS terrain portrayal concepts. Results from these 

studies indicated that significant improvements in SA can be realized over a wide range of DEMs and 

texturing concepts, and established recommended best practices for SVS terrain portrayal. Results from 

References 3, 4, 8, and 12 also indicated that 3-D guidance symbology, that combined a minimal 
12

highway-in-the-sky (HITS) depiction along with a flight path vector with guidance, provided significant 

benefits compared to other guidance concepts.

Traditional IMC operations feature approaches with long straight legs, gradual descents and large 

airspace requirements to accommodate historical surveillance, navigation, and flight technical 

performance considerations. These factors can be particularly restrictive at airports in mountainous areas. 

Several references (see Ref. 3-5, 7, 8, 10, 11, and 12) explored the benefits of SVS displays to enable 

advanced IMC approaches. These advanced IMC approaches resembled Visual Meteorological 

Conditions (VMC)-style approaches requiring more aggressive maneuvering than traditional IMC 

operations. If proven, these VMC-style approaches would allow a drastic reduction in the Instrument 

Flight Rules (IFR) ceiling and visibility landing minima below what would normally be in effect at 

terrain-challenged airports during periods of poor visibility. While it is possible that approach operations 

could be conducted with advanced autopilots, it is considered that keeping the pilots in the aircraft control 

loop provides a superior alternative both for safety of flight as well as potential implementation into the 

GA fleet. Results from these studies indicated that VMC-style approaches may be possible with SVS 

displays.

The current effort obtained flight test data for advanced displays and approaches not traditionally

available for General Aviation (GA) operations. This new data evaluated the potential of SVS displays to 

effectively transform IMC flight and enable advanced operations. Unlike previous research in this area 

that endeavored to show a significant improvement due to SVS display technology compared to a 

conventional baseline for specific IMC test conditions, this flight test investigated several points to more 

effectively measure, assess, and interpret the results. While the “gold standard” of IMC operations was 

included (the Instrument Landing System (ILS) approach), a nominal Visual Flight Rules (VFR)

approach was also included. This establishes the relationship between IMC and VMC operations within 

several common Human Factors (HF) measures and a metric to evaluate advanced approaches and display 

concepts. Two other conditions were included within the data set of significant note: 1) VFR-rated pilots 

flying an ILS approach in IMC with the baseline round dial display concept, and 2) an VMC-like

approach flown with the advanced display concepts in simulated IMC. The value of #1 was to establish a 

condition within the data set of unquestionably high risk that unfortunately arises in real-world 

operations. This condition also enables an evaluation of the effect of IFR pilot training across a wide 

range of display concepts. The VMC-like approach (#2) in simulated IMC was included to create a direct 

comparison of out-the-window visibility along with enabling the evaluation of a potentially advanced 

IMC approach with substantial operational benefits. The selection of pilots provided evaluated the effect 

of advanced display concepts on low-time VFR pilots along with low-time and high-time IFR-rated

pilots.

The recent advent of affordable electronic displays, with dimensions and other characteristics 

suitable for installation in small airplanes, offers GA avionics and airplane manufacturers more flexibility 

in presentation and placement of cockpit instruments and system controls than ever before possible 

according to the General Aviation Manufacturers Association (GAMA) (Ref. 13). These new 

technologies, which include SVS displays, can provide dramatic improvements in the capabilities of GA

airplanes flown by a single pilot in IMC according to Reference 13. The Federal Aviation Administration

(FAA) has also published an Advisory Circular (AC) [14] relating to the standardization of advanced 

displays, acknowledging Reference 13, and an AC on the acceptable means of compliance for synthetic 

vision and pathway depictions (Ref. 15). Overall, while AC23-26 (Ref. 15) defines criterion of how SVS 

displays can replace traditional displays, it does so within the realm of improved safety and does not 

discuss potential operational benefits due to this technology.

This paper documents the results of a flight test that was conducted to examine the hypothesis that 

SVS displays may significantly transform approach operations in IMC flight. The flight test was 

conducted using a representative GA aircraft modified to enable evaluation of four primary display 
13

concepts in rapid succession. Those primary display concepts were: 1) a baseline round-dial display 

concept; 2) a single-cue flight director display concept (without SVS terrain); 3) a pathway-based 

guidance display concept (without SVS terrain), and 4) the SVS display concept. The subject pilot 

population was as follows: 1) low-time VFR-rated pilots; 2) pilots just exceeding the minimum training 

and experience requirements to possess an IFR rating; and 3) very high-time, highly experienced IFRrated pilots. The evaluation tasks were: 1) a VFR approach; 2) a nominal Instrument Landing System 

(ILS) approach performed in simulated IMC and 3) a VMC-like approach performed in simulated IMC. 

The research location was one with substantial terrain-induced operational restrictions, especially in IMC. 

Three sub-hypotheses were examined to characterize and quantify the transformation of approach 

operations in IMC flight provided by advanced display concepts, especially those employing SVS. 

1) Using SVS displays, the workload and SA of all pilots flying approaches in IMC will be as 

good as, or better than, the same pilots flying in VMC using baseline round-dial displays. 

It is well-known that the reduced visibility in IMC has a substantial impact on pilot workload and SA 

compared to VMC flight. By demonstrating that this hypothesis is true, one could imply that SVS 

displays significantly transform approach operations in IMC flight by reducing pilot workload and 

increasing SA to levels better than or comparable to VMC.

2) Using SVS displays, low-time, non-instrument-rated pilots flying nominal ILS approaches in 

IMC will achieve flight technical error, workload, and SA results as good as, or better than, 

ratings from instrument-rated pilots conducting the same ILS approach using baseline, 

round-dial displays.

It should be noted that this experiment only examined nominal approaches and did not include traffic 

separation, hazardous weather, equipment failures, etc. Also, other important factors, such as objective 

measures of pilots’ decision-making ability or possible overconfidence were not evaluated. However, 

considering the substantial training and high standards to obtain an IFR type rating, demonstrating that 

this hypothesis is true could imply that SVS displays have the potential to significantly transform flying 

for low-time VFR pilots, perhaps even enabling them to fly safely in IMC. 

3) Using SVS displays, all pilots, including low-time, non-instrument-rated pilots, will be able to 

conduct IMC approaches with geometries similar to a VFR pattern with flight technical 

error, workload, and SA results as good as, or better than, results from instrument rated 

pilots conducting ILS approaches using baseline round-dial displays.

The third hypothesis concerns flight performance using SVS displays to conduct IMC landing 

approaches with geometries other than traditional long, straight-in final approach segments. Again, 

considering the substantial training and high standards required to obtain an IFR type rating, 

demonstrating that this hypothesis is true could imply that SVS displays have the potential to significantly 

transform flying for all pilots, perhaps even enabling low-time VFR pilots to fly approaches with curved 

segments safely in IMC. 

3.0 Method

A coordinated simulation and flight-test effort was conducted using the NASA Langley Research 

Center Integration Flight Deck simulator (IFD) and the NASA Langley Cessna 206 (C206) aircraft. This 

report presents results from the flight test effort (see Reference 11 for the IFD simulator results). 

3.1 Flight Test Aircraft and Apparatus

A C206 aircraft (see Figure 1) was modified for this flight test through installation of a 10.4 inch

diagonal Liquid Crystal Display (LCD) research display on the right side of the instrument panel; a 6.4
14

inch LCD research display in the top of the center radio stack; two electro-mechanical, simulator-grade,

round-dial gauges; two general-purpose computers; and an air data and attitude and heading reference 

system. The two electro-mechanical round-dial gauges were required since the 10.4 inch LCD, that was 

the largest that could be installed, was not large enough to simultaneously display all of the round-dial 

gauges. These modifications established an Evaluation Pilot (EP) station on the right side of the aircraft, 

as shown in Figure 2, as well as the required infrastructure to support research operations. In the 

conventional display configuration, seven 3 inch-diameter Baseline Round Dials (BRDs) displayed 

airspeed, attitude, altitude, turn and bank, heading, vertical speed, and localizer/glideslope course 

deviation indicators (CDIs). To allow rapid comparison with the other displays, most of the round-dial 

gauges were computer-generated and displayed on the 10.4 inch LCD. For the advanced display 

configurations, the research display system enabled evaluations of dual 6.4 inch advanced displays (using 

only a portion of the 10.4 inch LCD), such as a Navigation Display (ND) and Primary Flight Display 

(PFD) with HITS symbology. The conventional display and advanced display concept configurations 

could be changed rapidly in flight through computer configuration files. As configured for this flight test, 

the EP occupied the front right seat of the aircraft, and the safety pilot (SP) occupied the front left seat. 

A Test Engineer (TE) occupied the middle row right seat and was provided with display, keyboard, and 

mouse interfaces to operate the research system and direct the test. The third seating row of the C206

aircraft was removed.

Figure 1: NASA Langley Cessna 206.

3.2 Display Concepts

Four display concepts were evaluated: 

1) Conventional Baseline Round Dials (hereinafter referred to as BRD);

2) ND and PFD with Single-Cue Flight Director (hereinafter referred to as SCFD);

3) ND and PFD with HITS providing Pathway-Based Guidance (hereinafter referred to as PBG);

4) ND and SVS PFD with HITS (hereinafter referred to as SVS). 
15

Photographs of the four display concepts as implemented in the C206 are provided in Figures 2 

through 6. Note that the ND display was invariant for the SCFD, PBG, and SVS advanced display 

concepts.

Figure 2: Baseline Round-Dial Display Concept

The SCFD display concept was included in this experiment since it is included in many 

commercially available glass-cockpit displays and offers a significant improvement over the minimum 

required baseline round dials. It also serves as a secondary baseline for comparison with the PBG and 

SVS display concepts. The attitude-based SCFD is driven by a compensatory flight director control law 

as described in Appendix A. 
16

Figure 3: Single-Cue Flight Director Display Concept

Based upon results of research conducted on SVS displays to date, such as References 1 through 12, 

perspective pathway-based guidance symbologies have been used with SVS display concepts to enhance 

integration with the 3-D SVS terrain. This integration not only improves terrain awareness relative to the 

desired route, but also enhances pilots’ overall SA since route and outside terrain are provided in the same 

frame of reference. As a result, the PBG display concept was included to create a clear evaluation of the 

differences between a single-cue attitude-based flight director and pathway-based pursuit-based guidance 

symbology as well as the direct effect of SVS terrain (i.e., the only difference between the PBG and SVS 

display concepts was the SVS terrain). The HITS guidance used for this test is known as the Crow’s-Foot 

tunnel with tadpole guidance cue (driven using a modified pursuit control law) and is described in 

References 3, 4, 8, 9, 11, and 12. Right-angle magenta triads formed the corner points of the desired 

flight path tunnel. The tunnel was set to 2 dots of localizer error and 1 dot of glideslope error and limited 

to 600 ft wide and 350 ft tall. See Appendix B for a description of the HITS guidance system employed 

herein. 
17

Figure 4: Pathway-Based Guidance Display Concept

Figure 5: Synthetic Vision System Display Concept

Some minor variations from what could be considered to be standard baseline round-dial displays

were used, including: 1) use of a sky pointer (vs. turn pointer) for the bank-angle indication on the attitude 

indicator; 2) expanded pitch scale on the attitude indicator, resulting in a potentially more precise pitch 
18

scale than is common; and, 3) use of graphics displays to present airspeed, attitude, turn and bank, and 

directional gyro gauges with simulator-grade electro-mechanical gauges for altitude and vertical speed. 

When considering the level and quality of pilot training involved in this flight test (discussed below), the 

BRD variations were considered minimal, and that the BRD display concept evaluations were

representative of pilot performance with conventional displays.
19

Figure 6: Close-up of Display Concepts

The PFD and ND display concepts were presented over a display area of approximately 6.4 inch

diagonal. This size display is representative of the smaller range of displays being developed for GA. All 

Baseline Round Dials

Single Cue Flight Director Guidance

Pathway-Based Guidance

Synthetic Vision System
20

of the advanced display concepts incorporated the same ND, and each PFD used a 60-degree horizontal 

field of regard and a corresponding vertical field of regard of 45 degrees. The track-up ND used the same 

terrain database as the SVS PFD and included: 1) a god’s-eye view of terrain; 2) terrain awareness and 

warning system (TAWS) “peaks mode” speckling on the moving map (see References 16 and 17 for 

background on TAWS); 3) flight path and waypoints; ownship symbol and turn predictor “noodle;” 4) 

route information; 5) traffic symbology; and 6) PFD viewing wedge. Traffic symbology was not driven

by actual data but used pre-recorded simulated aircraft trajectories that would initiate movement when the 

C206 crossed over a specific start point for the approach. 

The SCFD concept had integrated airspeed and altitude tapes along with a sky-pointer/roll scale and 

a single-cue flight director. The PBG concept included the crow’s foot tunnel pathway or HITS with 

tadpole guidance command and a pitch-quickened velocity vector, as well as air traffic symbology. The 

SVS concept included the features of the PBG concept with a terrain background in place of the bluesky/brown-ground presentation. This particular combination of SVS terrain and guidance symbology was 

selected based on results of previous experiments (Ref. 2-5). Close-up photographs of the four display 

concepts are provided in Figure 6. In order to provide the details of the SVS PFD and ND computer 

generated screen captures are presented in Figures 7a and 7b.

For all display concepts, maximum displayed localizer and glideslope error was set to 2.0 dots 

deflection for the ILS approaches. Indicated angular error was based on the lateral and vertical distance 

from the centerline of the desired flight path and the distance along-path to a notional ILS navigation 

beam source (i.e., localizer and glideslope) and only displayed for the ILS approach evaluations. 

Figure 7a: Expanded view of SVS Display Concept PFD
21

Figure 7b: Expanded View of Navigation Display

3.3 SVS Terrain Portrayal

The SVS terrain portrayal used for this study is referred to as the Elevation-Based Generic (EBG) 

terrain portrayal concept, developed and evaluated in earlier studies (Ref. 1-4, 6, 9, 11, and 12). The 

Digital Elevation Model (DEM) used three arc-second resolution (approximately 295 ft (90m) postspacing) terrain data. Based on results from References 2 and 3, this level of DEM resolution provides 

good terrain awareness during IMC flight.

The EBG texturing concept consisted of twelve equal-height coloration bands. These bands 

corresponded to different absolute terrain elevation levels. The coloration bands were based on VFR 

sectional charts and applied to the various local-area altitude elevation bands, where lower terrain levels 

were rendered with darker colors, while higher terrain levels were assigned lighter colors. A specific 

shade of green was set to the field elevation of 1,175 ft above Mean Sea Level (MSL). The lightest VFR 

sectional color was set to the highest terrain within 50 nm of the research airport, approximately 4000 ft 

MSL. Cultural features, such as roads and rivers, were included as objects in the terrain database. 

Simulated daylight shading was also used to accentuate the terrain features. An example of the EBG 

texturing concept is provided in Figure 7a. This view was taken from the south east looking back towards 

the airport where the flight test was conducted.

The airport model included runways with markings along with visually prominent airport buildings. 

All models were overlaid on top of the underlying elevation database. Obstacles greater than 200 ft high 

within 20 nm of the airport were represented by narrow rectangular barber-pole striped objects portraying 

their height and location, as provided within the FAA’s published obstacle database.

3.4 Evaluation Pilots

Nineteen EPs with a minimum of a current FAA pilot’s certificate and Class III medical certificate, 

or equivalent, participated. EPs were categorized into pilot types by their experience level. The pilot 

categories and mean total number of flight hours of experience are shown in Table 1. The pilot spectrum 

was selected to enable an evaluation of SVS display technology with application to the diverse GA 
22

population. The VFR, IFR, and H-IFR rated pilots represented the spectrum of pilot skills currently 

operating within the National Airspace System (NAS). 

Low-time VFR-rated pilots were included to test the hypothesis that SVS displays can enable them 

to achieve flight technical error, workload and SA ratings that are as good as, or better than, IFR pilots 

flying in IMC using baseline round dials. Instrument-rated pilots with flight time less than 1,000 hours 

were included to establish a baseline characterization of minimum IFR-rated performance as well as to 

ascertain the effect that SVS, and other advanced display technologies, would have on this segment of the 

pilot population. Very experienced pilots with instrument ratings and flight time greater than 2,000 hours 

(hereinafter referred to as H-IFR pilots) were also included to determine the effect that SVS and other 

advanced display technologies would have on highly-experienced airline and professional test pilots and 

also to facilitate comparison of results from this study with other studies. All H-IFR pilots were current 

or retired professional pilots.

Table 1 Evaluation Pilot Data

Pilot Type Number 

of EPs

Mean Flight 

Time, Hours

Minimum 

Flight Time, 

Hrs

Maximum 

Flight Time, 

Hrs

Standard 

Deviation, Hours

VFR (<400 hrs) 6 241 102 400 107

IFR (<1000 hrs) 6 468 267 620 131

H-IFR (1000+ hrs) 7 4019 1980 7000 1811

Total 19 102 7000 2141

3.5 Scenarios, Training, and Operations

Flight tests were conducted using Roanoke Woodrum Field in Roanoke, Virginia (FAA identifier 

KROA). It was selected due to the presence of significant terrain and obstructions in its vicinity and 

convenient proximity to NASA Langley Research Center. In particular, Runway 24 was selected since 

terrain and obstacles pose significant operational challenges for approaches to this runway. Runway 24 

was not equipped with visual approach slope indicator lights or a precision approach path indicator.

Three approach scenarios were created. Two were conventional approaches, and one represented an 

advanced operational concept. The conventional scenarios included a VFR traffic pattern approach in 

VMC and an ILS approach in simulated IMC. The advanced operational concept was an IMC approach

that used the same flight path flown for the conventional VFR traffic pattern approach; however, this 

“VMC-like” approach was flown in simulated IMC. Figure 8 shows a plan view and side views of the 

evaluation approaches. The three approaches are described in the following sections.
23

Figure 8a: VFR Sectional Map View with Approaches Overlaid

Figure 8b: Sideview of VFR and VMC-like Approaches 

Figure 8c: Sideview of ILS approach 
24

3.5.1 VFR Approach

The VFR traffic pattern incorporated a downwind leg displaced approximately 1.4 nm abeam the 

runway at 1,000 ft above the touchdown zone, a base leg, and a 1.5 nm final approach segment. Data 

runs were initiated at 100 kts indicated airspeed (IAS) at 1,000 ft above the touchdown zone on the 

downwind leg heading at Point A in Figure 8. EPs were instructed to use visual references primarily to 

navigate through this approach. They were instructed to fly parallel to the runway and maintain airspeed 

and altitude with flaps retracted. Once abeam the runway numbers, EPs were to select 10 degrees of 

flaps, reduce speed to 90 kts IAS, establish a 1.5 nm final approach, and complete the approach. EPs 

were instructed to continue the approach to 50 ft above ground level (AGL), or to actual touchdown, 

depending on their level of experience and prevailing conditions. The VFR traffic pattern approach was 

included in this evaluation to provide a comparison to current VMC operations.

3.5.2 ILS Approach

The ILS approach included a 90-degree base leg, a 30-degree localizer intercept, and a 3.0-degree 

glideslope intercept point approximately nine nm from the runway touchdown markers. This approach

was performed with restricted visibility through the use of an IFR training visor, which eliminated outthe-window visibility. 

Because an ILS was not available for Runway 24 at KROA, Global Positioning System (GPS)

Wide-Area Augmentation System (WAAS) information was used to calculate and display simulated ILS 

localizer and glideslope data. The altitude above the touchdown zone elevation along with the distance to

the touchdown zone generated ILS glideslope-like deviation data. Lateral linear flight path error, 

combined with the distance along the flight path to a point 1,000 ft beyond the far (departure end) end of 

Runway 24, was used to generate ILS localizer-like deviation data. Although an ILS localizer is 

nominally scaled for a course width of (full scale fly-left to a full scale fly-right) of 700 feet at the runway 

threshold, the maximum simulated localizer deflection was established as +/-250 ft at the runway 

threshold (+/-2.0 degrees course width). Maximum glideslope deflection was set to +/- 0.7 degrees. 

The ILS approach task started at approximately 2,700 ft above the touchdown zone at 100 kts IAS on 

the base leg heading at Point B in Figure 8. EPs would immediately turn to the intercept heading, once 

the research approach was started at Point B, and intercept the localizer. Once established on the 

localizer, EPs were to select 10 degrees of flaps, reduce speed to 90 kts, intercept the glideslope and 

complete the approach. Pilots were instructed to continue to the decision height (DH) at 200 ft AGL and 

then call out “DH.” At the DH, the safety pilot would say, “Continue to 50 ft AGL,” “Continue to 

landing,” or “Runway not in sight,” depending on pilot skill and prevailing conditions and whether or not 

a simulated missed approach (MA) was required as indicated by the test matrix. Calling out the “DH” 

and potentially initiating a MA was included in the flight test to enhance the realism of the approach and 

to provide data regarding pilots’ ability to identify the “DH” as well as minimize further descent during 

the MA. (These data were analyzed and no effects were observed and are subsequently not reported.) 

The test matrix was designed such that approximately half of the IMC approach runs (ILS and VMC-like) 

included an MA. Each run ended either at touchdown or when positive rate of climb was established on 

the MA. The ILS approach was included to enable data comparisons with current precision approach

operations.

3.5.3 VMC-Like Approach

The VMC-like approach used the same flight path as the VFR approach; however, it was performed 

with GPS navigation and restricted visibility through the use of an IFR training visor. A waypoint 

(“NEWT”) was created for the VMC-like approach on the downwind approach path (see Figure 8) abeam 

the runway numbers to designate the point at which the EPs were to select 10 degrees of flaps and 
25

decelerate to 90 kts IAS. Localizer and glideslope errors were generated in the same manner as for the 

ILS approach. All EPs flew the VMC-like approach with each of the experimental display concepts 

except the BRD. The VMC-like approach had the same termination options as the ILS approach (i.e.,

continue to 50 ft, land, or MA). Each run ended either at touchdown or when positive rate of climb was 

established on the MA. The VMC-like approach represented a potential future operation that could be 

enabled by SVS displays.

3.5.4 Training

Evaluation pilots received training prior to data collection to minimize learning effects. Training 

included a one-hour briefing regarding the characteristics of the experiment along with a description of all 

display concepts, evaluation approaches, and subjective and objective measures. Next, EPs received 

approximately 2.5 hours of training using a desktop simulator where all test conditions were practiced. 

Pilots could elect to retry test conditions until they were completely comfortable with all aspects of the 

experiment. In general, EPs did not elect to retry training runs and were able to demonstrate acceptable 

performance. Acceptable performance was defined as CDI deflections within 1.75 dots for the ILS 

approach and controlling the velocity vector to be within approximately 5 degrees combined displacement 

of the guidance cue for the HITS display concepts (i.e., PBG and SVS). For the SCFD display concept, 

acceptable performance was approximately 5 degrees displacement between the guidance cue and 

reference waterline for both pitch and roll. During the portions of the BRD evaluations that did not have 

ILS data (i.e., VFR and initial segment of the ILS approaches) acceptable altitude performance was +/-

200 ft. Airspeed control within +/-10 kts was acceptable for all training scenarios. In addition, 

acceptable performance included pilot awareness and comfort with the displays, test conditions, and 

approaches. Once the training simulation was completed, EPs received approximately 1.2 hours of flight 

training in the C206 to become familiar with the aircraft, flight-crew protocol, research apparatus and 

scenarios. A specific set of training scenarios was selected to provide EPs with exposure to both types of 

advanced guidance symbologies (i.e., SCFD and PBG), baseline round dials, and both research approach

flight paths (i.e., the VFR/VMC-like and ILS). Other general airwork was also conducted during the 

flight. No vision-restriction devices were used for the flight training sessions.

3.5.5 Operations

For all scenarios, EPs were isolated from actual air traffic control (ATC) communication and asked 

to monitor a simulated ATC channel that would play pre-recorded ATC messages based on aircraft 

location. They answered questions about simulated air traffic and provided other qualitative data (i.e., 

situation awareness, workload, display readability, terrain awareness) immediately after each run. (An 

evaluation of the simulated air traffic responses indicated no usable data to differentiate between 

approaches, pilot types, or display concepts and is not reported herein. Even though pilots were briefed to 

expect questions about ATC instructions to other aircraft, that type of mental recall was found to be very 

different from what pilots are accustomed to and they simply did not recall any ATC instructions for other 

aircraft.) Isolating the EPs from actual ATC communications provided a more uniform workload. For 

the simulated-IMC scenarios, the EPs wore an IFR training visor. No vision restriction was used for the 

VFR approach.

During each approach, the SPs would establish the aircraft approximately 1.0 nm from the starting

waypoint (i.e., Waypoint A for the VFR/VMC-like approaches or Waypoint B for the ILS approach in 

Figure 8) on the appropriate heading, altitude, and speed, at which time control would be transferred to 

the EP. The EP would then complete the approach setup and perform the evaluation approach. Once the 

approach was completed, the SP took control of the aircraft and the EP answered the post-run 

questionnaires using an electronic questionnaire tool implemented on a pocket personal computer 

(described in Appendix C).
26

The order of test condition presentation was randomized (subject to some operational efficiencies) to 

further minimize learning, fatigue, and environmental effects. Generally, data collection was performed 

during two approximately 1.2-hour flights to avoid fatigue, though a few pilots completed the entire test 

matrix in one 2-hour flight. EP participation in the flight test usually spanned two days with training 

occurring on the first day. A substantial effort was made to conduct the formal research evaluations 

within four hours after sunrise in order to reduce atmospheric variability effects within the resulting data. 

Most of the data acquired during this flight test fell within this timeframe. All flight operations were 

conducted in clear day conditions. There were no wind limitations established for the data collection 

operations.

3.6 Independent Variables

Several independent variables were used for this test. Display concept, approach, and pilot type 

were explicitly controlled in the test matrix. The test condition (TC) variable was based on combinations 

of approach, display concept and pilot type. A summary listing of the independent variables of the study

is provided in Table 2.

Table 2: Independent Variables Used for This Experiment

# Name Levels

1 Display Concept (4 total) BRD, SCFD, PBG, or SVS

2 Approach (3 total) VFR, ILS, or VMC-like approaches

3 Pilot Type (3 total) VFR, IFR, or H-IFR

4 Test Condition (13 total) VFR approach with BRD

VFR approach with SCFD

VFR approach with PBG

VFR approach with SVS

ILS approach with BRD for IFR and H-IFR pilots only

ILS approach with SCFD

ILS approach with PBG

ILS approach with SVS

VMC-like approach with SCFD

VMC-like approach with PBG

VMC-like approach with SVS

ILS approach with SVS for VFR pilots only

VMC-like approach with SVS 

3.7 Test Matrix

The test matrix for approach, display concept, visibility, and pilot type is provided in Table 3. Note 

that the BRD display concept was not included in the test matrix for the VMC-like approach because of 

anticipated very poor performance that could involve safety of flight concerns. The poor performance 

was expected as a result of the piloting strategy required for traditional IMC operations with the BRD 

display concept. This method uses selected headings to fly, evaluated for tracking performance over at 

least 10 to 20 seconds before updating to manage lateral error. The relatively rapid pace for the VMClike approach would have been highly challenging for the BRD display concept, especially for the VFR 

pilots. In Table 3, “VMC” indicates that no vision restriction device was used to restrict the EP’s 
27

visibility. Similarly, “Sim IMC” indicates that a standard IFR training vision restriction device was used 

by the EPs.

The VFR and IFR pilots evaluated the BRD display concept for the ILS approach twice while the HIFR pilots evaluated the BRD display concept for the VFR traffic pattern twice. The repeat evaluations of 

these test conditions were included to provide more data for a multivariate discriminant analysis 

technique employed and described in Reference 18. Overall, each EP flew 12 total data-collection runs to 

complete the run matrix (four VFR traffic patterns, four ILS approaches, three VMC-like approaches, 1 

repeat).

Table 3: Test Matrix

Ident 

#

Approach Display 

Concept Visibility Pilot Type

1 VFR BRD VMC

VFR Pilots

2 VFR SCFD VMC

3 VFR PBG VMC

4 VFR SVS VMC

5 ILS BRD Sim IMC

6 ILS SCFD Sim IMC

7 ILS PBG Sim IMC

8 ILS SVS Sim IMC

9 VMC-Like SCFD Sim IMC

10 VMC-Like PBG Sim IMC

11 VMC-Like SVS Sim IMC

12 VFR BRD VMC

IFR Pilots

13 VFR SCFD VMC

14 VFR PBG VMC

15 VFR SVS VMC

16 ILS BRD Sim IMC

17 ILS SCFD Sim IMC

18 ILS PBG Sim IMC

19 ILS SVS Sim IMC

20 VMC-Like SCFD Sim IMC

21 VMC-Like PBG Sim IMC

22 VMC-Like SVS Sim IMC

23 VFR BRD VMC

H-IFR 

Pilots

24 VFR SCFD VMC

25 VFR PBG VMC

26 VFR SVS VMC

27 ILS BRD Sim IMC

28 ILS SCFD Sim IMC

29 ILS PBG Sim IMC

30 ILS SVS Sim IMC

31 VMC-Like SCFD Sim IMC

32 VMC-Like PBG Sim IMC

33 VMC-Like SVS Sim IMC

The TC variable was composed of combinations of specific approaches, display concepts, and pilot 

types, as defined in Table 4. The purpose of the TC variable was to provide explicit comparisons across 
28

approaches and provide key reference conditions in the data. One reference condition, TC 5, was defined 

by the IFR and H-IFR pilots flying the ILS approach with the BRD display concept. This condition 

represents the current standard for operations in IMC. As such, the VFR pilots’ data are not included in 

TC 5. In addition, to draw conclusions regarding the ability of the VFR pilots to pilot an aircraft with 

SVS displays in IMC, TC 12 and TC 13 were created for the ILS and VMC-like approaches, respectively. 

Note that the data for the VFR pilots’ evaluations of the SVS display concept for the ILS and VMC-like 

approaches are also included in TCs 8 and 11.

Table 4: Definition of Test Condition Variable.

Test 

Condition Approach Display Concept Visibility Pilot Types

1 VFR BRD VMC VFR, IFR, H-IFR

2 VFR SCFD VMC VFR, IFR, H-IFR

3 VFR PBG VMC VFR, IFR, H-IFR

4 VFR SVS VMC VFR, IFR, H-IFR

5 ILS BRD Sim IMC IFR, H-IFR

6 ILS SCFD Sim IMC VFR, IFR, H-IFR

7 ILS PBG Sim IMC VFR, IFR, H-IFR

8 ILS SVS Sim IMC VFR, IFR, H-IFR

9 VMC-Like SCFD Sim IMC VFR, IFR, H-IFR

10 VMC-Like PBG Sim IMC VFR, IFR, H-IFR

11 VMC-Like SVS Sim IMC VFR, IFR, H-IFR

12 ILS SVS Sim IMC VFR

13 VMC-Like SVS Sim IMC VFR

3.8 Dependent Measures

Quantitative and qualitative dependent measures were collected in this study. These measures are 

explained below.

3.8.1 Qualitative Measures

Post-run questionnaire results analyzed in this report consisted of the 3-dimensional Situation 

Awareness Rating Technique (SART, see Reference 19), 6-dimensional Task Load Index (TLX, see 

Reference 20), Display Readability Rating (DRR, see Reference 21) and Terrain Awareness (TA) 

question. All of the post-run questionnaires are included in Appendix C. The SART questionnaire was 

applied to measure the EP’s perceived ability to understand the current situation (i.e., situation 

awareness). All references to SA herein are from SART results.

The SART results are calculated by adding the results from the responses of the perceived supply of 

attentional resources and understanding of the situation questions and then subtracting the response from 

the demand on attentional resources question. However, an unconventional implementation of SART was 

employed using a 0 to 100 scale rather than the nominal 7-pt Likert scale. In this implementation, the 

three SART components (supply, understanding, and demand) were rated 0 to 100 by each pilot with 100 

representing “high” and a score of 0 representing “low.” As such, the SART scores can vary from a 
29

maximum of 200 to -100. A SART score of 200 would indicate that a pilot had the maximum perceived 

supply of attentional resources and understanding of the situation while also experiencing virtually no

demand on attentional resources. A SART score of -100 would indicate the pilot had a maximum 

perceived demand on attentional resources and the least possible supply of attentional resources and 

understanding of the situation.

Pilot subjective workload was assessed through use of the TLX scale. Pair-comparisons of the TLX 

factors were not administered to the EPs to obtain an individualized workload metric; instead, the TLX

results were calculated by determining the average response to the six questions: 1) Mental demand; 2) 

Physical demand; 3) Temporal demand; 4) Performance; 5) Effort; 6) Frustration level. See Appendix C 

for details. TLX results can vary from a maximum of 100, that would indicate the highest (worst) results 

for all 6 questions, to a minimum of zero (best), which would result from the lowest response for all 6 

questions.

The DRR scale was applied to assess the workload associated with obtaining information from the 

various research displays. Pilots were provided a definition of desired and adequate performance for the 

DRR scale and self-rated their own performance. Ratings are generated through the use of a decision 

tree. It requires the EPs to answer yes or no to series of questions that ultimately create a phrase 

association that best describes the pilot’s perception of a specific condition in the test matrix. The DRR 

ratings begin from a maximum (worst) of 10 which would indicate that “mandatory improvements to the 

display are required” combined with the statement that “the display symbology cannot be used for the 

required task”. The best rating,a DRR rating of 1, would indicate that the display was “satisfactory 

without improvement” combined with the statement that “pilot compensation was not a factor for desired 

performance”. The DRR is a slightly different form of pilot workload measure than TLX, and attempts to 

assess the amount of mental workload expended to absorb flight information from the display device to 

meet desired and/or adequate performance. In this sense, “absorb” means to comprehend the information 

being supplied and transform that information into useable situation awareness and cognitive processing. 

A specific question regarding terrain awareness was also included to acquire data specific to this 

element of SA. The TA results was based on the EP’s overall understanding of the terrain environment 

they were operating within. Terrain Awareness results could vary from a maximum (best) of 100, that 

would indicate that the EP had the best perceived understanding of the terrain environment, down to a 

minimum of zero (worst) that would indicate the least possible terrain awareness.

3.8.2 Quantitative Measures

Quantitative data recorded included flight path and airspeed errors. Parameters included: linear 

flight path errors (lateral and vertical in ft), angular flight path errors (localizer and glideslope in dots of 

deflection), and indicated airspeed errors (in knots). Quantitative error parameters are discussed for 

combined Segments 2 and 3 as well as Segment 3 alone (see Figure 8 for definitions of Segments 2 and 

3). Indicated airspeed errors are discussed separately for Segments 2 and 3 due to data analysis 

limitations. Root-mean-square (RMS) values of these error variables are included in the analyses 

performed and reported herein.

Linear lateral and vertical flight path and track-angle errors were calculated using a technique that 

determined the shortest lateral distance from the aircraft to the flight path centerline. The intersection of 

this line with the desired flight path was then used to establish reference conditions to generate error 

parameters. The linear errors were combined with range to the runway touchdown point to calculate 

glideslope errors, and the range to a point 1,000 ft beyond the far (departure end) end of Runway 24 for 

localizer errors. Proceeding in this manner enabled localizer and glideslope errors to be calculated, 

displayed, and analyzed the ILS approach and calculated and analyzed for the VFR and VMC-like 

approaches. Note that the localizer and glideslope error was only displayed for the ILS approach.
30

For the simulated-IMC approaches (i.e., ILS and VMC-like) the FAA IFR Practical Test Standards 

(PTS) were applied. To earn an instrument rating, pilots must demonstrate the ability to maintain flight 

path and airspeed errors within criteria outlined in the PTS. The accepted and desired IMC performance 

conditions were based on the FAA PTS criteria defined in Reference 22. The resulting “IFR PTS 

boundary” used for both the ILS and the VMC-like approaches began at a specified entry gate and ended 

at the DH. It was bounded by 3⁄4-scale maximum deflections (1.5 dots) of localizer and glideslope

indicators, and +/-10 kts of IAS error. Note that the instantaneous required airspeed change during the 

glideslope intercept from 100 +/-10 kts to 90 kts to +/-10 kts resulted in a potential limitation of the 

maximum amount of time pilots’ could achieve PTS performance. If a pilot was within the PTS IAS limit 

during the localizer tracking segment, but with an IAS above 100 kts, then at the point of glideslope 

intercept he would no longer be within the PTS criteria until he reduced IAS to be below 100 kts. It is 

considered that this momentary IAS exceedance would be acceptable performance during an IFR 

checkride. However, for the analysis performed herein the amount of time required to bring IAS back to 

within +/-10 kts error detracts from the time within PTS criteria. This artifact would also apply to the 

VMC-Like approaches since a speed change is required for that maneuver too. While exceeding the IFR 

PTS does not directly imply an imminent accident, boundary violation does provide an indication of an 

unstable approach and an increased accident risk or exposure. PTS results for all three approaches are 

discussed as the percentage of time the evaluation pilots were able to satisfy all three parameters 

simultaneously.

Linear lateral flight path error analysis for the various display concepts and approaches was valuable 

in support of RNP-style approaches that are currently being developed and used within the NAS. For 

RNP operations, the aircraft, avionics, and crew must be certified by the FAA to be capable of remaining 

within linear lateral limits of the centerline of the desired flight path 95% of the time. The required total 

system performance can be achieved by various combinations of flight management systems, navigation 

sensors, flight guidance systems, cockpit displays and other equipment, together with monitoring and 

alerting capabilities and additional training, which distinguishes performance-based navigation from other 

navigation methods. RNP aircraft certification typically specifies an accuracy level, such as 0.10 nm or 

0.30 nm, at which the aircraft can navigate (Reference 23). This is the combination of airframe and 

autoflight system for coupled operations. For manual operations, the crew is allocated part of the total 

system error budget. This is different compared to angularly-defined airspace limits consistent with most 

of the traditional IMC approach operations. For this report, a flight technical error criterion of 0.1 nm is 

used as a reference to compare and contrast results from the different display concepts, approaches, and 

pilot types for manually flown approaches. The results discussed herein only deal with the ability of the 

pilot to interact with the display concepts and minimize flight path error (i.e., only accounting for what is 

known as flight technical error) and does not account for errors within the navigational system (e.g.,

GPS). Dynamic taxi testing along the runway centerline at KROA showed that the installed Air Data, 

Attitude, and Heading Reference System (ADAHRS) equipment performance exhibited errors on the 

order of 10 to 20 ft (i.e., 0.0016 to 0.0033 nm). These errors were considered negligible for the in-flight 

results discussed herein

For the ILS Approach, Segment 3 extended from the glideslope intercept point at 8.4 nm and ended

at the decision height (i.e., 200 ft AGL, approximately 0.66 nm from the glideslope intercept point). For 

the advanced display concepts, guidance was provided throughout the entire ILS approach. However, for 

the BRD display concept, evaluation pilots flew a specified heading and altitude until intercepting the 

localizer from the start point, then flew a specified altitude while tracking the localizer to intercept the 

glideslope. The ILS approach was completed with the EPs following the localizer and glideslope signals 

to decision height. While all ILS evaluations were initiated from the same location, the guidance for the 

BRD display concept was not consistent for the entire approach and did not have positive lateral and 

vertical guidance throughout the localizer intercept and localizer tracking phases of the approach

(Segment 2). To compensate for the inconsistent guidance of the BRD display concept, comparisons of 
31

the flight path error data from the BRD display concept were limited to the localizer and glideslope

tracking phases of the ILS approach (Segment 3).

If, during a run, the localizer and glideslope indicators were simultaneously at their upper limits and 

the path error was increasing, the EP was deemed to have lost all flight path awareness and the TE would 

terminate the run. The SP was prepared to take control immediately in any situation involving safety-offlight concerns; however, no runs were aborted by the SP due to safety-of-flight issues during the test.

4.0 Data Analyses and Results

Results are presented in this section for subjective situation awareness, workload, display readability, 

and terrain awareness. Quantitative results include percent of time within the PTS, localizer errors, 

glideslope errors, indicated airspeed errors, lateral path errors, and vertical path errors,. The advanced 

display concepts provide guidance to minimize linear flight path errors. In order to best compare pilot 

performance when using the advanced display concepts linear path errors analyses are used. Linear 

pathway error analyses also provides the ability to assess performance compared to RNP criteria that 

could be associated with future advanced landing approaches. However, linear path error analyses 

adversely penalize displays that minimize angular pathway errors, such as the BRD with the localizer and 

glideslope errors. For this report both linear and angular pathway errors are employed. PTS criteria are 

employed for the localizer, glideslope and IAS error analyses. Required Navigation Performance criteria 

of +/-0.1 nm and +/-100 ft are used for the linear lateral and vertical error analyses, respectively. 

For all parameters, analyses are performed separately for each approach (i.e., VFR, ILS, and VMClike approaches) using all of the pilot group’s data. This enables a clear view of the display concepts and 

pilot types without considering operational task (i.e., approach types) effects. Then, in an effort to 

describe the results within a framework aligned with current operations, the VFR pilots’ data are removed 

from the ILS approach. To provide increased statistical power to identify effects of display concept, 

approach, and pilot type, data for the IMC approaches (i.e., ILS and VMC-like approaches) are analyzed 

together. To provide a direct comparison of the advanced display concepts and approaches with current 

IMC and VMC operations, the test condition analyses are provided. Recall that the test condition 

independent variable is a combination of approach and display concept (i.e., VFR approach + BRD, etc.).

During the course of the flight test experiment, one run had to be terminated due to the pilot losing 

complete awareness of the desired flight path. The test condition for this run was for a VFR pilot flying 

the ILS approach with the BRD display concept. For this ILS approach run, the pilot was able to 

maneuver near the desired flight path for several minutes and make some progress through the localizer 

intercept and localizer tracking phase of the approach. When the added workload of intercepting and 

tracking the glideslope was added, the pilot was apparently overloaded and never began the descent. The 

run was terminated at that time. The pilot had difficulties from the beginning of the approach with 

consistently poor performance up to the point of termination at approximately 2.5 nm from touchdown. 

The results were consistent throughout this ILS approach attempt and were included in the data.

Analyses of variance (ANOVAs) were conducted using the SPSSTM software package to test for 

statistical significance among the independent variables for the different measures. Results due to the 

independent variables are referred to as main effects and are differentiated from those resulting from 

interactions between the independent variables. In cases where the statistical significance factor (p) was 

less than 0.05, the independent variable differences were considered to be significant for the measure 

being evaluated. For cases where “p” was less than 0.01, differences were considered to be highly 

significant. Student-Newman-Keuls (SNK) post-hoc analysis results were used when “p” was less than 

0.05 and there were more than 2 items being compared. SNK analysis provides subsets of the results 

based on their statistical relationship to each other. Results in the same subset are considered to not be 

statistically different from others in that subset. Those in different subsets are considered to be 
32

statistically different. For the ordinal DRR data, a descriptive analysis of the medians, maximums, 

minimums, and percentiles was performed. 

4.1 Situation Awareness

SART results for the four display concepts and three approaches are presented in Figure 9 along with 

Table 5. Individual SART results for each evaluation are provided in Appendix D.

Table 5: SART Descriptive Statistics (all Pilots)

Display 

Concept Approach Mean StandardDeviation

Number of 

Data Points

BRD VFR 104 54.3 24

ILS 32.6 49.9 29

Total 67.5 60.1 53

SCFD VFR 78.2 54.1 19

ILS 75.8 33.9 18

VMC-Like 62.4 55.0 18

Total 72.3 48.4 55

PBG VFR 115 45.6 18

ILS 83.1 46.6 17

VMC-Like 87.6 46.5 19

Total 95.3 47.5 54

SVS VFR 113 48.5 19

ILS 114 53.6 21

VMC-Like 108 44.8 19

Total 112 48.5 59

Total VFR 101 52.0 75

ILS 78.5 54.0 74

VMC-Like 86.6 51.5 56

Total 88.8 53.2 205
33

Figure 9: SART Results, All Display Concepts, All Pilots and All Approaches

4.1.1 VFR approach

For the VFR approaches, EPs were instructed to navigate the aircraft using outside visual cues and to 

cross check with the guidance when it was provided by the advanced displays. An ANOVA conducted on 

the SART data for this approach for the main effects of display concept and pilot type resulted in no 

significant (p>0.05) results. VFR evaluations of the PBG and SVS display concepts showed trends 

toward increased SA, though not significant, compared to the BRD results. The pilots’ favorable SA 

ratings were most likely due to the presence of the HITS symbology, even though they were primarily 

performing a VFR task. Evaluation pilots were able to cross check their VFR-style navigation with the 

HITS information and easily confirm their position. In the case of the SCFD display concept, the 

evaluation pilots were unable to easily confirm their relative position with respect to the desired path

since flight director commands are a function of both track-angle and lateral offset errors. As a result, a 

given roll command did not provide intuitive direct indication of how far pilots were away from the 

desired flight path, leading to poorer ratings for the SCFD for the VFR approach compared to the other 

display concepts, including the BRD.

4.1.2 ILS approach, all pilots

An ANOVA conducted on the SART results for the main effects of pilot type and display concept 

for the ILS approach revealed that display concept (F(3,85)=22.4, p<0.01) was highly significant but was 

not significant for pilot type (p>0.05). There were no significant interactions between the main effects. 

A Student-Newman-Keuls (SNK) post-hoc analysis on the SART ratings resulted in the three unique 

subsets shown below in Table 6. The BRD display concept was rated as providing the least SA of the 

four display concepts tested. The middle subset was comprised of the SCFD and the PBG display 

concepts. The SCFD and PBG display concepts were rated as providing improved SA compared to the 

BRD display concept but there were no significant differences between these two display concepts. The 

0

50

100

150

200

VFR ILS VMC-Like

BRD

SCFD

PBG

SVS SART

Approach
34

significant increase in SA for the SCFD and PBG display concepts compared to the baseline round-dial 

concept is most likely due to the presence of the moving map ND and/or integrated PFD compared to the 

separate gauges of the baseline round-dial display concept. The SVS display concept was rated as 

providing the highest SA among the concepts tested. The significant difference in SA between the SVS 

and the PBG display concepts can be attributed to the addition of SVS terrain since this was the only 

difference between these two concepts. 

Situation Awareness results for the SVS concept used to fly the ILS approach were higher than those 

for the BRD display concept for the ILS approach. This result may reflect the intuitive integration of the 

outside environment, as provided by SVS terrain, with the desired flight path and aircraft state data.

Table 7 provides SART results for the three pilot groups for the ILS approach for each of the display 

concepts along with the mean for all display concepts. These data are presented to show that SA from the 

three pilots groups had some variability, although it was not significant. Overall, the H-IFR pilot group 

reported higher SA than the other two pilot groups, which may be attributed to the H-IFR pilots' order-ofmagnitude higher amount of flying time, as presented in Table 1. IFR pilots, however, reported lower SA

than the VFR pilots for three of the four display concepts evaluated, even though they had received the 

training required to acquire an IFR rating. An analysis of the constituent elements of SART for all 

display concepts for the ILS approach revealed that the VFR pilots reported significantly higher results 

(F(2,85)=4.3, p=0.02) for the 3

rd SA question (understanding of the situation) for the BRD display 

concept than the IFR pilots and were included in the subset with the H-IFR pilots. There were no other 

significant differences for the other SART questions for all the other display concepts (i.e., p>0.05). 

While the VFR pilots had not acquired the training to become IFR pilots, for all of the display concepts 

they reported SA results that were an average of 10 SA units higher than the IFR pilots as a group (but not 

statistically significant). All pilot groups exhibited similar trends in their SART results for the different 

display concepts with the BRD being the worst and SVS display the best and the SCFD and PBG display 

concepts in between. The bias in the VFR pilots’ SART results needs to be considered when drawing 

conclusions that compare pilot groups. An example of comparisons across pilot groups is provided

during the test condition analysis shown in this report.

Table 6: Mean SART SNK Results for all Pilots, ILS Approach

Display 

Concept

Number of 

Data Points

Subset

1 2 3

BRD 29 32.6

SCFD 18 75.8

PBG 17 83.1

SVS 21 113.5

Significance 1.0 0.5 1.0

Table 7: Mean SART SNK Results for All Pilots for All Display Concepts, ILS Approach

EP-type

Number 

of data 

points SART

SART

BRD

SART

SCFD

SART

PBG

SART

SVS

IFR 30 58.7 14.1 69.8 76.4 100.5

VFR 29 69.2 32.3 66.0 95.0 114.2

H-IFR 26 90.1 62.3 91.7 76.8 127.9

Significance 0.26
35

4.1.3 VMC-Like approach

An analysis of the VMC-like approach data was performed to determine the effect of display concept 

and pilot type for this advanced IMC approach. An ANOVA revealed that display concept was highly 

significant (F(2,56)=5.5, p<0.01) for SART ratings but pilot type was not significant (p>0.05) There 

were no significant interactions between display concept and pilot type. An SNK post-hoc analysis on 

SA resulted in two overlapping subsets as shown in Table 8. The SVS display provided significantly 

more SA than the SCFD but there were no differences between the PBG and SVS displays. The SA 

improvements with the SVS display concept over the SCFD were most likely due to the combination of 

HITS guidance symbology and presence of SVS terrain. 

Table 8: Mean SART SNK Results for All Pilots for All Display Concepts, VMC-Like Approach

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 18 62.4

PBG 19 87.6 87.6

SVS 19 108.4

Significance 0.08 0.15

4.1.4 ILS approach, without VFR pilot data

An analysis of the ILS approach data without the low-time VFR pilots’ data was performed to create 

a baseline reflecting current operations. An ANOVA conducted on SA for the IFR and H-IFR pilot data 

revealed that the main effect on SA was statistically significant (F(1,56)=6.247, p<0.017) for pilot type 

and highly significant (F(3,56)=12.475, p<0.01) for display concept. There were no significant 

interactions between pilot type and display concept. The IFR-rated pilots reported significantly lower SA

than the high-time pilots for all display concepts as shown in Table 9. This result may reflect the total 

number of flight hours for the high-time pilots, their overall level of skill with the BRD display concept, 

and also their ability to adapt to the advanced display concepts, resulting in greater overall situation 

awareness. An SNK post-hoc analysis on the SA revealed three unique subsets for display concept as 

shown below in Table 10. The BRD display concept provided the pilots with significantly lower SA than 

all other concepts tested as indicated in Figure 9. The SCFD and PBG display concepts were rated as 

providing significantly better SA than the BRD display concept, but there were no significant differences 

among the SCFD and PBG display concepts. The SVS concept provided the pilots with significantly 

higher SA than all other concepts tested, and this result was most likely due to the presence of the SVS 

terrain.

Table 9: Mean SART results for IFR and H-IFR Pilots for the ILS Approach

Pilot 

Type

Number of 

Data Points Mean

IFR 30 63.7

H-IFR 26 90.9
36

Table 10: Mean SART SNK Results for Display Concept for IFR and H-IFR Pilots for the ILS 

Approach

Display 

Concept

Number of 

Data Points

Subset

1 2 3

BRD 18 32.8

PBG 11 76.6

SCFD 12 80.8

SVS 15 113.3

Significance 1.000 0.77 1.000

4.1.5 ILS and VMC-Like approach analysis, all pilots, advanced displays

For the VMC-like approach, no data were acquired for the BRD display concept, as discussed 

previously. An ANOVA was conducted on the main factors of advanced display concepts (PBG, SCFD, 

SVS) and approach (ILS, VMC-like) for the SART ratings. Results indicate that display concept was 

highly significant (F(2,112)=10.851, p<0.01) and approach was not significant (p>0.05). Post-hoc 

analysis, shown in Table 11, revealed two unique subsets for the SA results. As shown in Figure 9, the 

SVS display provided significantly better SA than the PBG display due to the addition of terrain on the 

SVS display. Recall that the only difference between the SVS and PBG display concepts was the SVS 

terrain. There were no significant SA differences between the PBG and SCFD concepts.

Overall, the effect of ILS or VMC-like approach was fairly small for the PBG and SVS display 

concepts as compared to the SCFD display concept as shown in Figure 9. For the VMC-like approach, 

SART results for the SCFD, PBG, and SVS display concepts were as good or better than those for the 

BRD display concept for the ILS approach without the VFR pilots’ data. In addition, SA results for the 

SVS display concept as shown in Figure 9 were similar for all evaluations (i.e., VFR, ILS and VMC-like).

Table 11: Mean SART SNK Results for Advanced Display Concepts for All Pilots for ILS and 

VMC-Like Approach

Display 

Concept 

Number of 

Data Points

Subset

1 2

SCFD 36 69.1

PBG 36 85.5

SVS 40 111.1

Significance 0.086 1.000

4.1.6 Test condition analysis

A subsequent analysis of the SART data was performed based on test condition (TC), with post-hoc 

results presented in Table 12. An ANOVA performed on the SART data indicated that TC was highly 

significant (F(12,221)=5.585, p<0.01). In Table 12, three subsets were identified. TC 5, the ILS 

approach flown with the BRD concept, was rated as providing significantly less SA than the VFR 

evaluations of all display concepts. The SVS concepts (regardless of which approach was flown) were 

rated as providing the highest SA and were not found to be statistically different from TC 1 (i.e., VFR 

evaluation of the BRD display concept). The VMC-like evaluation of the SCFD display concept (TC 9)
37

was rated as providing significantly less SA for the VMC-like approach than the VFR evaluations of the 

other display concepts. 

Table 12: Mean SART SNK Results for Test Condition

Test Condition Name Test 

Condition

Number of 

Data Points

Subset

1 2 3

ILS Approach + BRD (no VFR pilots) 5 18 32.8

VMC-like Approach + SCFD 9 18 62.4 62.4

ILS Approach + SCFD 6 18 75.8 75.8

VFR Approach + SCFD 2 19 78.2 78.2

ILS Approach + PBG 7 17 83.1 83.1

VMC-like Approach + PBG 10 19 87.6 87.6

VFR Approach + BRD 1 24 104.0 104.0

VMC-like Approach + SVS 11 19 108.4 108.4

VFR Approach + SVS 4 19 112.6

ILS Approach + SVS 8 21 113.5

ILS Approach + SVS (only VFR pilots) 12 6 114.2

VFR Approach + PBG 3 18 114.9

VMC-like Approach + SVS (only VFR pilots) 13 6 115.5

Significance 0.064 0.065 0.315
38

4.2 Subjective Workload

Subjective workload results for the four display concepts and three approaches for all pilots are 

presented in Figure 10 and Table 13 and were generated through use of the TLX questionnaire and 

calculation process described in Appendix C. (Error bars in Figure 10 represent one Standard Deviation 

(SD).) Individual TLX results for each evaluation are provided in Appendix D.

Table 13: TLX Descriptive Statistics (all Pilots)

Display 

Concept Approach Mean StandardDeviation

Number 

of Data 

Points

BRD VFR 27.2 14.1 24

ILS 46.2 20.4 29

Total 36.1 19.1 53

SCFD VFR 36.4 16.1 19

ILS 36.1 17.1 18

VMC-Like 39.3 19.4 18

Total 37.3 17.3 55

PBG VFR 27.3 12.4 18

ILS 28.9 16.0 17

VMC-Like 34.8 15.9 19

Total 30.5 14.9 54

SVS VFR 27.8 14.8 19

ILS 25.4 15.7 21

VMC-Like 29.7 14.4 19

Total 27.6 14.9 59

Total VFR 29.7 14.9 75

ILS 33.8 18.3 74

VMC-Like 34.5 16.8 56

Total 33.1 17.3 221
39

Figure 10: TLX Results, All Display Concepts, All Pilots, and All Approaches

4.2.1 VFR approach

Results for all of the display concepts indicated low workload for the VFR approach. This result was

expected since the VFR approach was conducted without the vision restriction device in clear day VMC. 

An ANOVA conducted on the TLX results for the VFR approach for the main effects of display concept 

and pilot type resulted in no significant (p>0.05) results. 

4.2.2 ILS approach, all pilots

An ANOVA conducted on the TLX results for all pilots and display concepts for the ILS approach

revealed that the main effect was highly significant for display concept (F(3,85)=18.3, p<0.01) but not for 

pilot type (p>0.05). A SNK post-hoc analysis on the TLX data resulted in the three unique subsets for 

display concept shown below in Table 14. For the ILS approach for all pilots, workload was significantly 

decreased by 10 units for the SCFD display concept compared to the BRD display concept most likely 

reflecting the improvements due to the moving map and the integrated single-cue flight director on the 

PFD display. Workload was also significantly reduced by 7.2 units for the PBG display concept 

compared to the SCFD display concept reflecting the intuitive guidance provided by the HITS and flight 

path guidance symbology. Incorporation of the terrain on the SVS display concept provided a small 3.5

unit decrease in workload as compared to the PBG display concept. 

0

10

20

30

40

50

60

70

80

VFR ILS VMC-Like

BRD

SCFD

PBG

SVS TLX

Approach
40

Table 14: Mean TLX SNK Results for Display Concept for all Pilots and Display Concepts, ILS 

Approach

Display 

concept

Number of 

Data Points

Subset

1 2 3

SVS 21 25.4

PBG 17 28.9

SCFD 18 36.1

BRD 29 46.2

Significance 0.302 1.000 1.000

4.2.3 VMC-Like approach

An analysis of the VMC-like approach data was performed to determine the effect of display concept 

and pilot type for this advanced IMC approach. An ANOVA conducted on the TLX results for all pilots 

and display concepts revealed that the main effect was highly significant for display concept 

(F(2,56)=5.5, p<0.01) and also for pilot type (F(2,56)=8.5, p<0.01). There were no significant 

interactions between the main effects. An SNK post-hoc analysis on the TLX results revealed two

overlapping subsets for display concept and two unique subsets for pilot type, as shown in Tables 15 and 

16. While the PBG guidance symbology decreased workload by approximately 4.5 units compared to the 

SCFD for this approach, this improvement was not statistically significant. The effect of SVS terrain also 

decreased workload compared to the PBG display concept by a similar amount. The decreased workload

for the SVS display, due to the combination of HITS guidance symbology and SVS terrain, created a 

significant 9.4 unit reduction compared to the SCFD display concept. This can be seen in Table 15,

which shows that the SVS display concept was in the lower (best) workload subset while the SCFD 

display concept was in the higher (worst) subset. The PBG display concept was similar to both and 

included in each subset. It is believed that the higher demand of the VMC-like approach contributed to 

the significant results for pilot type as compared to the ILS approach. Overall, the VFR pilots provided 

significantly lower TLX results than the IFR and H-IFR pilots. It is speculated that the VFR pilots 

perceived the workload of the VMC-like approach flown with the advanced display concepts to be closer 

to the VFR approach, which is the only approach they are accustomed to flying. 

Table 15: Mean TLX SNK Results for the advanced Display Concepts for the VMC-Like 

Approach

Display 

Concept

Number of 

Data 

Points

Subset

1 2

SVS 19 29.7

PBG 19 34.8 34.8

SCFD 18 39.3

Significance 0.075 0.112
41

Table 16: Mean TLX SNK Results for Pilot type for the VMC-Like Approach

Pilot Type Number of 

Data Points

Subset

1 2

VFR 18 28.1

H-IFR 20 35.8

IFR 18 39.5

Significance 1.000 0.192

4.2.4 ILS approach without VFR pilot data

Results from an ANOVA conducted on the TLX data without the VFR pilot data indicated that pilot 

type (F(1,56)=5.0, p=0.032) was significant and display concept (F(3,56)=6.19, p<0.01) was highly

significant for TLX results for this analysis. The IFR pilots reported significantly higher workload (7 unit 

increase) than the H-IFR pilots. This result reflects the total flight hours for the high-time pilots, their 

overall level of skill with the BRD display concept, and possibly their ability to adapt to the advanced 

display concepts, resulting in lower overall subjective workload. An SNK post-hoc analysis on the TLX 

data for display concept resulted in the two overlapping subsets shown in Table 17. The BRD display 

concept was rated as providing significantly higher workload than the SVS and PBG concepts, but it was 

not significantly different from the SCFD concept. The SCFD was rated as having no significant

workload differences from any of the other display concepts. Pilots appear to more easily follow the 

HITS pathway guidance. While not significant, the addition of the SVS terrain lowered subjective 

workload four units compared to the PBG display concept while also significantly improving SA as noted 

above. 

Table 17: Mean TLX SNK Results for Display Concept for IFR and H-IFR Pilots for all Display 

Concepts for the ILS approach

Display 

Concept

Number of 

Data 

Points

Subset

1 2

SVS 15 27.6

PBG 11 31.6

SCFD 12 36.4 36.4

BRD 18 44.8

Significance 0.15 0.073

4.2.5 ILS and VMC-Like approach analysis, all pilots, advanced displays

For the VMC-like approach, no data were acquired for the BRD display concept as discussed 

previously. An ANOVA was conducted on the data for the three advanced display concepts for the ILS 

and VMC-like approach for all pilot types. Results indicated that display concept was highly significant 

(F(2,112)=8.558, p<0.01) and approach was significant (F(1,112)=5.544, p=0.021) for the workload 

ratings. The effect of display concept and approach are shown in Figure 10, where the SVS display 

provides the lowest workload for both approaches and the workload for the VMC-like approach increases 

for all of the advanced display concepts. Results from the SNK post-hoc analysis are presented in Table 

18. Both the PBG and SVS display concepts significantly reduced workload compared to the SCFD 

display concept, most likely due to the HITS pathway guidance. The workload for the SVS display 

concept was reduced by 4.5 units compared to the PBG display concept. The VMC-like approach
42

increased workload compared to the ILS approach for all advanced displays by 3.2, 5.9, and 4.3 units for 

the SCFD, PBG, and SVS displays, respectively, as shown in Table 13 and Figure 10. 

Table 18: Mean SNK TLX Results for Display Concept for all Pilots for the ILS and VMC-Like 

Approach (Advanced Display Concepts)

Display 

Concept

Number of 

Data Points

Subset

1 2

SVS 40 27.5

PBG 36 32.0

SCFD 36 37.7

Significance 0.068 1.000

4.2.6 Test condition analysis

An analysis was performed based on TC with post-hoc results presented in Table 19. Test Condition

was highly significant for TLX results (F(12,221)=4.172, p<0.01). Four overlapping subsets were formed 

from post-hoc TC analysis. The VFR evaluations of BRD, PBG, and SVS display concepts, the ILS 

evaluations of the PBG and SVS display concepts and the VMC-like evaluation of the SVS display 

concept were rated as providing the lowest workload and contained in subset 1. The ILS approach with 

the BRD display concept (TC 5) was rated as requiring the highest workload. Test Condition 5 grouped 

with the VMC-Like approach evaluations of the PBG and SCFD display concepts, the ILS approach 

evaluation of the SCFD display concept, and the VFR approach evaluation of the SCFD display concept. 

Workload ratings for all of the TCs in the highest workload subset were significantly higher than those in 

the lowest workload subset. These results indicate that for the ILS approach the SVS and PBG display 

concepts (but not the SCFD concept) offer significantly reduced workload compared to the BRD display 

concept. Only the SVS display concept provides significantly reduced workload for the VMC-like 

approach compared to the BRD display concept for the ILS approach (TC 5). For the ILS approach, the 

SVS display concept provides workload as low or lower than for the VFR approach with the BRD display 

concept (TC 1). The advanced display concepts may potentially enable advanced IMC operations, such 

as the VMC-like approach, since their workload results were less than the ILS approach with BRD 

display concept (TC 5). 
43

Table 19: Mean TLX SNK Results for Test Condition

Test Condition Name TC 

#

Number of 

Data Points

Subset

1 2 3 4

ILS Approach + SVS (only VFR pilots) 12 6 19.8

VMC-Like Approach + SVS (only VFR pilots) 13 6 23.7 23.7

ILS Approach + SVS 8 21 25.4 25.4

VFR Approach + BRD 1 24 27.2 27.2 27.2

VFR Approach + PBG 3 18 27.3 27.3 27.3

VFR Approach + SVS 4 19 27.8 27.8 27.8

ILS Approach + PBG 7 17 28.9 28.9 28.9

VMC-Like Approach + SVS 11 19 29.7 29.7 29.7

VMC-Like Approach + PBG 10 19 34.8 34.8 34.8

ILS Approach + SCFD 6 18 36.1 36.1 36.1

VFR Approach + SCFD 2 19 36.4 36.4 36.4

VMC-Like Approach + SCFD 9 18 39.3 39.3

ILS Approach + BRD (no VFR pilots) 5 18 44.8

Significance 0.2 0.1 0.1 0.1

4.3 Display Readability Rating

Display readability rating (DRR) results for the four display concepts and three approaches are 

presented in Figure 11 and Table 20 and were generated through use of the DRR questionnaire in 

Appendix C. Table 20 presents the percentage of the time that pilots selected a specific DRR rating for 

the combinations of display concept and approach. The median and total number of data points are also 

given. Figure 11 presents the cumulative percentage that pilots selected specific DRR ratings for each

display concept for the VFR, ILS, and VMC-Like approaches. Discussion is provided regarding the 

minimum, maximum, and median ratings received for each display concept and test condition. Further 

discussion is provided regarding the range of ratings required to capture at least 60% and 80% of the 

ratings to provide insight into the distribution of the results. Individual DRR results for each evaluation 

are provided in Appendix D. 
44

Table 20: Display Readability Percentage of Time Selected Results (all Pilots)

VFR Approach ILS Approach VMC-L Approach

DRR BRD SCFD PBG SVS BRD SCFD PBG SVS SCFD PBG SVS

1 25.0 15.8 27.8 45.0 10.3 22.2 17.6 57.1 21.1 26.3 41.2

2 33.3 31.6 44.4 35.0 20.7 50.0 47.1 28.6 42.1 42.1 35.3

3 29.2 21.1 22.2 10.0 20.7 11.1 29.4 9.5 5.3 15.8 5.9

4 8.3 26.3 5.6 10.0 17.2 11.1 0.0 0.0 15.8 10.5 11.8

5 4.2 0.0 0.0 0.0 13.8 5.6 0.0 4.8 0.0 5.3 5.9

6 0.0 0.0 0.0 0.0 6.9 0.0 5.9 0.0 10.5 0.0 0.0

7 0.0 0.0 0.0 0.0 3.4 0.0 0.0 0.0 5.3 0.0 0.0

8 0.0 5.3 0.0 0.0 6.9 0.0 0.0 0.0 0.0 0.0 0.0

9 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

10 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Median 2 3 2 2 3 2 2 1 2 2 2

Total 24 19 18 20 29 18 17 21 19 19 17
45

Figure 11: Cumulative Percentage of DRR, All Display Concepts and All Pilots

0

20

40

60

80

100

120

VFR Approach

BRD-VFR

SCFD-VFR

PBG-VFR

SVS-VFR

Cumulative Percent

0

20

40

60

80

100

120

ILS Approach

BRD-ILS

SCFD-ILS

PBG-ILS

SVS-ILS

Cumulative Percent

0

20

40

60

80

100

120

0 1 2 3 4 5 6 7 8 9 10

VMC-Like Approach

SCFD-VMC-L

PBG-VMC-L

SVS-VMC-L

Cumulative Percent

DRR
46

4.3.1 VFR approach, all pilots

For all the display concepts, the best rating received was a 1 “Excellent, Highly Desirable” 

characteristics. The PBG and SVS display concepts received a rating of 3 “Fair, Some Mildly Unpleasant 

Deficiencies” as the worst rating. The worst rating received for the BRD display concept was a 5 

“Moderately Objectionable Deficiencies”. One pilot rated the SCFD as having characteristics of an 8 

“Major Deficiencies”. For the VFR approach, DRR results indicate that all display concepts were rated as 

“satisfactory without improvement” or better (i.e., DRR ratings of 1 or a 2) at least 60% of the time as 

indicated in Figure 11. Display readability rating results for the SVS display concept indicate that it was 

rated as a 1 “Excellent, Highly Desirable” characteristics 45% of the time and either a 1 or a 2 “Good, 

Negligible Deficiencies” 80% of the time. The BRD and PBG display concept results needed to also 

include a rating of 3 “Fair, Some Mildly Unpleasant Deficiencies” to capture at least 80% of the ratings. 

The SCFD display concept results needed to also include ratings of 4 “Minor but Annoying 

Deficiencies” to capture 80% of the ratings. All display concepts, except the SCFD received a median 

DRR rating of 2 which is “good with negligible deficiencies” for this approach. The SCFD display 

concept received a median DRR rating of 3, which is “fair, some mildly unpleasant deficiencies” due to 

frustration involving the inability to differentiate lateral flight path from flight-path track-angle errors. 

4.3.2 ILS approach, all pilots

For all the display concepts, the best rating received was a 1 - “Excellent, Highly Desirable” 

characteristics. The worst rating received for the SCFD and SVS display concepts was a 5 “Moderately 

Objectionable Deficiencies”. The worst rating for the PBG display concept was a 6 “Very Objectionable

But Tolerable Deficiencies”. The BRD display concept received several ratings of 8 “Major 

Deficiencies” for the ILS approach as the worst rating. Display readability results indicate that all display 

concepts, except the BRD, were rated as “satisfactory without improvement” or better (i.e., DRR ratings 

of 1 or a 2) at least 60% of the time as indicated in Figure 11. The BRD display concept was rated as 

having “Minor But Annoying Deficiencies” or better (i.e., DRR ratings of 1 through 4) at least 60% of the 

time and having “Moderately Objectionable Deficiencies” (i.e., DRR ratings of 1 through 5) or better 

80% of the time. The SVS display concept was rated as a 1 having “Excellent, Highly Desirable” 

characteristics 57% of the time and DRR ratings of either a 1 or a 2 80% of the time. The PBG and 

SCFD display concepts were rated as “Fair, Some Mildly Unpleasant Deficiencies” or better 80% of the 

time. The SVS display concept was the only one to achieve 80% of the ratings being either a 1 or a 2. 

The SVS display concept was also the only one to receive a median rating of 1 for the ILS approach. 

Comparing the PBG and SVS display concepts reveals that the effect of the SVS terrain lead to a 40% 

increase in the percentage of DRR 1 ratings. It is believed that the presence of the SVS terrain facilitated 

integration of the other information elements on the display leading to the improved ratings. The BRD 

display concept received a median DRR rating of 3 “Fair, Some Mildly Unpleasant Deficiencies” for the 

ILS approach and the SCFD and PBG display concepts received median ratings of 2. 

4.3.3 VMC-Like approach, all pilots

For all the display concepts, the best rating received was a 1 “Excellent, Highly Desirable” characteristics. 

The worst rating received for the PBG and SVS display concepts was a 5 “Moderately Objectionable 

Deficiencies”. The worst rating for the SCFD display concept was a 7 “Major Deficiencies”. All display 

concepts were rated as a 2 with “Good, Negligible Deficiencies” or better at least 60% of the time as 

shown in Figure 11. Only the SVS and PBG display concepts received ratings of a 3 “Fair, Some Mildly 

Unpleasant Deficiencies” or better 80% of the time. The SCFD display concept received ratings of a 4 

“Minor but Annoying Deficiencies” or better 80% of the time. It is believed that the lack of turn 

anticipation of the SCFD display concept contributed towards the degraded DRR results. Comparing the 
47

PBG with the SVS display DRR results in Figure 11 concept revealed that the effect of SVS terrain 

increased the percentage of DRR 1 ratings by 17%. All display concepts received median ratings of 2 

“Good, Negligible Deficiencies”. 

4.3.4 ILS approach without low-time VFR pilot data

The DRR data were also analyzed for the ILS approach without the VFR pilot data. Table 21 presents 

the percentage of the time pilots selected a specific DRR rating for each display concept for the ILS 

approach without the VFR pilot data (along with the median and total number of data points). The best 

rating received for all four display concepts was a 1 - “Excellent, Highly Desirable” characteristics. The 

worst ratings for the SVS and SCFD display concepts was a 5 “Moderately Objectionable Deficiencies” 

whereas the worst rating for the PBG display concept was a 6 “Very Objectionable But Tolerable 

Deficiencies”. As was illustrated for all the pilots’ data, the effect of SVS terrain contributed to increase 

the percentage of DRR 1 ratings by approximately 40% compared to the PBG display concept. This 

result is a similar result to the data for all of the pilots. However, the SVS display was one DRR 1 rating 

short of receiving a median rating of 1. At least 60% and 80% of the time, the SVS display concept was 

rated as a 2 or better. The PBG display concept required the inclusion of DRR 3 “Fair, Some Mildly 

Unpleasant Deficiencies” results to capture at least 60% and 80% percent of the responses. The SCFD 

was rated as a 2 or better 60% of the time and a 3 or better 80% of the time and considered similar to the 

PBG results. The BRD display concept received the worst ratings and was rated as a 4 “Minor But 

Annoying Deficiencies” or better 60% of the time and a 5 “Moderately Objectionable Deficiencies” or 

better 80% of the time. Results in Table 21 show that all display concepts, except for the BRD, received 

a median DRR rating of 2 “Good, Negligible Deficiencies” while the BRD received a median DRR rating 

of 3 “Fair, Some Mildly Unpleasant Deficiencies”. 

Table 21: Display Readability Results for the ILS Approach, IFR and H-IFR pilots

DRR

ILS Approach

BRD SCFD PBG SVS

1 10.5 16.7 9.1 46.7

2 21.1 58.3 45.5 40.0

3 26.3 8.3 36.4 6.7

4 10.5 8.3 0.0 0.0

5 15.8 8.3 0.0 6.7

6 5.3 0.0 9.1 0.0

7 5.3 0.0 0.0 0.0

8 0.0 0.0 0.0 0.0

9 0.0 0.0 0.0 0.0

10 0.0 0.0 0.0 0.0

Median 3 2 2 2

Total 19 12 11 15
48

4.3.5 ILS and VMC-Like approach analysis, all pilots, advanced displays

Comparing the results for the ILS approach with the results from the VMC-Like approach reveals that 

the best rating received by the SCFD, PBG, and SVS display concepts were identical. The worst rating 

received for the SCFD display concept increased to a 7 (“Major Deficiencies”) from a 5 (“Moderately 

Objectionable Deficiencies”) for the ILS approach. The worst rating received for the PBG display 

concept improved from a 6 (“Very Objectionable But Tolerable Deficiencies”) to a 5 (“Moderately 

Objectionable Deficiencies”) and the worst rating for the SVS display concept remained unchanged at a 

DRR of 5. Comparing the DRR results in Figure 11 for the ILS and VMC-Like approaches reveals that 

the SCFD, PBD, and SVS display concepts were rated as a 2 (“Good, Negligible Deficiencies”) at least 

60% of the time. There was a slight increase in the ratings for the SVS and SCFD display concepts for 

the VMC-Like approach compared to the ILS approach. The SVS display received a rating of 3 (“Fair, 

Some Mildly Unpleasant Deficiencies”) or better 80% of the time instead of a 2. The SCFD display 

concept was rated as a 4 “Minor But Annoying Deficiencies” or better 80% of the time instead of a 3. 

The SVS display concept median rating increased to a 2 from a 1 for the VMC-Like approach compared 

to the ILS approach. The median ratings for the SCFD and PBG display concepts was a 2 “Good, 

Negligible Deficiencies” for both the ILS and VMC-Like approaches. These increases in ratings are 

attributed to the higher workload of the VMC-Like approach. 

4.3.6 Test condition analysis

Test condition analysis was conducting using a 4-level ranking method. First, the TCs were ranked 

based on median DRR received, then on the DRR required to capture at least 60% of the results, then on 

the DRR required to capture 80% of the results. Lastly, the TCs were ranked based on the maximum 

DRR rating received. The results of this analysis is presented in Table 22. The most favorably rated TCs 

were those for the VFR pilots only for the ILS and VMC-Like approaches with the SVS display concept 

that received median DRR results of 1 - “Excellent, Highly Desirable” characteristics. It is believed that 

the VFR pilots rated these conditions favorably due to the presence of the SVS terrain that provided them 

with flight information in an intuitive out-the-window manner they are most accustomed to. The ILS 

approach with the SVS display concept for all pilots also received a median DRR rating of 1. The VFR 

approach test conditions with the SVS and PBG display concepts received median DRR ratings of 2 

(“Good, Negligible Deficiencies”). The presence of the SVS terrain on the SVS display concept is 

believed to provide a higher level of information integration during IMC conditions than in VMC 

conditions which leads to the lower DRR ratings for most of the IMC TCs. The SCFD was rated slightly 

better than the PBG display concept for the ILS approach but slightly worse for the VMC-Like approach. 

It is believed that the presence of the highway-in-the-sky tunnel of the PBG display concept provides 

substantial benefits for curved approaches. The presence of the SVS terrain provided a larger reduction in 

the DRR ratings for the ILS approach than for the VMC-Like approach. For the ILS approach the 

reduction of DRR ratings due to the SVS terrain resulted in a lowering of the median rating to a 1 from a 

2 as well as reducing the range of DRR rating required to capture 80% results and maximum ratings. For 

the VMC-Like approach PBG and SVS displays concepts were rated identically to each other. All of the 

IMC TCs were rated better than the ILS + BRD display concept baseline, which does not include VFR 

pilots data for the test condition analysis. This indicates that the SCFD, PBG, and SVS display concepts 

provide improved display readability for the ILS approach, compared to what is currently the minimum 

required equipage, and could be expected to facilitate advanced IMC approaches similar to the VMC-Like 

approach considered herein. 
49

Table 22: DRR Results for Test Condition.

Test Condition Name TC Median <60% <80% Max

ILS Approach + SVS (only VFR pilots) 12 1 1 1 3

VMC-Like Approach + SVS (only VFR pilots) 13 1 1 3 4

ILS Approach + SVS 8 1 2 2 5

VFR Approach + SVS 4 2 2 2 4

VFR Approach + PBG 3 2 2 3 4

ILS Approach + SCFD 6 2 2 3 5

VMC-Like Approach + PBG 10 2 2 3 5

VMC-Like Approach + SVS 11 2 2 3 5

ILS Approach + PBG 7 2 2 3 6

VMC-Like Approach + SCFD 9 2 2 4 7

VFR Approach + BRD 1 2 3 3 5

VFR Approach + SCFD 2 3 3 4 8

ILS Approach + BRD 5 3 3 5 7

4.4 Terrain Awareness

The EPs were asked to rate their overall understanding of the terrain environment within which they 

were operating. Items to consider were the quantity and quality of the terrain information available to 

them and how comfortable they were with their terrain awareness. These data were acquired to augment 

other results and facilitate the determination of the effect that terrain awareness provides to overall SA. 

Terrain Awareness (TA) ratings for the four display concepts and three approaches are presented in 

Figure 12 and Table 23 and were generated through use of the TA question in Appendix C. (Error bars in 

Figure 12 represent one SD.) Individual TA results for each evaluation are provided in Appendix D.
50

Table 23: TA Descriptive Statistics (all Pilots)

Display 

Concept Approach Mean StandardDeviation

Number of Data 

Points

BRD VFR 84.5 11.9 24

ILS 9.2 9.7 29

Total 48.0 40.2 37

SCFD VFR 61.3 26.2 19

ILS 48.1 22.5 18

VMC-Like 46.1 23.0 18

Total 52.0 24.5 55

PBG VFR 69.0 23.5 18

ILS 51.7 30.2 17

VMC-Like 55.4 28.7 19

Total 58.8 28.1 54

SVS VFR 87.8 10.6 19

ILS 84.8 17.5 21

VMC-Like 79.7 25.1 19

Total 84.1 18.6 59

Total VFR 75.9 22.0 75

ILS 49.8 34.5 74

VMC-Like 60.7 29.1 56

Total 60.2 32.1 221
51

Figure 12: Terrain Awareness Results for All Display Concepts, All Pilots and All Approaches

4.4.1 VFR approach discussion

For the VFR approach, TA ratings were high for the BRD display concept (84.5) but decreased for the 

SCFD and PBG display concepts. Even though evaluation pilots were instructed to fly a VFR approach 

using the out-the-window environment for navigation, EPs occasionally used the guidance provided by 

the advanced display concepts (SCFD, PBG, and SVS) to cross check their flight path. An ANOVA 

conducted on the TA results for the VFR approach for the main effects of display concept and pilot type 

resulted in highly significant results for display concept (F(3,80)=8.75, p<0.01) but not for pilot type. A 

subsequent SNK post-hoc analysis resulted in two unique subsets shown in Table 24. Evaluation pilots 

noted that the increased attention allocated to looking at the displays for cross-check information versus 

out-the-window navigation reduced their terrain awareness, especially for the SCFD display concept. In 

Table 24 terrain awareness significantly decreased 15.5 units for the PBG display concept compared to 

the BRD. Terrain awareness also decreased another 7.7 units for the SCFD compared to the PBG display 

concept. However, for the SVS display concept, terrain awareness ratings were slightly higher than for 

the BRD display concept. Traditionally pilots needed to look outside the aircraft to see terrain, and other 

threats such as traffic, and inside the aircraft for flight and navigation information. The ratio of the time 

pilots look outside the aircraft to within is frequently used as a evaluation criterion for new avionics and 

procedures. However, the presence of SVS terrain on the PFD alters this paradigm since terrain 

information is provided in both eyes-out and eyes-in locations.

0

20

40

60

80

100

120

VFR ILS VMC-Like

BRD

SCFD

PBG

SVS TA

Approach
52

Table 24: Mean TA SNK Results for Display Concept, for all Pilots for the VFR Approach 

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 19 61.3

PBG 18 69.0

BRD 24 84.5

SVS 19 87.8

Significance 0.20 0.58

4.4.2 ILS approach, all pilots

An ANOVA conducted on the TA results for the ILS approach for the main effects of display concept 

and pilot type resulted in highly significant results for display concept (F(3,85)=59.18, p<0.01) but not for 

pilot type. A subsequent SNK post-hoc analysis resulted in three unique subsets shown in Table 25. 

From Table 25 it can be seen that the BRD display concept was rated much worse in terms of TA with a 

38.9 unit decrease compared to the SCFD display concept for the ILS approach and was the only member 

of this subset. This reflects the EPs very low awareness of the terrain environment within which they 

were operating with the BRD display concept without out-the-window visibility. For this condition, EPs 

created a mental model of the terrain outside the aircraft through their knowledge of where they were 

along the ILS approach path combined with their mental recall of the operational area and paper charts. 

Compared to the BRD display concept, the SCFD and PBG display concepts yielded significant increases 

in terrain awareness of 38.9 and 42.5 units respectively, and were the only two members of the 

intermediate terrain awareness subset, reflecting the incorporation of the ND. Terrain awareness for the 

SVS display concept was significantly increased by 33.1 units compared to the PBG display concept for 

the ILS approach. This result is due to the presence of a perspective display of 3-D terrain on the PFD

and its intuitive integration with the flight information and guidance symbology. Note that the level of 

terrain awareness for the SVS display concept for the ILS approach was virtually identical to the level of 

terrain awareness for the BRD display concept for the VFR approach as shown in Figure 12.

Table 25: Mean TA SNK Results for Display Concept, for all Pilots for the ILS Approach 

Display 

Concept

Number of 

Data Points

Subset

1 2 3

BRD 29 9.2

SCFD 18 48.1

PBG 17 51.7

SVS 21 84.8

Significance 1.00 0.55 1.00
53

4.4.3 VMC-Like approach, all pilots

An ANOVA conducted on the TA results for all pilots and display concepts revealed that the main 

effect was highly significant for display concept (F(2,56)=10.1, p<0.01) but not for pilot type (p>0.05). 

There were no significant interactions between the main effects. An SNK post-hoc analysis on the TA 

results provided two unique subsets for display concept as shown in Table 26. Even though the outside 

terrain was provided in a gods-eye-view format on the ND, the addition of the SVS terrain on the PFD 

significantly improved perceived terrain awareness for the SVS display concept compared to the PBG 

display concept. Overall, the SVS display concept was the only member of the highest (best) terrain 

awareness subset due to the effective integration of terrain with guidance and flight information 

symbology. The improvement in terrain awareness due to guidance symbology for the PBG display 

concept was approximately 9 units and not statistically different compared to the SCFD display concept. 

It is likely that the improved terrain awareness was related to the reduced workload for the PBG display 

concept that allowed pilots more time to absorb terrain information from the ND.

Table 26: Mean TA SNK Results for Display Concept for all Pilots for the VMC-Like Approach 

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 18 46.1

PBG 19 55.4

SVS 19 79.7

Significance 0.228 1.00

4.4.4 ILS approach without low-time VFR pilot data

Results for TA were analyzed without the low-time VFR pilot data, enabling a more direct comparison 

of the four display concepts within current operations. Results from an ANOVA indicated that display 

concept was highly significant (F(3,56)=43.26, p<0.01) for TA. Pilot type was not significant for TA

(p>0.05). As can be seen in the Table 27, three unique subsets were formed. The lowest (worst) TA was 

for the BRD display concept and represents the nominal level of terrain awareness for operations with 

traditional avionics. Compared to the BRD display concept, the incorporation of the gods-eye view of 

terrain on the ND combined with more intuitive flight path information significantly increased terrain 

awareness 39.9 and 47.7 units for the SCFD and PBG display concepts, respectively. However, for the 

SVS display concept, terrain awareness increased an additional 26.8 units compared to the PBG display 

concept. Subsequent SNK results, provided in Table 27, show that the SVS display concept was the only 

member of the highest (best) TA rating group. The SCFD and PBG display concepts were statistically 

similar and were in the middle subset. 
54

Table 27: Mean TA SNK Results for the ILS Approach, IFR and H-IFR Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2 3

BRD 18 10.9

SCFD 12 50.8

PBG 11 58.6

SVS 15 85.5

Significance 1.00 0.30 1.00

4.4.5 ILS and VMC-Like approach analysis, all pilots, advanced displays

In an effort to compare the advanced display concepts with each other and also to evaluate the 

potential of these displays to support an advanced IMC approach for all pilot types, an ANOVA was 

performed on the TA data for the advanced displays with all pilots’ data for the ILS and VMC-like 

approaches. Results indicated that display concept was highly significant (F(2,112)=29.606, p<0.01) for 

this measure. Pilot type was significant (F(2,112)=2.91, p=0.01), but approach was not for the TA 

ratings. Post-hoc SNK analysis revealed no significant TA differences for pilot type. However, in Table 

28, it can be seen that the VFR pilots reported lower TA than the IFR pilots, and that the IFR pilots 

reported lower TA than the H-IFR pilots. This trend could be attributed to pilots’ experience, with the 

more experienced pilots having the ability to better control the aircraft while absorbing terrain 

information from the various displays. 

Table 28: Mean TA SNK Results for Pilot Type for the ILS and VMC-Like Approach for all Pilots

EP-type Number of 

Data Points Mean

VFR 36 57.7

IFR 37 61.6

H-IFR 39 65.8

Significance 0.22

Post-hoc SNK analysis for display concept revealed two unique subsets for the TA ratings (see Table 

29). The SVS display concept was rated as providing the most terrain awareness. While the mean TA 

ratings were higher for the PBG display concept compared to the SCFD concept, the difference was not 

enough to be significant. The addition of SVS terrain appears to have increased TA for the two IMC 

approaches, with an increase in TA of almost 29 units. 
55

Table 29: Mean TA SNK Results for Display Concept for the ILS and VMC-Like Approach, All 

Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 36 47.1

PBG 36 53.7

SVS 40 82.4

Significance 0.18 1.00

4.4.6 Test condition analysis

A subsequent analysis on TA was performed based on TC with post-hoc results presented in Table 30. 

Test Condition was highly significant for TA results (F(12,221)=20.583, p<0.01). Four overlapping

subsets were formed from post-hoc analysis. The highest (best) rated group included all of the VFR test 

conditions, as would be expected, except for the SCFD display concept. As noted previously, evaluation 

pilots reported some frustration with the SCFD for the VFR approach. As a result TC 2 (VFR approach

with SCFD) had lower TA results, which put it in the next lower subset. All of the simulated IMC 

evaluations of the SVS display concept were grouped with the highly rated VFR evaluations. This 

indicates that, for TA, out-the-window visibility had little effect for the SVS display concept. Terrain 

Awareness results may also indicate that advanced IMC approaches, like the VMC-like approach, are 

feasible with SVS.

The worst TA subset was comprised of only TC 5 (ILS approach evaluation of the BRD display 

concept). While this type of result has been observed for the other variables (i.e., SART, TLX, DRR), the 

separation of TC 5 from the others is much more pronounced for TA. Subset 2 is comprised of ILS and 

VMC-like evaluations of the SCFD and PBG display concepts and is only slightly different in 

composition than subset three. Note that subset three includes TC 3 (VFR approach with the PBG display 

concept) and doesn’t include TC 9 (VMC-like approach with the SCFD display concept). Overall, the TC 

analysis of terrain awareness indicates that all the advanced display concepts provided superior TA than 

TC 5 (ILS approach, BRD display concept) for both ILS and VMC-like approaches, most likely due to 

the presence of terrain on the ND. However, the SVS display concept provides additional significant 

improvements in terrain awareness with results for simulated IMC that are as good or better than those for

the VFR approach with the BRD.
56

Table 30: Mean TA SNK Results for Test Condition

Test Condition Name TC 

#

Number 

of Data 

Points

Subset

1 2 3 4

ILS Approach + BRD (no VFR pilots) 5 18 10.9

VMC-Like Approach + SCFD 9 18 46.1

ILS Approach + SCFD 6 18 48.1

ILS Approach + PBG 7 17 51.7 51.7

VMC-Like Approach + PBG 10 19 55.4 55.4

VFR Approach + SCFD 2 19 61.3 61.3

VFR Approach + PBG 3 18 69.0 69.0

VMC-Like Approach + SVS 11 19 79.7

VMC-Like Approach + SVS (only VFR pilots) 13 6 80.7

ILS Approach + SVS (only VFR pilots) 12 6 82.8

VFR Approach + BRD 1 24 84.5

ILS Approach + SVS 8 21 84.8

VFR Approach + SVS 4 19 87.8

Significance 1.0 0.2 0.1 0.1
57

4.5 PTS Results

This section discusses the results against IFR Practical Test Standard (PTS) criteria. Recall that the 

PTS criteria are composed of a combination of lateral, vertical, and airspeed errors. PTS results are 

presented in terms of a percentage of time the evaluation pilots were able to satisfy all three criterion

simultaneously during Segments 2 and 3 (i.e., from points A and B in Figure 8 down to 200 ft AGL). 

PTS results are shown in Figure 13 and presented in Table 31. (In Figure 13, the error bars represent one 

SD.) Subsequent sections of this report discuss each of the parameters employed for the PTS criteria

analyses, sometimes focusing only on the final approach segment of the total approach (i.e., Segment 3, 

simultaneous localizer and glideslope tracking).

Table 31: PTS Descriptive Statistics (% Time), Segments 2 and 3 for all Pilots

Display 

Concept Approach Mean StandardDeviation

Number of 

Data Points

BRD ILS 73.4 23.7 30

Total 73.4 23.7 30

SCFD ILS 88.2 9.9 18

VMC-Like 89.5 10.5 18

Total 88.8 10.1 36

PBG ILS 90.4 9.8 16

VMC-Like 90.1 12.5 19

Total 90.2 11.2 35

SVS ILS 90.7 7.9 27

VMC-Like 92.0 10.5 19

Total 91.2 9.0 46

Total ILS 84.5 17.2 91

VMC-Like 90.5 11.1 56

Total 86.8 15.4 147
58

Figure 13: Percent Time within IFR PTS Performance Metrics, All Pilots, Segments 2 and 3

4.5.1 ILS approach, all pilots

An ANOVA conducted on the PTS data for all pilots for the ILS approach for the main effects of 

display concept and pilot type resulted in highly significant results for display concept (F(3,85)=8.28, 

p<0.01) and significant results for pilot type (F(2,85)=3.64, p=0.03). There were no significant 

interactions between the main effects. Subsequent SNK post-hoc analysis revealed two unique subsets

for both display concept and pilot type as shown in Tables 32 and 33. The BRD display concept was the 

only member of the worst PTS performance group. The incorporation of the ND, with desired flight path

and ownship symbology, combined with the SCFD guidance or HITS pathway presentations of the 

advanced display concepts, greatly facilitated the evaluation pilots’ ability to remain within the PTS 

criteria. All of the advanced display concepts (i.e., SCFD, PBG, and SVS) were included in the best PTS 

performance group. A significant 14.8% improvement in PTS results occurred for the SCFD display 

concept compared to the BRD. The PBG and SVS display concepts improved PTS results by 2.2% and 

2.9%, respectively, compared to the SCFD.

As anticipated for the ILS approach, the VFR pilots’ PTS performance was significantly worse than 

the IFR and H-IFR pilots. Table 34 presents PTS results for display concept and pilot type and the 

difference between results for the VFR pilots compared to the average results for the IFR and H-IFR pilot 

groups and shows that differences between the different pilot types were much larger for the BRD display 

concept than the advanced display concepts. The decrease in VFR pilot performance was 4.6 times worse 

(i.e., 23.7/5.2=4.6) for the BRD display concept compared to SCFD and 9.9 times (i.e., 23.7/2.4=9.9) 

40

50

60

70

80

90

100

110

ILS VMC-Like

BRD

SCFD

PBG

SVS

% Time in within PTS Boundary

Approach
59

worse than the SVS display concept. This result can be attributed to the VFR pilots’ lack of training in

integrating disparate information from the individual BRD display gauges. However, the highlyintegrated and intuitive display of information of the advanced display concepts greatly improved the 

VFR pilots ability to fly without outside visual cues. This was especially true for the SVS display 

concept. Subsequent sections of this report provide analyses of the constituent elements of the PTS 

criteria (i.e., localizer, glideslope and IAS errors) providing insight into the distribution of exceedances 

along the approach.

Table 32: Mean PTS (% of Time) SNK Results for Display Concept for all Pilots, ILS Approach.

Display 

Concept

Number of 

Data Points

Subset

1 2

BRD 30 73.4

SCFD 18 88.2

PBG 16 90.4

SVS 21 91.1

Significance 1.00 0.797

Table 33: Mean PTS (% of Time) SNK Results for Pilot Type for All Display Concepts, ILS 

Approach.

Pilot Type Number of 

Data Points

Subset

1 2

VFR 29 75.9

H-IFR 27 88.0

IFR 29 88.7

Significance 1.00 0.849
60

Table 34: Mean PTS (% of Time) Results for Pilot Type and Display Concept, ILS Approach.

Display 

Concept

Pilot 

Type Mean VFR pilots results compared to the average of IFR and H-IFR pilots

Number of Data 

Points

BRD VFR 58.5 -23.7 11

IFR 82.8 11

H-IFR 81.1 8

SCFD VFR 84.7 -5.2 6

IFR 90.8 6

H-IFR 89.0 6

PBG VFR 85.7 -6.7 6

IFR 94.4 4

H-IFR 92.5 6

SVS VFR 89.4 -2.39 6

IFR 92.4 8

H-IFR 91.1 7

4.5.2 VMC-Like approach, all pilots, advanced displays

An ANOVA conducted on the PTS results for the main effects of display concept (F(2,56)=1.25, 

p=0.30) and pilot type (F(2,56)=0.88, p=0.42) resulted in no significant (p<0.05) differences for either 

and no significant interactions. Tables 35 and 36 provide the mean results for display concept and pilot 

type. While not statistically significant, the SVS display concept enabled pilots to remain within the PTS 

criteria 2.5% longer than the SCFD. In addition, the VFR pilots were able to generate similar PTS results 

to the IFR pilots and only 3.9% worse than the H-IFR pilots, even though the VMC-like approach

included two 90-degree descending turns during the last 3.4 nm of the approach.

Table 35: Mean PTS (% of Time) SNK Results for Display Concept for all Pilots, VMC-Like 

Approach

Display Concept Number of 

Data Points Subset

SCFD 18 89.5

PBG 19 90.1

SVS 19 92.0

Significance 0.503
61

Table 36: Mean PTS (% of Time) SNK Results for Pilot Type for All Display Concepts, VMC-Like 

Approach

Pilot Type Number of 

Data Points Subset

IFR 18 89.0

VFR 18 89.2

H-IFR 20 93.1

Significance 0.169

4.5.3 ILS approach Segment 3 without VFR pilot data

An ANOVA was conducted on the ILS approach Segment 3 results without the VFR pilot data for the 

main effects of display concept and pilot type. Results of the ANOVA indicated no significant effect of 

display concept or pilot type (p>0.05). It is likely that instrument training for the IFR and H-IFR pilots

enabled them to maintain acceptable performance even with the BRD display concept. However, while 

the results for the display concept were not statistically significant, the BRD display concept tended 

toward the worst PTS performance of the tested concepts, as shown in Table 37. IFR and H-IFR pilots 

were only able to satisfy all three of the PTS criteria simultaneously 83.1% of the time. This establishes 

the baseline minimum required PTS performance within the data set considered herein. The SCFD 

display concept improved PTS results by 7.8% compared to the BRD and reflects the improved 

capabilities due to the ND and integrated PFD with the single-cue flight director. The PBG and SVS 

display concepts improved PTS results by 1.8% and 3.3% compared to the SCFD display concept, 

respectively. 

Table 37: PTS (% of Time) Results for Display Concept for Segment 3 of the ILS Approach, IFR 

and H-IFR Pilots Only

Display 

Concept Mean

BRD 83.1*

SCFD 89.2

SVS 90.9

PBG 92.0

*Note: This value is used herein as the baseline performance for an ILS Approach.

In contrast to the analysis with the VFR pilot data, the effect of display concept was substantially 

reduced, as can be seen through comparison of the ANOVA results as well as Tables 32 and 37. This 

result indicates that the VFR pilots had a much harder time with the BRD display concept than the 

instrument rated pilots.

4.5.4 ILS and VMC-Like approaches, Segment 3, all pilots, advanced displays

Figure 14 presents PTS criteria results for Segment 3 of the ILS- and VMC-Like approach maneuvers 

for all display concepts. An ANOVA was performed on the PTS data for the advanced displays for 
62

segment 3 of the ILS and VMC-like approaches in an effort to compare the advanced display concepts 

with each other and also to evaluate their ability to support advanced IMC approaches for all pilot types. 

Display concept, approach, and pilot type were not significant (p>0.05) for this analysis and there were no 

significant interactions. While not significant, Tables 38, 39, and 40 provide information to assess the 

interactions between display concept, approach and pilot type. General trends in the data indicate that the 

VFR pilots performed 2 to 6% worse than the IFR and H-IFR pilots and that the effect of pilot type was 

the smallest for the SVS display concept. Overall, PTS performance results for the VMC-like approach

with the advanced display concepts were as good or better than the acceptable PTS performance for the 

ILS approach as established herein. This result indicates that the advanced display concepts can 

potentially enable pilots to accurately perform approaches other than classic ILS-type approaches, with 

long straight legs, across a broad range of pilot skills.

Figure 14: Percent Time within IFR PTS performance Metric, All Pilots, Segment 3 Only

40

50

60

70

80

90

100

110

ILS VMC-Like

BRD

SCFD

PBG

SVS

% Time in within PTS Boundary

Approach
63

Table 38: PTS (% of Time) Criteria for Display Concept and Approach interactions, ILS and 

VMC-Like Approaches, Segment 3

Display 

Concept Approach Mean

SCFD ILS 87.8

VMC-Like 85.4

PBG ILS 89.6

VMC-Like 85.5

SVS ILS 89.9

VMC-Like 92.1

Table 39: PTS (% of Time) Criteria Results for Display Concept and Pilot Type Interactions, ILS 

and VMC-Like Approaches, Segment 3

Display 

Concept Pilot Type Mean

SCFD VFR 84.3

IFR 86.8

H-IFR 88.7

PBG VFR 84.5

IFR 86.8

H-IFR 90.4

SVS VFR 88.9

IFR 91.3

H-IFR 92.4

Table 40: PTS (% of Time) Criteria Results for Approach and Pilot Type Interactions, ILS and 

VMC-Like Approaches, Segment 3

Approach Pilot Type Mean

ILS VFR 86.1

IFR 91.7

H-IFR 89.6

VMC-Like VFR 85.7

IFR 85.4

H-IFR 91.5
64

4.5.5 Test condition analysis, Segment 3

An ANOVA was performed on the PTS results for TC with SNK post-hoc results presented in Table 

40. The analysis was limited to segment 3. Test Condition was not significant (p>0.5) for PTS results. 

From Table 40, it can be seen that the worst PTS results were for TC 5 (ILS approach with BRD display 

concept) and the best results were for TC 11 (VMC-like approach for the SVS display concept). Of the 

simulated-IMC evaluations, TC 5 was again the worst while all combinations of advanced display 

concepts and IMC approaches (i.e., ILS or VMC-like) were better, though not significantly. Not only did

the advanced display concepts slightly improve PTS results compared to the BRD, but this improvement 

was accomplished when also including the VFR pilot data. Note that TC 8 (ILS approach with SVS 

display concept) was 2.2% worse than TC 11 (VMC-like approach with SVS display concept). One 

possible explanation for this result involves the VFR pilots and their increased familiarity with the VMClike approach, due to its similarity to the VFR approach, compared to the ILS approach. Occasionally, 

VFR evaluation pilots would forget to reduce their airspeed from 100 kts to 90 kts when intercepting the 

glideslope for the ILS approach, but never missed the airspeed reduction at waypoint NEWT for the 

VMC-like approach. Recall that waypoint NEWT was located abeam the runway numbers on the 

downwind leg and was a significant event location for the VFR approach. It is also worthwhile to note 

that the effect of the SVS terrain led to a 7% improvement in evaluation pilots’ performance compared to 

the PBG display concept for the VMC-like approach. This is likely due to the increased awareness 

provided by the SVS terrain background for this highly dynamic approach segment. It is believed that the 

SVS terrain served as an integrator for the other disparate pieces of information provided on the PFD (i.e.,

attitude, altitude, flight path, heading, etc.) and was particularly valuable for the VFR pilots. Further 

discussion regarding the elements of the PTS metric are provided in subsequent sections. In particular it 

is shown that the effect of the SVS terrain enables a significant improvement in the pilots ability to 

minimize glideslope error. Results for the VFR pilot evaluations of the SVS display concept for the ILS 

and VMC-like approaches were better than TC 5 (ILS approach with BRD display concept). Although 

the PTS results were not statistically significant, they indicate that advanced IMC approaches may be 

possible for the advanced display concepts, especially for the SVS display concept.

Table 41: Mean PTS (% of Time) SNK Criteria Results for TC, All pilots, Segment 3

Test Condition Name TC Number of 

Data Points Mean

ILS Approach + BRD (no VFR pilots) 5 19 83.1

VMC-Like Approach + SCFD 9 18 85.4

VMC-Like Approach + PBG 10 19 85.5

ILS Approach + SVS (VFR pilots) 12 6 87.7

ILS Approach + SCFD 6 18 87.8

ILS Approach + PBG 7 16 89.6

ILS Approach + SVS 8 21 89.9

VMC-Like Approach + SVS (VFR pilots) 13 6 90.2

VMC-Like Approach + SVS 11 19 92.1

Significance 0.715
65

4.6 Localizer Error

Using localizer error results to describe the differences between the different display concepts and 

approaches is beneficial because the large majority of traditional IMC operations employ this type of 

guidance strategy. It is an especially valuable parameter to work with since the BRD display concept 

only displayed angular errors to the pilots. Comparing display concepts only on linear lateral errors 

substantially skews results towards the advanced display concepts since their guidance cue displacements 

are based on linear lateral error and are not affected by range to touchdown. Localizer error results for the 

four display concepts and three approaches are presented in Figure 15, 16, 17, and 18 along with Table 

42.

Table 42: Localizer Error (dots) Descriptive Statistics for all Pilots

Display 

Concept Approach Mean StandardDeviation

Number of 

Data Points

BRD VFR 2.78 1.30 19

ILS 0.75 0.46 18

Total 1.65 1.37 37

SCFD VFR 0.40 0.37 19

ILS 0.17 0.09 18

VMC-Like 0.43 0.30 18

Total 0.34 0.30 55

PBG VFR 0.18 0.08 18

ILS 0.09 0.03 16

VMC-Like 0.19 0.07 19

Total 0.16 0.08 53

SVS VFR 0.20 0.15 18

ILS 0.10 0.03 20

VMC-Like 0.17 0.07 19

Total 0.15 0.10 58

Total VFR 1.03 1.38 79

ILS 0.34 0.41 85

VMC-Like 0.26 0.22 56

Total 0.57 0.93 220

4.6.1 VFR approach, all pilots

As can be seen in Figure 15 and Table 43, the VFR approach evaluation of the BRD display concept 

resulted in large localizer error values compared to the advanced display concepts. This increase in 

localizer error is again due to the EPs’ flight path navigation being driven totally by the out-the-window 

view for the BRD display concept. A primary source of localizer error for the VFR approach evaluation 

of the BRD display concept occurred during the base leg. EPs would fly base legs that sometimes were 

inside of the intended flight path while still maintaining reasonable VFR approach conditions. Providing

EPs with moving maps, desired flight path and ownship representation as included in the navigation 

display, along with flight path error and flight path guidance symbologies on the advanced display 

concepts, greatly reduced localizer error, as indicated in Figure 15 and Table 43. This result was 

generated even though the EPs were primarily flying a VFR eyes-out approach and using the guidance 
66

information to cross check their out-the-window navigation. Note that the PBG and SVS display 

concepts produced the lowest level of localizer error (approximately 0.19 dots mean). An ANOVA 

conducted on the localizer error data for the VFR approach for the main effects of display concept and 

pilot type resulted in highly significant results for display concept (F(3,79)=162.79, p<0.01) and nonsignificant results for pilot type (p>0.05). 

Table 43: Mean Localizer Error (dots) SNK results for Display Concept VFR Approach for All 

pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

PBG 18 0.18

SVS 18 0.20

SCFD 19 0.40

BRD 24 2.78

Significance 0.328 1.000
67

Figure 15: VFR Approach, Localizer Error, By Display Concept and Pilot Group

-3

-2

-1

0

1

2

3

VFR Approach, BRD, All Pilots LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

Localizer Error, dots Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

VFR Approach, PBG, All Pilots

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

Localizer Error, dots

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

VFR Approach, SCFD, All Pilots

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

Localizer Error, dots

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

0 1 2 3 4 5 6 7

VFR Approach, SVS, All Pilots LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

LocalizerDot(43)

Localizer Error, dots

Distance to Touchdown, nm

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

VFR Pilots

IFR Pilots

H-IFR Pilots
68

4.6.2 ILS approach, all pilots

For the ILS approach for all pilots, the localizer error decreased for all display concepts compared to 

the results for the VFR approach, as shown in Figures 15 and 16 and Table 44. The largest decrease was

for the BRD display concept, which decreased from 2.8 dots for the VFR approach to 0.49 dots for the 

ILS approach. This reduction was due primarily to the increased level of flight path precision associated 

with IMC operations where pilots relied entirely on flight path guidance and error cues to minimize flight 

path error, along with the different nature of the ILS approach with long, straight approach legs. Figure 

16 also shows that the effect of pilot group was eliminated for the advanced display concepts. This result 

was not the case for the BRD display concept, where the VFR pilot data indicates a difference compared 

to the IFR and H-IFR pilot data. Figure 16 also shows that the 1.5 dot IFR PTS criterion was met for the 

advanced display concepts for both Segment 2 and 3 of the ILS approach and not met for the BRD 

display concept for the VFR pilots for Segment 3. Again, Segment 2 did not include specific guidance 

cues for the BRD display concept and pilots were merely flying a localizer intercept heading after 

approach initialization, then tracking the localizer and maintaining altitude until intercepting the 

glideslope. 

An ANOVA conducted on the localizer error data for the ILS approach for the main effects of 

display concept and pilot type revealed that display concept was highly significant (F(3,85)=40.9, p<0.01) 

and pilot type was not significant (p>0.05). There was a significant interaction between display concept 

and pilot type (F(6,85)=2.63, p=0.025). Post-hoc analysis resulted in the two subsets shown in Table 44. 

The worst subset was comprised of only the BRD display concept. The advanced display concepts 

comprised the best subset with the SCFD reducing localizer error by 77% compared to the BRD. The 

PBG and SVS display concepts reduced localizer error further compared to the SCFD by 41% and 47%,

respectively.

The relatively poor performance of the BRD compared to the advanced display concepts can be 

partially attributed to the lack of guidance during Segment 2 of the ILS approach, along with the 

incorporation of the VFR pilot data. Table 45 provides results to support the pilot/display-concept 

interaction and shows that for the BRD display concept, the VFR EPs flew much worse than the IFR and 

H-IFR pilots. The data seems indicative of the lack of IFR training for the VFR pilots. They did receive

the same training associated with this flight test, but it can also be stated that for the VFR-rated EPs, the 

out-the-window view is key to integrating and assimilating all the information they are being presented. 

When that out-the-window view is no longer available, VFR EPs had a challenging time integrating the 

information presented via the BRD display concept. However, the integrated presentation of flight 

information on the advanced display concepts enabled the VFR pilots to produce localizer error results 

similar to the IFR and H-IFR pilot groups. 

Table 44: Mean Localizer Error (dots) SNK results for Display Concept for ILS Approach for all 

Pilots

Display Concept Number of 

Data Points

Subset

1 2

PBG 16 0.09

SVS 21 0.10

SCFD 18 0.17

BRD 30 0.75

Significance 0.596 1.000
69

Figure 16: ILS Approach, Localizer Error, By Display Concept and Pilot Group

-3

-2

-1

0

1

2

3

ILS Approach, BRD, All Pilots

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

LocalizerDot(

Localizer Error, dots

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

ILS Approach, PBG, All Pilots Localizer

Localizer

Localizer

Localizer

Localizer

Localizer

Localizer

Localizer

Localizer

Localizer

Localizer

Localizer

Localizer

Localizer

Localizer

Localizer

Localizer Error, dots

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

ILS Approach, SCFD, All Pilots

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

Localizer Error, dots

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

0 1 2 3 4 5 6 7 8 9 10

ILS Approach, SVS, All Pilots LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

Localizer Error, dots

Distance to Touchdown, nm

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

VFR Pilots

IFR Pilots

H-IFR Pilots
70

Table 45: Localizer Error Results (dots) for Display Concept and Pilot Type Interactions for the 

ILS Approach

Display 

Concept Pilot Type Mean Standard Deviation

Number of Data 

Points

BRD VFR 1.04 0.51 11

IFR 0.57 0.28 11

H-IFR 0.60 0.41 8

SCFD VFR 0.21 0.08 6

IFR 0.18 0.11 6

H-IFR 0.11 0.08 6

PBG VFR 0.10 0.03 6

IFR 0.08 0.03 4

H-IFR 0.10 0.04 6

SVS VFR 0.10 0.05 6

IFR 0.09 0.03 8

H-IFR 0.10 0.03 7

4.6.3 VMC-Like approach, all pilots

Figure 17 shows very different results for the three advanced display concepts for the VMC-like 

approach. While results are very similar for Segment 2 (i.e., the straight, constant-altitude leg), 

substantial differences are evident in Segment 3 (i.e., two descending 90-degree turns). The differences 

apparent for the SCFD display concept compared to the PBG and SVS display concepts are located at the 

turns to base leg and final approach and can be attributed to the lack of turn anticipation for this display 

concept. All of the advanced display concepts included the ND with the moving map, flight path and

ownship symbology. It is evident from these data that the evaluation pilots were not looking at the ND 

very much during this approach, especially for Segment 3, due to the rapid pace and relatively high

workload. An ANOVA conducted on the localizer data for the VMC-like approach for the main effects 

of display concept and pilot type revealed highly significant results for display concept (F(2,56)=13.69, 

p<0.01) and non-significant results for pilot type (p>0.05). There was no significant interaction between 

display concept and pilot type. Subsequent post-hoc analysis, as shown in Table 46, reveals two unique 

subset-groups. The lowest (best) subset was comprised of the SVS and PBG display concepts, while the 

SCFD was the only member of the higher (worst) subset with localizer error 2.4 times larger than the 

mean for the SVS and PBG display concepts.

Table 46: Mean Localizer Error (dots) SNK results for Display Concept for the VMC-Like 

Approach for all Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

SVS 19 0.17

PBG 19 0.19

SCFD 18 0.43

Significance 0.655 1.000
71

Figure 17: VMC-Like Approach, Localizer Error, By Display Concept and Pilot Group

-2

-1

0

1

2

3

4

VMC-Like Approach, PBG, All Pilots

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

LocalizerD

Localizer Error, dots

IFR PTS Criterion

Segment 3 Segment 2

IFR PTS Criterion

-2

-1

0

1

2

3

4

0 1 2 3 4 5 6 7

VMC-Like Approach, SVS, All Pilots

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

Localizer Error, dots

Distance to Touchdown, nm

IFR PTS Criterion

Segment 3 Segment 2

IFR PTS Criterion

-2

-1

0

1

2

3

4

VMC-Like Approach, SCFD, All Pilots

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

LocalizerDo

Localizer Error, dots

IFR PTS Criterion

Segment 3 Segment 2

IFR PTS Criterion

VFR Pilots

IFR Pilots

H-IFR Pilots
72

4.6.4 ILS approach, Segment 3, no VFR pilots

Results for localizer error were analyzed without the VFR pilot data for Segment 3 (glideslope and 

localizer tracking), enabling a more direct comparison of the four display concepts within current IMC 

operations. Results from an ANOVA revealed that display concept was highly significant 

(F(3,56)=38.847, p<0.01) and that pilot type (i.e., IFR or H-IFR) was not significant for localizer error. 

In addition, there were no significant (p>0.05) interactions between pilot type and display concept for this 

measure. Subsequent post-hoc analysis produced two unique subsets, as shown in Table 47. The 

advanced display concepts (i.e., PBG, SVS, and SCFD) all produced significantly lower localizer error 

than the BRD display concept. This statistically significant result is visually supported by Figure 16, 

which shows the improvement in localizer error for the advanced display concepts. The average localizer 

error for the advanced display concepts was approximately one-fifth the result for the BRD display 

concept. This substantial decrease in localizer error could possibly enable meaningful decreases in the 

obstacle clearance provisions required for ILS approaches due to flight technical error considerations. 

The range of localizer error for the advanced display concept subset was 0.03 dots. Note that the baseline 

minimum required level of localizer error as defined by the IFR and H-IFR pilot data for the ILS BRD

test herein is 0.39 dots.

Table 47: Mean Localizer Error (dots) SNK results for the ILS Approach for the IFR and H-IFR 

Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

PBG 10 0.07

SVS 15 0.09

SCFD 12 0.10

BRD 19 0.39*

Significance 0.82 1.00

*Note: This value is used herein as the baseline performance for an ILS Approach
73

Figure 18: Localizer Error, Segment 3 for ILS and VMC-Like Approaches, All Pilots

4.6.5 ILS and VMC-Like approach analysis, Segment 3, all pilots

Figures 16 and 17 show a visual comparison of localizer error results for the ILS and VMC-like 

approaches, respectively. Note that in Figure 17 a large increase in localizer error is shown for the SCFD 

display concept. This large increase in localizer error was due to the turning nature of the VMC-like 

approach and the general lack of turn anticipation of the SCFD display concept (even though desired 

flight path information was provided on the ND). An ANOVA was conducted on the localizer error data 

for all pilots for Segment 3 of the ILS and VMC-like approaches to compare and contrast each advanced 

display concept as well as to assess the capabilities of advanced display concepts to support advanced 

IMC operations. The ANOVA analysis indicated that approach (F(1, 111)=56.034, p<0.01) and display 

concept (F(2,111)=15.733, p<0.01) were highly significant and pilot type was not significant for this 

measure. The lack of effect due to pilot type is important and indicates that all types of pilots performed 

the ILS and VMC-like approaches similarly with the advanced display concepts with respect to localizer 

error. This result was not the case for the ILS evaluations of the BRD display concept, as shown in 

Figure 16. There was also a significant interaction between display concept and approach

(F(2,111)=12.071, p<0.01) for localizer error. As can be seen in Figures 16, and 17 and Table 48, 

localizer error results for the ILS approach were lower than for the VMC-like approach. 

0

0.2

0.4

0.6

0.8

1

1.2

ILS VMC-Like

SCFD

PBG

SVS

Localizer Error, rms dots

Approach
74

Table 48: Mean and Standard Deviation for Localizer Error (dots) for the ILS and VMC-Like 

Approach, All Pilots

Approach Mean Standard 

Deviation

ILS 0.09 0.04

VMC-Like 0.34 0.30

Figures 16 and 17 and Table 49 also show the effect of display concept for the ILS and VMC-like 

approaches. It can be seen that the higher (worst) results were for the SCFD. Results for the SCFD were 

approximately twice as large as those for the PBG and SVS display concepts. This result was most likely 

due to the lack of turn anticipation for the SCFD display concept that was an issue for Segment 3 of the 

VMC-like approach and Segment 2 of the ILS approach. Frequently, EPs would overshoot the base leg

and/or final approach and in doing so they exceeded the IFR PTS criteria.

Table 49: Mean Localizer Error (dots) SNK results, ILS and VMC-Like Approach for all Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

SVS 40 0.14

PBG 35 0.16

SCFD 36 0.34

Significance 0.623 1.000

The interaction between approach and display concept can be seen by comparing Figures 16 and 17 

and Table 50. Mean localizer error increased for all display concepts for the VMC-like approach as 

compared to the ILS approach. However, a much larger increase was experienced for the SCFD display 

concept than for the PBG or SVS display concepts. Localizer error increased by factors of 5.2, 2.9, and 

2.3 for the SCFD, PBG, and SVS display concepts, respectively, for the VMC-like approach as compared 

to the ILS approach. As stated previously, the lack of turn anticipation of the SCFD appears to be the 

reason for the much larger increase in localizer error for this display concept, yielding results that 

exceeded those generated by IFR and H-IFR pilots flying the BRD display concept for the ILS approach

(i.e., 0.39 dots).

Increases in localizer error due to approach type differences, as described by the ratio of localizer

error for the VMC-like to ILS approach, were much larger than lateral flight path error under the same 

conditions. This discrepancy between localizer error and lateral flight path error results was due to the 

fact that there were two turns included in the VMC-like approach occurring at different ranges to 

touchdown. In essence, localizer error can be described as range-weighted lateral flight path error with 

more emphasis placed on errors closer to touchdown. Lateral flight path errors during the turn to final for 

the VMC-Like approach generated larger localizer errors than for the turn to base leg.

Figures 16 and 17 and Table 49 indicate that, for the VMC-like approach, the increase in localizer 

error for the SVS display concept was smaller than for the PBG display concept. While this was not 

statistically significant, it does suggest that the increase of SA provided by the SVS terrain may actually 

help pilots negotiate the turns of the VMC-like approach. Overall, localizer error results would suggest

that the VMC-like approach is feasible, but only for the PBG and SVS display concepts.
75

Table 50: Mean and Standard Deviation Results for Localizer Error (dots) for the ILS and VMCLike Approach for all Pilots

Display 

Concept Approach Mean Standard Deviation

SCFD ILS 0.113 0.047

VMC-Like 0.584 0.421

PBG ILS 0.079 0.026

VMC-Like 0.233 0.085

SVS ILS 0.089 0.033

VMC-Like 0.202 0.084

4.6.6 Test condition analysis, Segment 3

An ANOVA was performed on localizer error for TC with post-hoc results presented in Table 51. 

Test Condition was revealed to be highly significant for localizer error results (F(12,220)=37.76, p<0.01). 

Post-hoc analysis only generated two unique subsets. The highest (worst) subset was TC 1, VFR 

approach with BRD display concept, which had significantly higher localizer error than all other display 

concept and approach combinations evaluated. This result is again not surprising since pilots used only 

the out-the-window view for navigation. There were no significant localizer error differences among the

other TCs and those mean localizer errors covered a range from 0.584 dots (TC 9) to 0.079 dots (TC 7). 

TC 7 (ILS approach with PBG display concept) localizer error was only 13.5% of the localizer error for 

TC 9 (VMC-like approach with SCFD display concept). Comparing TC 5 (ILS approach with BRD 

display concept) with the other TCs shows that all evaluations by all pilots of the advanced display 

concepts resulted in lower localizer error for the ILS and VMC-like approaches, except for TC 9 (VMClike approach with SCFD display concept). This result, again, is probably due to the lack of turn 

anticipation of the SCFD. Another substantial result was that even though IMC evaluations of the PBG 

and SVS display concept included VFR pilot data (TCs 7, 8, 10, and 11), the resulting localizer error was 

less than for the BRD display concept for the ILS with only IFR and H-IFR pilot data. Results for only 

the VFR pilots for the SVS display concept evaluations for the ILS and VMC-like approaches were also 

less than TC 5. The localizer error data suggest that advanced IMC operations, such as the VMC-like 

approach, are feasible with the PBG and SVS advanced display concepts tested, but not for the SCFD 

display concept. 
76

Table 51: Mean Localizer Error (dots) SNK results for Test Condition

Test Condition TC 

#

Number of 

Data Points

Subset

1 2

ILS Approach + PBG 7 16 0.079

ILS Approach + SVS 8 21 0.089

ILS Approach + SVS (VFR pilots) 12 6 0.092

ILS Approach + SCFD 6 18 0.113

VMC-Like Approach + SVS 11 19 0.202

VFR Approach + PBG 3 18 0.226

VMC-Like Approach + PBG 10 19 0.233

VMC-Like Approach + SVS (VFR pilots) 13 6 0.239

VFR Approach + SVS 4 18 0.260

ILS Approach + BRD (no VFR pilots) 5 19 0.391

VFR Approach + SCFD 3 19 0.536

VMC-Like Approach + SCFD 9 18 0.584

VFR Approach + BRD 1 24 4.036

Significance 0.888 1.000
77

4.7 Glideslope Error

Comparison of glideslope error results for the various display concepts and approaches is also 

valuable in a similar manner to localizer error (i.e., linear vertical error comparisons favor the advanced 

display concepts since their guidance cues are not affected by range to touchdown). In addition, 

glideslope error analysis and comparisons are valuable due to the linkages with traditional IMC 

operations. Glideslope error results for the four display concepts and two simulated-IMC approaches are 

presented in Figures 19, 20, 21, and 22 and Table 52. Note that the glideslope error results for the VFR 

approach are not plotted in Figure 22 for plotting clarity reasons.

Table 52: Glideslope Error (dots) Descriptive Statistics for all Pilots

Display 

Concept Approach Mean Standard Deviation

Number of 

Data Points

BRD VFR 1.24 0.88 24

ILS 0.89 0.66 30

Total 1.05 0.78 54

SCFD VFR 0.32 0.25 19

ILS 0.16 0.07 18

VMC-Like 0.30 0.17 18

Total 0.26 0.19 55

PBG VFR 0.33 0.14 18

ILS 0.24 0.11 16

VMC-Like 0.42 0.23 19

Total 0.34 0.18 53

SVS VFR 0.47 0.38 18

ILS 0.21 0.09 21

VMC-Like 0.32 0.11 19

Total 0.33 0.25 58

Total VFR 0.63 0.66 79

ILS 0.45 0.52 85

VMC-Like 0.35 0.18 56

Total 0.49 0.53 220

4.7.1 VFR approach, all pilots

As can be seen in Figure 19 and Table 52, the VFR approach evaluation of the BRD display concept

resulted in larger glideslope error values compared to the advanced display concepts. This result is most 

likely due to the nature of the EPs flight path navigation being driven totally by the out-the-window view 

for the BRD display concept along with altimeter checks at the turn to final from the base leg. Altimeter 

information included in the BRD display concept helped to keep the mean glideslope error to only 1.24 

dots. The apparent increases in glideslope error with decreasing range to touchdown was due to the fact 

that vertical flight path error remained relatively constant throughout the VFR approaches with the 

advanced display concepts as range decreased. An ANOVA conducted on the glideslope error results for 

the main effects of display concept and pilot type revealed highly significant results for display concept 

(F(3,79)=21.5, p<0.01) and non-significant results for pilot type (p>0.05). Subsequent post-hoc analysis 
78

resulted in two subsets, as shown in Table 53. As expected, the BRD was the sole member of the worst 

subset, and the advanced display concepts comprised the best subset. Note that glideslope error for the 

SVS display concept was 0.14 dots worse than the PBG. This may be attributed to the evaluation pilots 

frequently comparing the outside visual scene to that provided by the SVS terrain and paying slightly less 

attention to the guidance cues.

Table 53: Mean Glideslope Error (dots) SNK results for Display Concept for the VFR Approach

for all Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 19 0.32

PBG 18 0.33

SVS 18 0.47

BRD 24 1.235

Significance 0.55 1.000
79

Figure 19: VFR Approach, Glideslope Error, By Display Concept and Pilot Group

-3

-2

-1

0

1

2

3

VFR Approach, BRD, All Pilots

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

Glide Slope Error, dots

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

VFR Approach, PBG, All Pilots

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

Glide Slope Error, dots

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

VFR Approach, SCFD, All Pilots

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

Glide Slope Error, dots

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

0 1 2 3 4 5 6 7

VFR Approach, SVS, All Pilots

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

GlideSlopeD

Glide Slope Error, dots

Distance to Touchdown, nm

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

VFR Pilots

IFR Pilots

H-IFR Pilots
80

4.7.2 ILS approach, all pilots

For the ILS approach for all pilots, the glideslope error decreased for all display concepts compared 

to the VFR approach, as can be seen by comparing Table 54 with Table 53 and by comparing Figures 19 

and 20. Figure 20 shows glideslope error for the four display concepts for all pilots for the ILS approach. 

Note that glideslope error results for the BRD display concept occasionally exceeded the IFR PTS 

criteria. While most of the evaluations that exceeded the IFR PTS criteria were for VFR pilots flying 

with the BRD display concept, one was for an H-IFR EP. Figure 20 also shows that results for the 

advanced display concepts were all well within the IFR PTS criteria. A slight decrease in glideslope error

is also visible for the SVS compared to the PBG display concept, potentially indicating the effect of SVS 

terrain to improve glideslope error. An ANOVA conducted on the glideslope error results for the main 

effects of display concept and pilot type revealed highly significant results for display concept 

(F(3,85)=20.8, p<0.01) and non-significant results for pilot type (p>0.05). There were no significant 

interactions. The lowest glideslope error was for the SCFD display concept (0.16 dots) and the highest 

was for the BRD display concept (0.89 dots). The advanced display concepts had a range of glideslope

error results of approximately 0.16 to 0.24 dots and were included in the lowest (best) subset, while the 

BRD was the only member of the higher (worst) subset. This result was expected due to the integrated 

nature of the advanced display concepts along with their guidance cues. The inclusion of the VFR pilot 

data also degrades the results for the BRD display concept.

Table 54: Mean Glideslope Error (dots) SNK results for Display Concept, ILS Approach for all 

Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 18 0.16

SVS 21 0.21

PBG 16 0.24

BRD 30 0.89

Significance .779 1.000

While results were not statistically significant for pilot type, post-hoc analysis did produce two 

subsets as shown below in Table 55. There was a 0.27 dot difference in glideslope error for the VFR 

pilots compared to the mean for the H-IFR and IFR pilots for all displays tested. Figure 20 shows that 

much of the difference in glideslope error due to pilot type was likely due to the BRD display concept.

Table 55: Mean Glideslope Error (dots) SNK results for Display Concept, VFR Approach for all 

Pilots

Pilot Type Number of 

Data Points

Subset

1 2

H-IFR 27 0.32

IFR 29 0.39

VFR 29 0.62

Significance 0.475 1.000
81

Figure 20: ILS Approach, Glideslope Error, By Display Concept and Pilot Group

-3

-2

-1

0

1

2

3

ILS Approach, BRD, All Pilots

GlideSlopeDot(44)

Glide Slope Error, dots

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

ILS Approach, PBG, All Pilots

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

GlideSlopeDot(4

Glide Slope Error, dots

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

ILS Approach, SCFD, All Pilots

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

Glide Slope Error, dots

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

-3

-2

-1

0

1

2

3

0 1 2 3 4 5 6 7 8 9 10

ILS Approach, SVS, All Pilots GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

GlideSlopeDot(44

Glide Slope Error, dots

Distance to Touchdown, nm

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

VFR Pilots

IFR Pilots

H-IFR Pilots
82

4.7.3 VMC-Like approach

Unlike localizer error results, Figure 21 shows that glideslope error was fairly similar for all three 

advanced display concepts for the VMC-like approach. However, the worst results were for the PBG 

display concept with several excursions above the PTS criteria evident in Figure 21 during the final 

approach leg (approximately 1.8 nm to the 0.66 nm). An ANOVA conducted on the glideslope error data 

for the VMC-like approach for the main effects of display concept and pilot type revealed significant 

results for display concept (F(2,56)=3.52, p=0.04) and non-significant results for pilot type (p>0.05). 

There was no significant interaction between display concept and pilot type. Subsequent post-hoc 

analysis resulted in two subsets, as shown in Table 56. However, in this case the highest (worst) subsetgroup was comprised of only the PBG display concept, with the SVS display concept being grouped with 

the SCFD in the lowest (best) subset. Part of this result is due to the improved vertical flight path error 

control of the single-cue flight director compared to the HITS guidance discussed previously. However, 

for glideslope results, the inclusion of the SVS terrain with the HITS guidance significantly reduced 

glideslope error by 0.10 dots compared to the PBG display concept. This reduction of the glideslope error

resulted in the SVS display concept grouping with the SCFD. The fact that this is significant here as 

opposed to the vertical flight path error is likely due to the range-weighted nature of glideslope error

combined with the increased presence of the SVS terrain with runway models on final approach.

Vertical flight path errors generate larger glideslope errors as range to the runway touchdown zone 

decreases. As a result glideslope error is more sensitive to vertical error during the last several miles

during final approach than at the beginning of the approach. If results for vertical flight path error varied

similarly during the approach for all display concepts, then statistical analysis results for vertical flight 

path and glideslope errors would produce similar SNK groupings. However, as shown in Tables 84

(vertical flight path error) and 56 (glideslope error) the SVS display grouped in the worst group for 

vertical flight path error and in the best group for glideslope error. In order for this to occur, vertical 

flight path error results had to decrease during the approach for the SVS display concept more than the 

SCFD and PBG display concepts. The distribution of vertical flight path and glideslope errors can be 

seen in Figures 33 and 21. From Figure 33, it can be observed that vertical flight path error for the SVS 

display concept appears to become similar to the SCFD as range to touchdown decreases. Results for the 

PBG display concept appear to remain constant and even increase during the last 1.5 nm of the approach. 

From Figure 21, it can be seen that the glideslope error results for the SVS display closely resemble those 

for the SCFD and remain within IFR PTS criteria throughout the approach. Figure 21 also reveals that 

results for the PBG display concept not only increased but exceeded IFR PTS criteria during the last 1.5 

nm of the approach. It is considered that the ability to integrate information is different for the VFR, IFR, 

and H-IFR pilots groups. Pilot training associated with achieving an IFR pilot rating combined with 

thousands of hours of actual IMC flight time develops the ability to integrate information provided by 

separate displays. While PFDs provide information in a more integrated manner than the BRD, the 

background SVS terrain further facilitates the pilots’ ability to mentally process the data. Results for pilot 

type were not significant for this analysis, however, from Figure 21 it can be seen that the VFR pilots and 

one of the IFR pilots were having difficulty remaining within the PTS criteria with the PBG display 

concept. Glideslope error results for the SVS display concept reveal similar performance for all pilot 

groups. Again, it is believed that the SVS terrain serves as an integrator of other information allowing all 

pilot groups to achieve not only superior situation awareness but also improved performance.
83

Table 56: Mean Glideslope Error (dots) SNK results for Display Concept, VMC-Like Approach

for all Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 18 0.30

SVS 19 0.32

PBG 19 0.42

Significance 0.675 1.000
84

Figure 21: VMC-Like Approach, Glideslope Error, By Display Concept and Pilot Group

-2

-1

0

1

2

3

4

VMC-Like Approach, PBG, All Pilots

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide

Glide Slope Error, dots

IFR PTS Criterion

Segment 3 Segment 2

IFR PTS Criterion

-2

-1

0

1

2

3

4

0 1 2 3 4 5 6 7

VMC-Like Approach, SVS, All Pilots

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

GlideSl

Glide Slope Error, dots

Distance to Touchdown, nm

IFR PTS Criterion

Segment 3 Segment 2

IFR PTS Criterion

-2

-1

0

1

2

3

4

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glid

Glide Slope Error, dots

IFR PTS Criterion

Segment 3 Segment 2

IFR PTS Criterion

VMC-Like Approach, SCFD, All Pilots

VFR Pilots

IFR Pilots

H-IFR Pilots
85

4.7.4 ILS approach, Segment 3, no VFR pilots

Results for glideslope error were analyzed without the VFR pilot data, enabling a more direct 

comparison of the four display concepts within current IMC operations. Results from an ANOVA 

revealed that display concept was highly significant (F(3,56)=16.68, p<0.01) and that pilot type (i.e., IFR 

or H-IFR) was not significant (p>0.05). In addition, there were no significant interactions between pilot 

type and display concept. Subsequent post-hoc analysis produced two unique subsets, as shown in Table 

57. With a mean of 0.76 dots error, the BRD display concept had significantly higher glideslope error

than the advanced display concepts. There were no significant glideslope error differences among the 

SCFD, SVS, and PBG concepts. The average mean glideslope error for the advanced display concepts 

was approximately one quarter of glideslope error for the BRD display concept. These results are 

visually supported by Figure 20. Again, as with localizer error, the reduction in glideslope error for the 

advanced display concepts may support reductions in the amount of airspace required for ILS approaches

with the advanced display concepts. As discussed previously for vertical flight path error results, while 

other limitations may apply, obstacle clearance margins could be reduced by a factor of four. Note that 

the baseline minimum required level of glideslope error defined by the IFR and H-IFR BRD pilot data for 

the test herein is 0.76 dots.

Table 57: Mean Glideslope Error (dots) SNK results for the ILS Approach, IFR and H-IFR Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 12 0.14

SVS 15 0.20

PBG 10 0.21

BRD 19 0.76*

Significance 0.759 1.000

*Note: This value is used herein as the baseline performance for an ILS Approach
86

Figure 22: Glideslope Error For Segment 3, for Approaches 2 and 3, All Pilots

4.7.5 ILS and VMC-Like approach analysis, Segment 3, all pilots

Comparison of Figures 20 and 21 provides a visual indication of the effect of approach and display 

concept for the advanced display concepts and the ILS and VMC-like approaches. From these figures, it 

can be seen that glideslope error increased for all the advanced display concepts, with the largest increase 

for the PBG display concept. While the HITS guidance concept provided a good blending of lateral and 

vertical flight path error performance overall, the VMC-like evaluation of the PBG display concept 

resulted in higher levels of glideslope error with several evaluations exceeding the IFR PTS criteria. 

Figure 21 appears to indicate that the presence of the SVS background limited the increase in glideslope

error for the SVS display concept while flying the VMC-like approach from 2.0 nm to the end of the 

approach.

An ANOVA was performed on the glideslope error data for all pilots for segment 3, for the ILS and 

VMC-like approaches for the main effects of pilot type, display concept, and approach to assess the 

capabilities of advanced display concepts to support advanced IMC operations. The ANOVA analysis 

revealed that approach (F(1,111)=60.084, p<0.01) was highly significant and that display concept

(F(2,111)=4.729, p=0.011) and pilot type (F(2,111)=3.220, p=0.045) were also significant but at lower 

levels. Unlike most of the previous analysis for the ILS and VMC-like approaches, pilot type was 

significant for this analysis. There were no significant interactions. Subsequent post-hoc analysis 

generated two unique subsets for display concept, as shown in Table 58, but no subsets for pilot type.

The PBG display concept had significantly higher glideslope error compared to the SVS and SCFD 

display concepts. There were no significant glideslope error differences between the SVS and SCFD 

0

0.2

0.4

0.6

0.8

1

1.2

ILS VMC-Like

SCFD

PBG

SVS

Glide Slope Error, rms dots

Approach
87

concepts. This result reflects the inclusion of SVS terrain information in the SVS display concept, which

appears to have significantly reduced glideslope error compared to the PBG display concept. Much of 

this result was due to the superior performance, or lack of substantial degradation of performance, for the 

VFR pilots for the VMC-like approach with the SVS display concept. The integration of SVS terrain 

with HITS guidance symbology most likely produced an environment where the VFR evaluation pilots

were able to perform the VMC-like approach as well as the other pilot groups.

During initial system checkout, several pilots evaluated the research displays during the checkout 

flights to determine if the flight test implementation was similar to what was implemented in the 

simulation. Pilots were asked to provide qualitative inputs regarding flight path tracking error 

performance and workload. Flight path and angular errors gains of the SCFD were adjusted to achieve 

representative performance. The gains used for the research flights are listed in Appendix A.

As implemented for this flight test, the SCFD provided significantly improved glideslope error 

results compared to the PBG display concept. This was due to the single-cue flight director drive 

equations combined with the prominent display of pitch attitude commands on the SCFD display that 

provided a large emphasis on vertical error control. Results for glideslope error for the PBG and SVS 

display concepts could potentially be improved through adjustment of the tadpole guidance cue (see 

Appendix B for description of guidance cue). For this study, the tadpole guidance cue was located along 

the desired flight path 15 seconds in front of ownship. This timing parameter was established from work 

with large transport-type aircraft (Reference 8) and could potentially be reduced for GA applications. 

Having the guidance cue located closer to ownship would help to reduce flight path error. Overall, the 

guidance cue of the HITS symbology was designed to provide a good blending of lateral and vertical 

error control, which it accomplished. It provided much better results than the SCFD, when considering 

combined lateral and vertical errors, SA, and TLX, and DRR results.

Table 58: Mean Glideslope Error (dots) SNK results for the ILS and VMC-Like Approaches for all 

Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 36 0.28

SVS 40 0.31

PBG 35 0.41

Significance 0.334 1.000

The interactions between display concept and pilot type are illustrated in Figures 20 and 21 and 

Table 59. It can be seen that the VFR pilots performed worse in terms of glideslope error than the IFR 

and H-IFR pilots for all display concepts with VFR pilot data being worse than the data from the other 

two pilot groups. However, the level of increase of glideslope error for the VFR pilots compared to the 

average of the means of the IFR and H-IFR pilots data was not the same for all display concepts, with 

increases of 0.12, 0.09, and 0.04 dots for the SCFD, PBG, and SVS display concepts, respectively. The 

increase in glideslope error for the VFR pilots for the SVS display concept was only approximately one 

third to one half of the increase for the SCFD and PBG display concepts. This lower level of increase for 

the SVS display concept is likely due to the presence of the SVS terrain providing the VFR pilots with SA

unavailable for the other advanced display concepts due to the simulated-IMC test conditions.
88

Table 59: Statistical Interaction Results for Glideslope Error (dots) for Display Concept and Pilot 

Type for the ILS and VMC-Like approaches

Display 

Concept Pilot Type Mean Standard Deviation

Number of 

Data Points

SCFD VFR 0.36 0.21 12

IFR 0.25 0.22 12

H-IFR 0.23 0.17 12

Total 0.28 0.20 36

PBG VFR 0.47 0.39 12

IFR 0.41 0.25 10

H-IFR 0.36 0.20 13

Total 0.41 0.29 35

SVS VFR 0.34 0.17 12

IFR 0.28 0.16 14

H-IFR 0.32 0.14 14

Total 0.31 0.15 40

Total VFR 0.39 0.27 36

IFR 0.31 0.21 36

H-IFR 0.30 0.18 39

Total 0.33 0.23 111

Table 60 shows the glideslope error results for the advanced display concepts for each IMC 

approach. As stated previously, the largest increase in glideslope error was for the PBG display concept,

which experienced a 0.29 dot increase for the VMC-like approach vs. the ILS approach. The smallest

increase in glideslope error was for the SVS display concept with a 0.19 dot increase for the VMC-like 

approach. This positive result for the SVS display concept glideslope error was primarily because the 

VFR pilots performed much better using the SVS display concept for the VMC-like approach than they 

did using other display concepts. Overall, glideslope errors for all display concepts for the VMC-like 

approach were less than the ILS evaluation of the BRD display concept for the IFR and H-IFR pilots (i.e.,

0.76 dots), indicating that the VMC-like approach may be feasible for these display concepts.
89

Table 60: Statistical Interaction Glideslope Error (dots) Data for Display Concept and Approach

for the ILS and VMC-Like Approaches

Display 

Concept Approach Mean Standard Deviation

Number of 

Data Points

SCFD ILS 0.16 0.08 18

VMC-Like 0.39 0.23 18

Total 0.28 0.20 36

PBG ILS 0.25 0.11 16

VMC-Like 0.55 0.33 19

Total 0.41 0.29 35

SVS ILS 0.22 0.09 21

VMC-Like 0.41 0.14 19

Total 0.31 0.15 40

Total ILS 0.21 0.10 55

VMC-Like 0.45 0.25 56

Total 0.33 0.23 111

4.7.6 Test condition analysis, Segment 3

An ANOVA was performed on glideslope error for TC with post-hoc results presented in Table 61.

TC was highly significant for glideslope error results (F(12,220)=14.266, p<0.01). Post-hoc analysis of 

the glideslope error generated two unique subsets. The VFR approach with the BRD (TC 1) had 

significantly higher glideslope error than all other approach and display concept combinations tested. 

Most likely, this result is because the pilots used the out-the-window method of navigation. No 

significant differences were seen in this test. All of the other TCs were grouped into one subset, and they 

had a range of mean RMS glideslope errors of approximately 0.6 dots (from 0.163 dots to 0.760 dots). 

TC 6 (ILS approach with SCFD display concept) yielded the lowest glideslope error of 0.16 dots. Test 

Condition 5 (ILS approach with the BRD display concept, no VFR pilots) had the highest glideslope error

of 0.76 dots for this subset, which was 4.7 times larger than the result for TC 6. Overall, all IMC 

evaluations of the advanced display concepts (TC 6, 7, 8, 9, 10, and 11) yielded lower glideslope errors 

than TC 5, including the IMC VFR-only pilot evaluations of the SVS display concept (TC 12 and 13). 

This result indicates that the advanced display concepts provided all pilot types the ability to control 

glideslope error as well or better than the current baseline standard (BRD display concept for the ILS 

approach for IFR and H-IFR pilots) employed in this test. The VFR approach results for the advanced 

display concepts (i.e., TC 2, 3, and 4) were also as good or better than TC 5, even though EPs were 

performing out-the-window navigation combined with cross checking from their displays.
90

Table 61: Mean Glideslope Error (dots) SNK results for Test Condition

Test Condition TC # Number of 

Data Points

Subset

1 2

ILS Approach + SCFD 6 18 0.163

ILS Approach + SVS 8 21 0.220

ILS Approach + PBG 7 16 0.254

ILS Approach + SVS (VFR pilots) 12 6 0.272

VMC-Like Approach + SCFD 9 18 0.388

VMC-Like Approach + SVS (VFR pilots) 13 6 0.406

VMC-Like Approach + SVS 11 19 0.414

VFR Approach + SCFD 2 19 0.416

VFR Approach + PBG 3 18 0.426

VMC-Like Approach + PBG 10 19 0.546

VFR Approach + SVS 4 18 0.629

ILS Approach + BRD (no VFR pilots) 5 19 0.760

VFR Approach + BRD 1 24 1.840

Significance 0.177 1.000
91

4.8 Indicated Airspeed Error

Airspeed error is valuable to describe the differences between the display concepts, approaches, and 

pilot types. It can be viewed as a consistent measure of pilot awareness, since the task of controlling 

airspeed is similar from one approach to the next. Airspeed results are presented in Figures 23, 24, and 25

for the VFR, ILS, and VMC-like approaches, respectively. Figure 26 along with Tables 62 and 63 

present mean RMS airspeed error results. Recall that for Segment 2, pilots were asked to maintain 100 

kts and then decelerate and change speed to 90 kts for Segment 3. Due to limitations in data processing, 

airspeed errors are only presented separately for Segment 2 and Segment 3 of the approaches evaluated. 

Table 62: IAS Error (kts) Descriptive Statistics for Segment 2 for all Pilots

Display 

Concept Approach Mean Standard Deviation

Number of 

Data Points

BRD VFR 4.6 1.5 24

ILS 5.0 2.3 30

Total 4.9 2.0 54

SCFD VFR 4.7 1.9 19

ILS 5.8 2.9 18

VMC-Like 4.6 2.4 18

Total 5.0 2.4 55

PBG VFR 3.9 2.3 18

ILS 4.4 3.2 16

VMC-Like 4.5 2.1 19

Total 4.3 2.5 53

SVS VFR 3.6 1.9 18

ILS 4.5 1.6 21

VMC-Like 4.6 2.6 19

Total 4.3 2.1 58

Total VFR 4.3 1.9 79

ILS 5.0 2.5 85

VMC-Like 4.6 2.3 56

Total 4.6 2.4 220
92

Table 63: IAS Error (kts) Descriptive Statistics for Segment 3 for all Pilots

Display 

Concept Approach Mean StandardDeviation

Number of 

Data Points

BRD VFR 3.6 1.4 24

ILS 5.9 2.5 30

Total 4.9 2.3 54

SCFD VFR 5.6 2.2 19

ILS 6.3 2.9 18

VMC-Like 6.1 2.2 18

Total 6.0 2.4 55

PBG VFR 5.0 1.6 18

ILS 5.6 2.1 16

VMC-Like 6.1 2.9 19

Total 5.6 2.3 53

SVS VFR 5.7 2.1 18

ILS 5.6 2.1 21

VMC-Like 5.4 1.6 19

Total 5.6 1.9 58

Total VFR 4.9 2.0 79

ILS 5.8 2.4 85

VMC-Like 5.9 2.3 56

Total 5.5 2.3 220

4.8.1 VFR approach all pilots, S2 and 3

As can be seen from Figure 23 and Tables 62 and 63, the difference in airspeed error for the BRD 

and the advanced display concepts was much less pronounced than for other data measures discussed 

previously. The BRD display concept produced less error than the other display concepts as shown in 

Figure 23 where the pilots were able to maintain airspeed well within the +/-10 kts PTS criterion most of 

the time. The difference in IAS error between the BRD and advanced display concepts was likely related 

to the manner in which airspeed was presented to the pilot and also the content and use of that 

information for the advanced display concepts. For the BRD display concept, airspeed was a simple 

gauge that pilots looked at for current airspeed (position) and trend information (rate of movement). For 

the advanced display concepts, current airspeed was presented digitally on the airspeed tape, which also 

provided airspeed rate information (tape movements, acceleration trend line). The acceleration arrow on 

the velocity vector also provided airspeed rate information for the PBG and SVS display concepts (see 

Figure 6). ANOVAs conducted on Segment 2 and 3 IAS errors for the main effects of display concept 

and pilot type resulted in non-significant results for Segment 2, but highly significant results for display 

concept (F(3,79)=6.65, p<0.01) for Segment 3. Subsequent post-hoc results are shown in Table 64, where 

it can be seen that two unique subsets were formed. The lowest (best) subset was comprised only of the 

BRD display concept with the advanced display concepts being included within the highest (worst) 

subset. IAS error was 1.4 kts less for the BRD display concept compared to the PBG, and 2.2 kts less 

compared to the SVS display concept. These results show that the round-dial method of airspeed 

presentation may be superior to the integrated digital readouts and tapes of the advanced display concepts 
93

for the VFR approach where a relatively low number of disparate pieces of information need to be 

integrated (i.e., out-the-window-view, airspeed, and altitude).

Table 64: Mean IAS Error (kts) SNK results for the VFR Approach, Segment 3 for all Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

BRD 24 3.6

PBG 18 5.0

SCFD 19 5.6

SVS 18 5.7

Significance 1.000 0.37
94

Figure 23: VFR Approach, IAS Performance for All Pilots and Display Concepts

70

80

90

100

110

120 VFR Approach, BRD, All Pilots

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IndicatedAirspeedKn

IAS Error, kts

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

70

80

90

100

110

120

VFR Approach, PBG, All Pilots

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IndicatedAirsp

IAS Error, kts

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

70

80

90

100

110

120

VFR Approach, SCFD, All Pilots

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IndicatedAirspee

IAS Error, kts

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

70

80

90

100

110

120

0 1 2 3 4 5 6 7

VFR Approach, SVS, All Pilots

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IAS Error, kts

Distance to Touchdown, nm

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

VFR Pilots

IFR Pilots

H-IFR Pilots
95

4.8.2 ILS approach, all pilots, Segments 2 and 3

Figures 23 and 24 and Tables 62 and 63 provide a visual comparison of airspeed control 

performance for the VFR and ILS approaches, respectively. For the ILS approach for all pilots, IAS error 

increased for all display concepts except for the SVS display concept compared to the VFR approach. 

The largest increase was for the BRD display concept with an increase of 2.3 kts (Segment 3). The 

advanced display concepts led to much lower increases of IAS error for the ILS approach compared to the 

VFR approach. IAS error increased 0.7 kts and 0.4 kts for the SCFD and PBG display concepts, 

respectively. IAS error actually decreased for the SVS display concept for the ILS approach, but only 

slightly, 0.1 kts. The fact that IAS error increased much more for the BRD display concept than for the 

advanced display concepts indicates that while the IAS gauge potentially provided improved airspeed 

awareness compared to the integrated airspeed tapes of the advanced display concepts, it was integrated 

within the BRD display concept, resulting in increased overall workload. This resulted in a larger effect 

of approach in IAS error for this display concept. An ANOVA conducted on the IAS error for Segments 

2 and 3 revealed that pilot type was significant for Segment 2 (F(2,85)=4.76, p=0.012) and Segment 3 

(F(2,85)=4.45, p=0.016) and non-significant for display concept. The interaction between display 

concept and pilot type was not statistically significant. Tables 65 and 66 show the results from the SNK 

post-hoc analysis and that the VFR pilots had significantly worse IAS error than the IFR or H-IFR pilots. 

While the results presented in Tables 65 and 66 indicate a significant difference between the VFR and 

IFR/H-IFR pilots, the increased IAS error for the VFR pilots was not the same for all display concepts. 

Table 67 provides the IAS error increase for the VFR pilots compared to the average of the IFR and HIFR pilots for all of the display concepts. From Table 67, it can be seen that the increase of IAS error for 

the VFR pilots was lower for the SVS display concept compared to all the others, which could reflect the 

lower workload and increased situation awareness of the SVS display concept. While the statistical 

analysis revealed no significant interactions between display concept and pilot type, results provided in 

Table 67 do show a trend in the data that indicates that the degraded VFR-pilot IAS error control was 

mitigated for the SVS display concept. 

Table 65: Mean IAS Error (kts) SNK results for the ILS Approach, Segment 2 for all Pilots

Pilot Type Number of 

Data Points

Subset

1 2

H-IFR 27 3.9

IFR 29 4.9

VFR 29 6.2

Significance 0.115 1.000

Table 66: Mean IAS Error (kts) SNK results for the ILS Approach, Segment 3 for all Pilots 

Pilot Type Number of 

Data Points

Subset

1 2

IFR 29 5.1

H-IFR 27 5.7

VFR 29 6.8

Significance 0.264 1.000
96

Figure 24: ILS Approach, IAS Performance for All Pilots and Display Concepts

70

80

90

100

110

120 ILS Approach, BRD, All Pilots

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IAS Error, kts

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

70

80

90

100

110

120 ILS Approach, PBG, All Pilots

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IndicatedAirspeedK

IAS Error, kts

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

70

80

90

100

110

120 ILS Approach, SCFD, All Pilots

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IndicatedAirspeedKnot(12)

IAS Error, kts

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

70

80

90

100

110

120

0 1 2 3 4 5 6 7 8 9 10

ILS Approach, SVS, All Pilots

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IndicatedAirspe

IAS Error, kts

Distance to Touchdown, nm

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

VFR Pilots

IFR Pilots

H-IFR Pilots
97

Table 67: Increase in mean IAS Error (kts) for the VFR Pilots Compared to the Average of the 

IFR and H-IFR Pilots for the ILS Approach, Segments 2 and 3 

Display 

Concept

Segment

2 3

BRD 1.9 1.9

SCFD 2.4 1.0

PBG 3.5 1.6

SVS 0.2 0.9

4.8.3 ILS approach, Segment 3, no VFR pilots

Results for IAS error were analyzed without the VFR pilot data, enabling a more direct comparison 

of the four display concepts within current IMC operations. Results from an ANOVA with display 

concept and pilot type as main effects revealed no significant (p>0.05) differences for IAS error for either 

the main effects or their interactions for segment 2 or segment 3. Although not statistically significant, HIFR pilots controlled IAS approximately 1.0 kts better than the IFR pilots for Segments 2 and 3. Table 68 

shows the mean IAS error for the ILS approach without the VFR pilot data. The lowest IAS error was for 

the PBG display concept for both segments and the worst was for the SCFD display concept. Note that 

IAS error for Segment 3 for the BRD display concept was 5.2 kts, which establishes the baseline 

minimum required level for ILS approaches for the data herein. Unlike previous measures, the BRD 

display concept did not produce the worst performance. While the PBG and SVS display concepts were 

within +/-4% of the BRD display concept result, the IAS error for the SCFD was 15% higher than the 

BRD display concept. 

Table 68: Mean IAS Error (kts) for Segment 2 and Segment 3 of the ILS Approach, IFR and HIFR Pilots

Display 

Concept

Number of 

Data Points

Mean 

Segment 2

Mean 

Segment 3

PBG 10 3.1 5.0

BRD 19 4.5 5.2*

SVS 15 4.7 5.3

SCFD 12 5.0 6.0

*Note: This value is used herein as the baseline performance for an ILS Approach

4.8.4 VMC-Like approach, all pilots, Segments 2 and 3

An ANOVA conducted on the IAS error results for the main effects of display concept and pilot type 

revealed no statistically significant results for either Segment 2 or Segment 3. Table 69 provides IAS 

error results for the three advanced display concepts for the VMC-like approach. 
98

Table 69: IAS Error (kts) for Segment 2 and Segment 3 of the VMC-Like Approach for All Pilots

Display 

Concept

Number of 

Data Points

Mean 

Segment 2

Mean 

Segment 3

PBG 19 4.5 6.1

SVS 19 4.6 5.4

SCFD 18 4.6 6.1

4.8.5 ILS and VMC-Like approach analysis, Segment 3, all pilots

Figure 25 provides airspeed results for the VMC-like approach for the advanced display concepts. 

Figures 24 and 25 provide a visual comparison of the pilots’ ability to control airspeed for the ILS and 

VMC-like approaches, respectively. An ANOVA was conducted on the IAS error data for all pilots for 

Segment 3 of the ILS and VMC-like approaches for the main effects of pilot type, display concept, and 

approach to assess the capabilities of advanced display concepts to support advanced IMC operations. 

There were no significant (p>0.05) main effects or interaction effects for IAS error. However, the highest 

(worst) IAS error for these two approaches was for the VFR pilots (6.3 kts), with the IFR pilots and HIFR pilots generating the same IAS error (5.7 kts). While this result was not statistically or operationally

significant, it appears to be a trend in the data and reflects pilot training and hours of experience. Overall, 

the VFR pilots’ IAS error was 10% worse than the other two pilot groups. With regard to display 

concept, the worst (largest) IAS error was for the SCFD (6.3 kts) while the best (smallest) was for the 

SVS (5.5 kts). This result is supported by Figure 25, which shows that the airspeed control for the SCFD 

display concept would exceed the PTS criteria more often than for the PBG or SVS display concepts. 

The IAS error for the PBG display concept (5.8 kts) was very close to the SVS display concept. This 

result implies that the SA provided by the SVS terrain, which yields no information regarding airspeed, 

did not affect the pilots’ ability to control airspeed. The SVS display concept decreased IAS error by 

13% compared to the SCFD for the ILS and VMC-like approaches. For all advanced display concepts, 

the average IAS error for each approach (i.e., ILS or VMC-Like) was virtually identical at 5.9 kts. This 

implies that any increase in workload due to the VMC-like approach was not reflected at all in the 

airspeed error data.

Table 70 provides a description of the IAS errors for the advanced display concepts with the VFR 

pilot data included compared to the ILS BRD display concept baseline (i.e., BRD display concept for the 

ILS approach for the IFR and H-IFR pilots). This comparison is presented in order to discuss the 

capability of the advanced display concepts to accommodate low-time VFR pilots and enable advanced 

IMC approaches. As can be seen in Table 70, the largest increase in the IAS error was for SCFD display 

concept, which was 22.9% and 18.2% higher than the baseline for the ILS and VMC-like approaches, 

respectively. The smallest increase in IAS error was for the SVS display concept, which was only 5.6% 

and 5.4% higher than the baseline error for the ILS and VMC-like approaches, respectively. The fact that 

the SVS display concept was the only one to limit increases in IAS error for VMC-like approach suggests

that this display concept was the only one of those tested that could potentially enable advanced IMC 

operations for virtually all currently certificated pilots. While none of these results were statistically 

significant, they do show trends in the data that support workload and situation awareness results 

discussed previously.
99

Table 70: Percent IAS Error Increase Compared to ILS Evaluation of BRD Display Concept (IFR 

and H-IFR Pilots) for Segment 3 for ILS and VMC-Like Approaches (all Pilots)

Display 

Concept

Approach

ILS VMC-Like

SCFD 22.9% 18.2%

PBG 8.3% 18.2%

SVS 5.6% 5.4%
100

Figure 25: VMC-Like Approach, IAS Performance for all Pilots and Display Concepts

70

80

90

100

110

120

VMC-Like Approach, PBG, All Pilots

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

Indicat

IAS Error, kts

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

70

80

90

100

110

120

0 1 2 3 4 5 6 7

VMC-Like Approach, SVS, All Pilots

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

Indicate

IAS Error, kts

Distance to Touchdown, nm

Segment 3 Segment 2

IFR PTS Criterion

IFR PTS Criterion

70

80

90

100

110

120

Indic

Indic

Indic

Indic

Indic

Indic

Indic

Indic

Indic

Indic

Indic

Indic

Indic

Indic

Indic

Indic

Indic

Indic

IAS Error, kts

Segment 3

VMC-Like Approach, SCFD, All Pilots

IFR PTS Criterion

IFR PTS Criterion

Segment 2

VFR Pilots

IFR Pilots

H-IFR Pilots
101

Figure 26: IAS Error for Segment 3, All Pilots, All Approaches

4.8.6 Test condition analysis, Segment 3

A subsequent analysis was performed on IAS error based on TC, with post-hoc results presented in 

Table 71. As previously performed for other analyses, VFR pilot data was not included for TC 5 (BRD 

display concept for the ILS approach) in order to provide a more comprehensive measure with strong ties 

to current operations. Test Condition was highly significant for IAS error results (F(12,220)=2.73, 

p<0.01). As can be seen in Table 71, two overlapping subsets were formed from SNK post-hoc analysis. 

The subset with the lowest IAS error was comprised of TC 1 (VFR approach with BRD display concept), 

TC 3 (VFR approach with PBG display concept), and TC 5 (ILS approach with BRD display concept, 

IFR and H-IFR pilots only). The result is not surprising and reflects the low-workload environment of the 

VFR approach and superior presentation of airspeed information to the pilot afforded by the BRD display 

concept. Results for the VFR approach for the PBG display concept being included in the low- and highIAS error groups is a little surprising and is potentially due to the lack of SVS terrain background for this 

display concept. During the research evaluations, pilots would periodically check and verify that the SVS 

terrain inside matched the terrain outside and may have become slightly distracted. This distraction was 

not there for the PBG display concept, since there was no terrain presented on the PFD. TC 5 was also 

included in the low- and high-IAS error groups even though there was a 1.6 kts increase compared to TC

1. Note that the increase in airspeed error due to outside visibility was smaller for the advanced display 

concepts than for the BRD. The highest IAS error subset was comprised of all the other TCs (i.e., 2 

through 13). As discussed previously, the ILS BRD baseline condition (i.e., TC 5) provided lower 

airspeed error than the other TCs in simulated IMC, but these differences were not significant. The

difference between TC 5 and SVS IMC evaluations (i.e., TC 8 and 11) is smaller compared to differences 

from the other advanced display concepts. It also needs to be recognized that the advanced display 

0

2

4

6

8

10

VFR ILS VMC-Like

BRD

SCFD

PBG

SVS

Indicated Airspeed Error, rms kts

Approach
102

concepts included the VFR pilot data, whereas TC 5 did not. Results for the VFR pilot evaluations of the 

SVS display concept for the ILS and VMC-like approaches were higher than TC 5, but not significantly. 

This implies that advanced IMC operations may be possible for the advanced display concepts.

Table 71: Mean IAS Error (kts) SNK results for TC

Test Condition TC # Number of 

Data Points

Subset

1 2

VFR Approach + BRD 1 24 3.6

VFR Approach + PBG 3 18 5.0 5.0

ILS Approach + BRD (no VFR pilots) 5 19 5.2 5.2

VMC-Like Approach + SVS 11 19 5.4

ILS Approach + SVS 8 21 5.6

ILS Approach + PBG 7 16 5.6

VFR Approach + SCFD 2 19 5.6

VFR Approach + SVS 4 18 5.7

VMC-Like Approach + SVS (VFR pilots) 11 6 5.7

VMC-Like Approach + SCFD 9 18 6.1

VMC-Like Approach + PBG 10 19 6.1

ILS Approach + SVS (VFR pilots) 12 6 6.2

ILS Approach + SCFD 6 18 6.3

Significance 0.054 0.686

4.9 Lateral Flight-Path Error

Lateral flight path error results for the four display concepts and three approaches are presented in 

Table 72. Figures 27, 28, and 29 show real-time lateral flight path error for the VFR, ILS, and VMC-like 

approaches, respectively. Figure 30 illustrates RMS lateral flight path error results for the ILS and VMClike approaches. Error results for the VFR approach evaluation of the BRD were too large to plot 

conveniently as they were approximately five times greater than other results.
103

Table 72: Lateral Flight-Path Error (ft) Descriptive Statistics for all Pilots 

Display 

Concept Approach Mean Standard Deviation

Number of 

Data Points

BRD VFR 1096.4 501.4 24

ILS 624.1 428.2 30

Total 834.0 515.4 54

SCFD VFR 141.2 117.1 19

ILS 140.1 95.5 18

VMC-Like 138.6 84.2 18

Total 140.0 98.4 55

PBG VFR 70.2 36.5 18

ILS 73.8 35.0 16

VMC-Like 80.2 38.2 19

Total 74.9 36.2 53

SVS VFR 79.7 54.2 18

ILS 72.9 30.7 21

VMC-Like 70.9 36.8 19

Total 74.4 40.6 58

Total VFR 401.2 540.9 79

ILS 281.9 361.8 85

VMC-Like 95.8 63.3 56

Total 277.4 411.6 220

4.9.1 VFR approach

Evaluation pilots were instructed to fly the VFR approach and cross check their out-the-window 

navigation with the guidance and navigation information provided from the ND and PFD. As can be seen

from Figures 27 and Table 72, lateral flight path error for the VFR approach for the BRD display concept 

was approximately an order of magnitude larger than for the advanced display concepts, which is not an 

unexpected result. An ANOVA performed on the lateral flight path error for the main effects of display 

concept and pilot type resulted in highly significant results for display concept (F(3, 79)=72.0, p<0.01) 

and non-significant results for pilot type (p>0.05). Subsequent post-hoc analysis produced two subsets, as 

shown in Table 73. For the VFR approach with the BRD display concept, evaluation pilots relied 

completely on out-the-window navigation techniques to perform the approach, which resulted in a mean 

RMS lateral error of 1,096 ft. The data for the VFR approach for the BRD display concept exhibits some 

unusual characteristics due to the method of lateral flight path error calculation.
104

Figure 27: VFR Approach Lateral Error for Each Display, Each Pilot Group

-6000

-4000

-2000

0

2000

4000

6000

VFR Approach, BRD, All Pilots ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

ComputedLatera

Lateral Error, ft

Segment 3 Segment 2

-6000

-4000

-2000

0

2000

4000

6000

VFR Approach, PBG, All Pilots

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

Lateral Error, ft

Segment 3 Segment 2

-6000

-4000

-2000

0

2000

4000

6000

VFR Approach, SCFD, All Pilots

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

ComputedLateralP

Lateral Error, ft

Segment 3 Segment 2

-6000

-4000

-2000

0

2000

4000

6000

0 1 2 3 4 5 6 7

VFR Approach, SVS, All Pilots

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

ComputedLate

Lateral Error, ft

Distance to Touchdown, nm

Segment 3 Segment 2

VFR Pilots

IFR Pilots

H-IFR Pilots
105

During the VFR approaches with the BRD display concept, pilots would sometimes cut the corner 

from the downwind leg to the base leg and again from the base leg to final approach, which resulted in a 

tighter approach than desired. If the aircraft became closer to the final approach leg than the downwind 

leg, the lateral flight path error calculation reference would jump from the downwind leg to the final 

approach leg. However, since the aircraft was still flying a track angle closer to the downwind leg than 

final approach, the sign on the error would jump from negative to positive. The primary effect was that 

the magnitude of the resulting flight path error was less than it should have been. For example, if the EP 

departed prematurely from the downwind leg and had a lateral error of 0.8 nm, the recorded lateral error 

would have been approximately 0.6 nm since that point may have been referenced to the final approach 

course. While this does bring into question the accuracy of the lateral error data for this approach with 

the BRD display concept, the data are considered useable for the intent of this report to approximately 

describe the ability of pilots to fly a specific VFR approach path with the BRD display concept. 

For the advanced display concepts, EPs had several other sources of guidance and navigation to work 

with during the VFR approach and were able to achieve much lower levels of lateral flight path error as

can be seen in Figure 27. The SCFD reduced lateral flight path error to 141.2 ft. The PBG and SVS 

display concepts reduced lateral flight path error further to 70.2 and 79.7 ft, respectively. Note that the 

increased lateral flight path error for the SCFD, compared to the PBG and SVS display concepts, 

although not significant, was likely associated with the lack of turn anticipation and flight path awareness 

for this display concept.

Table 73: Mean Lateral Flight-Path Error (ft) SNK Results for Display Concept for all Pilots for 

the VFR Approach

Display 

Concept

Number of 

Data 

Points

Subset

1 2

PBG 18 70.2

SVS 18 79.7

SCFD 19 141.2

BRD 24 1096.4

Significance 0.35 1.00

4.9.2 ILS approach, all pilots

An ANOVA performed on the lateral flight path error for the main effects of display concept and pilot 

type resulted in highly significant results for display concept (F(3,85)=25.7, p<0.01) and non-significant 

results for pilot type (p>0.05). Subsequent post-hoc analysis produced two subsets, as shown in Table 74. 

In Figure 28, note that the lateral flight path error for the BRD display concept frequently exceeded the 

0.1 nm FTE criterion, especially for Segment 2 and initial stages of Segment 3 of the approach. (The 

label “RNP 0.1 criterion” is used for illustration to place significance on 0.1 nm FTE. The data are time 

history data and do not imply 95% probability of exceedance or confidence levels.) Part of the reason 

that the BRD display concept exceeded the RNP 0.1 criterion from approximately 7.0 to 8.4 nm was due 

to the low sensitivity of the localizer course deviation indicator (CDI) at this range to touchdown. During 
106

Segment 2, pilots were flying a localizer intercept heading with the BRD display concept and had no 

direct indication regarding the location of the desired path which contributed to large lateral flight path 

errors. 

The effect of pilot group can also be seen in Figure 28, but only for the BRD display concept, where 

the VFR pilots exhibited worse lateral flight path error than the IFR or H-IFR pilots groups. No 

discernible difference is observable for pilot type for the other display concepts. One ILS approach that 

required termination can be seen in Figure 28 by the red line that terminates at about 2.5 nm from 

touchdown. For this run, the VFR pilot had no awareness of desired heading or relationship to the 

intended course, as evidenced by a failure to capture and track the ILS resulting in crossing back and forth 

across the final approach flight path.
107

Figure 28: IFR Approach Lateral Error, By Display Condition and Pilot Group 

-3000

-2000

-1000

0

1000

2000

3000

ILS Approach, BRD, All Pilots

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Lateral Error, ft

Segment 3 Segment 2

RNP 0.1 Criteria

RNP 0.1 Criteria

-3000

-2000

-1000

0

1000

2000

3000 ILS Approach, PBG, All Pilots ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

ComputedL

Lateral Error, ft

Segment 3 Segment 2

RNP 0.1 Criteria

RNP 0.1 Criteria

-3000

-2000

-1000

0

1000

2000

3000

ILS Approach, SCFD, All Pilots

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Laterr Error, ft

Segment 3 Segment 2

RNP 0.1 Criteria

RNP 0.1 Criteria

-3000

-2000

-1000

0

1000

2000

3000

0 1 2 3 4 5 6 7 8 9 10

ILS Approach, SVS, All Pilots

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Lateral Error, ft

Distance to Touchdown, nm

Segment 3 Segment 2

RNP 0.1 Criteria

RNP 0.1 Criteria

VFR Pilots

IFR Pilots

H-IFR Pilots
108

The SCFD display concept reduced lateral flight path error approximately 78% to only 140.1 ft

compared to the BRD display concept, as shown in Figure 28 and Table 74. However, results in Figure 

28 indicate that the SCFD display concept exceeded the 0.1 nm (i.e., FTE) criterion in Segment 2, likely 

the result of the lack of turn anticipation for this display concept. Lateral flight path error results for the 

PBG concept were better than the SCFD, reducing lateral flight path error by 47%. The SVS display 

concept had the lowest lateral error of all and was slightly better than the PBG. Except for a few very 

brief times during the initial stages of the approach, the PBG and SVS display concepts were well within 

the 0.1 nm FTE criterion for all pilot groups. For lateral flight path error results, it should be noted that 

evaluation pilots were responding to CDI needle deflections for the BRD display concept, and, at large 

distances from the runway, small angular CDI deflections represented large flight path errors. 

Conversely, SCFD, PBG, and SVS display guidance concepts were based on rectilinear flight path errors 

and provided consistent guidance cues throughout the approach. Figure 28 also illustrates that the only 

display concept to indicate a difference between the different pilot groups was the BRD display concept.

Table 74: Mean Lateral Flight-Path Error (ft) SNK Results for Display Concept for all Pilots for 

the ILS Approach 

Display 

Concept

Number of 

Data Points

Subset

1 2

SVS 21 72.9

PBG 16 73.8

SCFD 18 140.1

BRD 30 624.1

Significance 0.684 1.000

4.9.3 VMC-Like approach, all pilots

Figure 29 presents lateral flight path error results for the VMC-like approach. It can be seen from 

Figure 29 that the SCFD display concept exceeded the RNP 0.1 criterion, especially during the turn to 

final approach (i.e., at 1.0 nm), but also during the turn to base leg to a lesser degree (i.e., at 2.4 nm). 

Results from Figure 29 also show that all three pilot types performed similarly. An ANOVA performed 

on the lateral flight path error for the main effects of display concept and pilot type resulted in highly 

significant results for display concept (F(3,56)=11.9, p<0.01) and non-significant results for pilot type 

(p>0.05). Subsequent post-hoc analysis produced two unique subsets, as shown in Table 75. Unlike 

results for the ILS approach, which indicated no statistical difference between the advanced display 

concepts, results for the VMC-like approach indicate that the SCFD display concept was significantly 

worse than the PBG and SVS display concepts. This result is again attributable to the lack of turn 

anticipation of the SCFD display concept. The inclusion of SVS terrain resulted in an 11.6% reduction in 

lateral flight path error compared to the PBG display concept. Lateral flight path error results for the SVS 

display concept were nearly one half of (48.8% less) those for SCFD display concept.
109

Table 75: Mean Lateral Flight-Path Error (ft) SNK Results for Display Concept for all Pilots for 

the VMC-Like Approach 

Display 

Concept

Number of 

Data Points

Subset

1 2

SVS 19 70.9

PBG 19 80.2

SCFD 18 138.6

Significance 0.560 1.000
110

Figure 29: VMC-Like Approach Lateral Error, By Display Concept and Pilot Group

-3000

-2000

-1000

0

1000

2000

3000

VMC-LikeApproach, PBG, All Pilots 

Lateral Error, ft

RNP 0.1 Criteria

Segment 3

RNP 0.1 Criteria

Segment 2

-3000

-2000

-1000

0

1000

2000

3000

0 1 2 3 4 5 6 7

VMC-Like Approach, SVS, All Pilots

Com

Com

Com

Com

Com

Com

Com

Com

Com

Com

Com

Com

Com

Com

Com

Com

Com

Com

Com

Lateral Error, ft

Distance to Touchdown, nm

RNP 0.1 Criteria

Segment 3 Segment 2

RNP 0.1 Criteria

-3000

-2000

-1000

0

1000

2000

3000

VMC-Like Approach, SCFD, All Pilots

Lateral Error, ft

RNP 0.1 Criteria

Segment 3 Segment 2

RNP 0.1 Criteria

VFR Pilots

IFR Pilots

H-IFR Pilots
111

Figure 30: Lateral Error for Segment 3 for the ILS and VMC-Like Approach, All Pilots

4.9.4 ILS approach, Segment 3, without VFR pilot data

Results for lateral flight path error were analyzed without the VFR pilot data during Segment 3, 

enabling a more direct comparison of the four display concepts within current IMC operations. Results 

from an ANOVA performed on the lateral flight path error results for the main effects of display concept 

and pilot type revealed that display concept was highly significant (F(3,56)=22.92, p<0.01) while pilot 

type was not significant (p>0.05). There were no significant interactions between display concept and 

pilot type. A subsequent SNK post-hoc analysis yielded two unique subsets, as shown in Table 76. The 

lowest (best) subset was composed of all of the advanced display concepts with an average mean of 

approximately 54 ft, which was only one fifth of the lateral flight path error for the BRD display concept. 

This result is consistent with previous research and indicates that large decreases in flight path error are 

possible with advanced display concepts. The highest (worst) subset contained only the BRD display 

concept with a mean of 272 ft, which, for the purposes of this report, establishes the acceptable 

performance for the ILS approach in today’s operations. Also note that the effect of pilot type is only 

apparent for the BRD display concept in Figure 28, indicating the capability of the advanced display 

concepts, especially the PBG and SVS, to greatly enhance the ability of VFR pilots to maintain lateral 

path error as good as higher-hour better-trained pilots, for the ILS approach. 

0

50

100

150

200

250

300

ILS VMC-Like

SCFD

PBG

SVS

Lateral Error, rms ft

Approach
112

Table 76: Mean Lateral Flight-Path Error (ft) SNK results, for the ILS Approach, IFR and H-IFR 

Pilots

Display Concept Number of 

Data Points

Subset

1 2

PBG 10 49.1

SCFD 12 54.5

SVS 15 59.3

BRD 19 272.4*

Significance 0.96 1.00

*Note: This value is used herein as the baseline performance for an ILS Approach

4.9.5 ILS and VMC-Like approach analysis, Segment 3, all pilots

An analysis of the Segment 3 data for the ILS and VMC-like approaches was performed to compare 

the differences between nominal ILS localizer and glideslope tracking to the turning and descending flight 

of the VMC-like approach. Figures 28 and 29 provide a visual comparison of the results for the ILS and 

VMC-like approaches to assess the capabilities of advanced display concepts to support advanced IMC 

operations. Note that results for the SCFD display concept show that the “RNP 0.1 criterion” (i.e., FTE 

of 0.1 nm) was frequently exceeded in several runs at approximately 0.9 nm during the turn to final 

approach. There was also one run for an IFR pilot that exceeded the RNP 0.1 criterion during the turn to 

base leg at approximately 2.2 nm. Results for the PBG and SVS display concepts indicate reduced lateral 

flight path error compared to the SCFD display concept and remained well within the 0.1 nm FTE (“RNP 

0.1 criterion”). There was also an increase for lateral flight path error for the VMC-like approach 

compared to the ILS approach for Segment 3. This was expected due to the nature of the VMC-like 

approach with two turns during Segment 3.

An ANOVA was conducted on the lateral flight path error data for all pilots for Segment 3 of the ILS 

and VMC-like approaches. The ANOVA analysis indicated that display concept (F(2,111)=15.05, 

p<0.01) and approach (F(1,111)=28.63, p<0.01) were highly significant for lateral path error while pilot 

type (F(2,111)=2.15, p=0.12) was not significant. The interactions between display concept and approach

were also highly significant (F(2,111)=12.29, p<0.01); however, the interaction between pilot type and 

display concept, and pilot type and approach, were not significant. Subsequent post-hoc analysis yielded 

two subsets as shown in Table 77. In the lower (best) subset were the SVS and PBG display concepts 

with an average mean lateral flight path error of approximately 65 ft, which was approximately one half

of the error for SCFD. One reason for the reduced lateral flight path error for the SVS and PBG display 

concepts involves the turn anticipation provided by the HITS guidance. Even in the presence of the ND 

that provided route and turn anticipation information to the evaluation pilots, the HITS guidance was 

especially valuable for the VMC-like approach.
113

Table 77: Mean Lateral Flight-Path Error (ft) SNK results, Segment 3 of the ILS and VMC-Like 

Approaches for all Pilots

Display Concept

Number of 

Data Points

Subset

1 2

SVS 40 64.2

PBG 35 65.8

SCFD 36 118.0

Significance 0.88 1.000

Table 78, along with Figures 28, 29, and 30, provides information regarding the effect of approach and 

display concept that illustrates the increase in lateral flight path error for the VMC-like approach 

compared to the ILS approach. Both the SCFD and PBG display concepts exhibited larger increases in 

lateral flight path error than the SVS display concept due to approach in Segment 3. Lateral flight path

error for the VMC-like approach increased by factors of 2.78, 1.48, and 1.16 for the SCFD, PBG, and 

SVS display concepts, respectively, compared to the ILS approach. The smaller increase in lateral error 

for the VMC-like approach can be attributed to the increased situation awareness provided by the HITS 

guidance and SVS terrain for the VMC-like approach.

Table 78: Lateral Flight-Path Error Statistical Results (ft), Segment 3 of the ILS and VMC-Like 

Approaches for all Pilots

Display 

Concept Approach Mean Standard Deviation

SCFD ILS 62.4 25.7

VMC-Like 173.6 113.7

PBG ILS 52.2 18.8

VMC-Like 77.3 30.2

SVS ILS 59.5 23.7

VMC-Like 69.3 29.6

4.9.6 Test condition analysis, Segment 3

An ANOVA analysis was performed on lateral flight path error for TC for Segment 3 for all three 

approaches with post-hoc results presented in Table 79. Test Condition was revealed to be highly 

significant for lateral flight path error results (F(12,220)=34.724, p<0.01) and was not significant for pilot 

type (p>0.05). Two unique subsets were formed in subsequent post-hoc analysis. The highest (worst) 

subset was comprised of TC 1 (VFR approach with BRD display concept). This result was expected in 

that the EPs had to rely exclusively on their out-the-window navigation to negotiate the VFR approach 

and is consistent with current VFR operations. There were no significant lateral path error differences 

among the other test conditions. It is important to note that results for TCs 3, 4, 6, 7, 8, 10, and 11 had a 

group average mean of 70 ft, which was only approximately one third of the lateral flight path error 

results for TCs 2, 5, and 9, with an average mean of 206 ft. Although not statistically significant, the TCs 

for the advanced display concepts for the IMC approaches (i.e., TCs 6 through 11) were all lower than for 
114

TC 5 (ILS approach for BRD). In addition, results for the VFR pilots’ evaluations of the SVS display 

concept for the ILS and VMC-like approaches were also lower than TC 5. Based on lateral flight path 

error, advanced IMC approaches, such as the VMC-like approach, appear feasible for the PBG and SVS 

advanced displays tested in this study. From Figure 29, it can be seen that results for the SCFD display 

concept violated the RNP 0.1 criterion (i.e., 0.1 nm FTE) for the VMC-like approach, indicating that this 

type of manually-flown approach may not be appropriate for this display concept.

Table 79: Mean Lateral Flight-Path Error (ft) SNK results for TC

Test Condition TC # Number of 

Data Points

Subset

1 2

ILS Approach + PBG 7 16 52.2

ILS Approach + SVS 8 21 59.5

ILS Approach + SVS (VFR pilots) 12 6 59.8

ILS Approach + SCFD 6 18 62.4

VMC-Like Approach + SVS 11 19 69.3

VMC-Like Approach + PBG 10 19 77.3

VFR Approach + PBG 3 18 77.9

VMC-Like Approach + SVS (VFR pilots) 13 6 84.2

VFR Approach + SVS 4 18 89.7

VFR Approach + SCFD 2 19 172.7

VMC-Like Approach + SCFD 9 18 173.6

ILS Approach + BRD (no VFR pilots) 5 19 272.4

VFR Approach + BRD 1 24 1454.9

Significance 0.747 1.000
115

4.10 Vertical Flight Path Error

No clear criterion were available for the vertical flight path error for the IMC approaches. For the 

discussions herein, +/-100 ft will be used as a relevant criterion for comparisons and assessments. 

Figures 31, 32, and 33 show real-time vertical flight path error data for the VFR, ILS and VMC-like 

approaches, respectively. Figure 34 and Table 80 show mean RMS results for vertical flight path error.

Table 80: Vertical Path Error (ft) Descriptive Statistics for all Pilots

Display

Concept Approach Mean Standard Deviation

Number of 

Data Points

BRD VFR 99.6 46.6 24

ILS 171.8 117.6 30

Total 139.7 99.1 54

SCFD VFR 25.2 15.0 19

ILS 25.6 15.0 18

VMC-Like 26.6 15.1 18

Total 25.8 14.8 55

PBG VFR 31.7 12.3 18

ILS 40.7 19.2 16

VMC-Like 35.2 16.7 19

Total 35.7 16.3 53

SVS VFR 36.9 22.4 18

ILS 39.7 18.0 21

VMC-Like 30.1 13.8 19

Total 35.7 18.4 58

Total VFR 51.9 43.0 79

ILS 83.5 96.4 85

VMC-Like 30.7 15.4 56

Total 58.7 68.8 220

4.10.1 VFR approach

For the VFR approach, the EPs were instructed to maintain altitude until abeam the runway numbers, 

then descend through base leg and final approach segments. For the BRD display concept, no guidance 

information was provided to the EPs and they relied on the out-the-window view to manage flight path 

along with cross checking their altimeter at specific points along the approach. As can be seen in Figure 

31 and Table 81, vertical flight path error for the BRD display concept was approximately 100 ft, which is 

reasonable for this type of approach.

Note that issues with the VFR approaches that were flown with cut-corners at the turn to base leg from 

the downwind leg created some problems with the vertical flight path error data as well. This was 

previously described for the lateral flight path error. The main problem was that when the lateral flight 

path error calculation shifted the desired flight path reference point to the final approach leg when the 

corner between the downwind leg and base leg was cut, the desired altitude also changed. Unlike lateral 
116

flight path error, this anomaly usually increased the magnitude of the vertical flight path error results for 

the BRD display concept. However, the data does approximately represent the capability of pilots to fly a 

VFR approach with the BRD display concept especially for Segment 2.

Provisions of guidance cues and vertical flight path error information within the advanced display 

concepts enabled EPs to substantially lower their vertical flight path errors to approximately 40% of the 

results for the BRD display concept, as shown in Figure 31 and Table 81. An ANOVA conducted on the 

vertical flight path error data for the VFR approach for the main effects of display concept and pilot type 

resulted in significant results for display concept (F(3, 79)=45.0, p<0.01) but not for pilot type (p>0.05) or 

the interaction of pilot type with display concept.

Table 81: Mean Vertical Flight-Path Error (ft) SNK Results for Display Concept for all Pilots for 

the VFR Approach

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 19 25.2

PBG 18 31.7

SVS 18 25.2

BRD 24 99.6

Significance 0.313 1.000
117

Figure 31: VFR Approach, Vertical Error, By Display Concept and Pilot Group

-400

-200

0

200

400

VFR Approach, BRD, All Pilots

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

Vertical Error, ft

Segment 3 Segment 2

-400

-200

0

200

400

VFR Approach, PBG, All Pilots

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

Vertical Error, ft

Segment 3 Segment 2

-400

-200

0

200

400

VFR Approach, SCFD, All Pilots

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Comp

Vertical Error, ft

Segment 3 Segment 2

-400

-200

0

200

400

0 1 2 3 4 5 6 7

VFR Approach, SVS, All Pilots

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Compute

Vertical Error, ft

Distance to Touchdown, nm

Segment 3 Segment 2

VFR Pilots

IFR Pilots

H-IFR Pilots
118

4.10.2 ILS approach, all pilots

An ANOVA conducted on the vertical flight path error data for the ILS approach for the main effects 

of display concept and pilot type produced highly significant results for display concept (F(3, 85)=25.7, 

p<0.01) but not for pilot type (p>0.05). There was no significant interaction between display concept and 

pilot type. Subsequent post-hoc analysis on display concept resulted in two unique subsets, as shown in 

Table 82. For the advanced display concepts, vertical flight path error was reduced approximately by a 

factor of five compared to the BRD display concept, as shown in Figure 32. This result was partially due 

to the SCFD and HITS guidance systems providing constant sensitivity to vertical flight path errors for 

the entire ILS approach and the advanced display concepts integrated information format. Note that one 

dot of vertical CDI deflection corresponds to approximately 360 ft of vertical flight path error at the 

glideslope intercept point. There is also an indication that the SCFD display concept had lower vertical 

flight path error than the PBG and SVS display concepts which had pathway-based guidance as indicated 

in Figure 32. The reduced vertical flight path error results for the SCFD was due to the combination of 

the prominent single-cue-flight-director symbology with the pitch-attitude command system that 

emphasized attitude and vertical error control. While the effect of pilot type was not statistically 

significant, the operational effect of pilot type can be observed in Figure 32 where the VFR pilots’ results 

can be seen to be somewhat worse than the other two pilot groups for the BRD and PBG display 

concepts. The effect of pilot group was reduced for the SVS display concept as seen in Figure 32, which 

appears to show the positive effect of SVS terrain to aid in vertical error reductions when compared to 

results for the PBG display concept. The presence of the SVS terrain significantly increased pilots’ 

situation awareness and their ability to integrate the information provided, likely leading to reduced 

vertical flight path error.

Table 82: Mean Vertical Flight Path Error (ft) SNK Results for Display Concept for all Pilots,ILS 

Approach

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 18 25.6

SVS 21 39.7

PBG 16 40.7

BRD 30 171.8

Significance 0.761 1.000
119

Figure 32: ILS Approach, Vertical Error, By Display Concept and Pilot Group

-400

-300

-200

-100

0

100

200

300

400

ILS Approach, BRD, All Pilots

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

ComputedVerti

Vertical Error, ft

Segment 3 Segment 2

-400

-300

-200

-100

0

100

200

300

400 ILS Approach, PBG, All Pilots

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Vertical Error, ft

Segment 3 Segment 2

-400

-300

-200

-100

0

100

200

300

400

ILS Approach, SCFD, All Pilots

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

ComputedV

Vertical Error, ft

Segment 3 Segment 2

-400

-300

-200

-100

0

100

200

300

400

0 1 2 3 4 5 6 7 8 9 10

ILS Approach, SVS, All Pilots

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Computed

Vertical Error, ft

Distance to Touchdown, nm

Segment 3 Segment 2

VFR Pilots

IFR Pilots

H-IFR Pilots
120

4.10.3 VMC-Like approach, all pilots

Results for vertical flight path error, illustrated in Figure 33, indicate somewhat similar 

characteristics for all display concepts for the VMC-like approach. However, the PBG display concept 

does show some degraded performance compared to the SCFD, with several vertical error excursions in 

excess of +/-100 ft. An ANOVA conducted on the vertical flight path error data for the VMC-like 

approach for the main effects of display concept and pilot type did not produce any significant (p>0.5) 

results for display concept, pilot type, or interaction between these two factors. 
121

Figure 33: VMC-Like Approach, Vertical Error, By Display Concept and Pilot Group

-400

-300

-200

-100

0

100

200

300

400

VMC-Like Approach, PBG, All pilots

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

ComputedVertica

Vertical Error, ft

Segment 3 Segment 2

-400

-300

-200

-100

0

100

200

300

400

0 1 2 3 4 5 6 7

VMC-Like Approach, SVS, All pilots

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

ComputedVertic

Vertical Error, ft

Distance to Touchdown, nm

Segment 3 Segment 2

-400

-300

-200

-100

0

100

200

300

400

Comput

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Compu

Vertical Error, ft

Segment 3 Segment 2

VMC-Like Approach, SCFD, All pilots

VFR Pilots

IFR Pilots

H-IFR Pilots
122

4.10.4 ILS approach, Segment 3, without VFR pilot data

Vertical flight path error was analyzed without the VFR pilot data. Results from an ANOVA 

revealed that display concept was highly significant (F(3,56)=19.634, p<0.01) for vertical path error. 

Neither pilot type (i.e., either IFR or H-IFR) nor the interaction between pilot type and display concept 

were significant (p>0.05) for this measure. These results match the effects shown in Figure 32. 

Subsequent post-hoc analysis yielded two unique subsets, as shown in Table 83. The advanced display 

concepts comprised the lower (best) vertical path error subset with the BRD display concept being the 

only member of the higher (worst) subset. The lowest vertical flight path error was for the SCFD display 

concept with results being approximately one half as high as the results for the PBG and SVS display 

concepts. The differences between the SCFD, PBG, and SVS display concepts were not statistically 

significant, but this result may indicate the ability of the SCFD display concept to improve vertical flight 

path error for this approach segment. The highest (worst) subset contained only the BRD display concept 

with a mean error of 145 ft, which established the nominal performance for the ILS approach in today’s 

operations for the data considered in this report. Overall, the average of the mean vertical flight path error 

for the advanced display concepts was 27.2 ft, which was less than one-fifth of the error for the BRD 

display concept. Again, this result is likely due to the variable sensitivity of the CDI needle movement in 

response to vertical flight path error with range during the ILS approach and the integrated, intuitive

guidance provided by the SCFD and HITS guidance systems. These results suggest that terrain clearance 

margins required for ILS operations today could be greatly reduced with advanced display concepts. 

Currently, approach procedures are designed according to Reference 24 (FAA Order 8260.3, United 

States Standard for Terminal Instrument Procedures (TERPS)). For a nominal 3-degree glideslope, the 

altitude clearance is approximately 892 ft at the beginning of Segment 3 (i.e., 8.4 nm from touchdown). 

If it is assumed that total system error is composed of navigation system error and flight technical error, 

then it is possible that this amount of margin could be reduced if flight technical error is significantly

reduced.

Table 83: Mean Vertical Flight-Path Error (ft) SNK results for the ILS Approach, IFR and H-IFR 

Pilots Only

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 12 16.2

PBG 10 32.5

SVS 15 33.0

BRD 19 145.0*

Significance 0.70 1.0

*Note: This value is used herein as the baseline performance for an ILS Approach

4.10.5 ILS and VMC-Like approach analysis, Segment 3, all pilots

Figures 32 and 33 show a visual comparison of the vertical flight path errors for the advanced display 

concepts for the ILS and VMC-like approaches. An ANOVA was conducted on the vertical flight path

error data for all pilots for Segment 3 of the ILS and VMC-like approaches for the main effects of pilot 

type, display concept, and approach to compare and contrast each advanced display concept as well as 

assess the capabilities of advanced display concepts to support advanced IMC operations. The ANOVA 

analysis indicated that pilot type (F(2,111)=7.701, p<0.01) and display concept (F(2,111)=8.96, p<0.001) 

were highly significant and approach was not (p>0.05) for vertical path error. There were no significant 
123

interactions between the main effects for this measure. Table 84 shows that the IFR and H-IFR pilot

groups exhibited significantly less vertical flight path error than the VFR pilot group. The VFR pilots’

vertical flight path error was approximately 10 ft, or 37%, more than the IFR and H-IFR pilots. Table 84

provides SNK results for display concept for the ILS and VMC-like approaches and shows that the SCFD 

had significantly less vertical path error (approximately 12 ft) than the PBG and SVS display concepts. 

There were no significant vertical path error differences between the PBG and SVS display concepts. 

This result is likely due to the superior vertical flight path guidance provided by the SCFD system for 

these approaches. The SCFD symbology provided a compelling indication of the commanded pitch 

attitude to control vertical flight path error. However, as shown previously, the SCFD did exhibit poor 

lateral flight path error results primarily due to the lack of turn anticipation. Overall, vertical flight path 

error results indicate that the VMC-like approach may be feasible for the advanced display concepts.

Table 84: Mean Vertical Flight-Path Error (ft) SNK results for Pilot Type, ILS and VMC-Like 

Approaches for all Pilots

Pilot Type Number of 

Data Points

Subset

1 2

IFR 36 27.6

H-IFR 39 28.6

VFR 36 39.1

Significance 0.765 1.0

Table 85: Mean Vertical Flight-Path Error (ft) SNK results for the Advanced Display Concepts, 

ILS and VMC-Like Approaches for all Pilots

Display 

Concept

Number of 

Data Points

Subset

1 2

SCFD 36 23.6

SVS 40 33.9

PBG 35 37.6

Significance 1.000 0.252

4.10.6 Test condition analysis, Segment 3

A subsequent analysis was performed on vertical flight path error based on TC, with post-hoc results 

presented in Table 86. TC was highly significant for vertical flight path error results (F(12,209)=20.555, 

p<0.01). Three unique subsets were formed from SNK post-hoc analysis of the vertical flight path error. 

The lowest vertical path error was recorded for the advanced display concepts regardless of approach 

(VMC-like, ILS, or VFR). The VFR approach evaluation of the BRD display concept had significantly 

worse vertical path error than any of the advanced displays for any approaches, but had significantly 

better vertical-path error than TC 5 (ILS approach with BRD display concept without the VFR pilots). 

The BRD display concept for the ILS approach had significantly worse vertical path error than all other 

display concept and approach combinations. These results appear to indicate that advanced IMC 

approaches similar to the VMC-like approach, are feasible with the advanced display concepts from the 

standpoint of vertical flight path error, since all IMC evaluations of these display concepts (i.e., TCs 9, 

10, 11) were significantly lower than the results for ILS approach with the BRD display concept (TC 5). 
124

Results for the VFR pilots’ evaluation of the SVS display concept for both the ILS and VMC-like 

approaches were significantly better than TC 5.

Table 86: Mean Vertical Flight-Path Error (ft) SNK results for TC

Test Condition TC 

#

Number of 

Data Points

Subset

1 2 3

ILS Approach + SCFD 6 18 21.7

VMC-Like Approach + SCFD 9 18 25.5

VFR Approach + SCFD 2 19 25.9

VMC-Like Approach + SVS 11 19 29.7

VMC-Like Approach + SVS (VFR pilots) 13 6 30.9

VFR Approach + PBG 3 18 31.1

VMC-Like Approach + PBG 10 19 36.0

ILS Approach + SVS 8 21 37.6

ILS Approach + PBG 7 16 39.4

VFR Approach + SVS 4 18 40.2

ILS Approach + SVS (VFR pilots) 12 6 49.0

VFR Approach + BRD 1 24 116.7

ILS Approach + BRD (no VFR pilots) 5 19 145.0

Significance 0.704 1.000 1.000

Figure 34: Vertical Error, Segment 3, for Approaches 2 and 3, All Pilots

0

10

20

30

40

50

60

70

80

ILS VMC-Like

SCFD

PBG

SVS

Vertical Error, rms ft

Approach
125

5.0 Summary of Results

Qualitative results for the SVS display concept evaluations in simulated IMC were not statistically 

different from the VMC evaluations, but were statistically different from the IMC evaluations of the BRD

display concept. This was true for all qualitative parameters used for this study (i.e., SART, TLX, DRR, 

and TA). Note that the reference condition for the IMC evaluation of the BRD did not include the VFRrated pilots’ data to provide a better comparison metric that had strong ties to existing operations. The 

incorporation of SVS terrain provided a significant improvement in the results compared to the PBG 

display concept for situation awareness measures (SART and TA). The effect of the SVS terrain 

depiction decreased workload measures (TLX, DRR) compared to the PBG display concept, but not 

significantly.

Quantitative results, such as PTS, lateral, vertical, localizer and glideslope errors, indicate that the 

advanced display concepts (SCFD, PBG, and SVS) provided significantly better performance than the 

BRD display concept for the ILS approach. Lateral and vertical error results for the advanced displays 

were reduced by approximately a factor of 4 for the ILS approach compared to the BRD display concept. 

Significant effects of pilot type were observed in several parameters, such as PTS, localizer, glideslope, 

and IAS errors; however, those effects were primarily limited to results from the BRD display concept,

with the VFR-rated pilots performing significantly worse than the IFR and H-IFR pilots. The SCFD 

display concept generated significantly worse lateral flight path and localizer error results compared to the 

PBG and SVS display concept evaluations for the VMC-like approach. This is thought to be due to a lack 

of turn anticipation of the SCFD. While RMS values of lateral flight path and localizer errors were not 

statistically different from those from the BRD ILS evaluations by the IFR and H-IFR pilots, graphical 

results indicated that PTS and FTE standards were exceeded for the SCFD display concept during the 

turns to base leg and final approach for the VMC-like approach. Several exceedances of the IFR PTS 

criteria were also observed for the PBG display concept for the VMC-like approach during the last 2 nm. 

This indicates that the SCFD and PBG display concepts would not likely be suitable for advanced IMC 

approaches, such as the VMC-like approach. Results for the SVS display concept indicate that all pilots 

can fly an advanced VMC-like approach and generate flight technical error (i.e., lateral, vertical, localizer, 

glideslope and IAS errors) results as good as or better than IFR and H-IFR pilots using the BRD display 

concept for the ILS approach. Glideslope error results for the SCFD display concept were significantly 

superior to the PBG display concept results due to the emphasis on attitude control and the underlying 

flight director guidance algorithm performance. However, pilot workload was increased significantly for 

the SCFD compared to the PBG display concept. The incorporation of SVS terrain for the SVS display 

concept improved flight path control and significantly reduced glideslope error compared to the PBG 

display concept while maintaining workload. Glideslope error results for the SVS display concept were 

similar to results for the SCFD. The depiction of terrain on the SVS display concept greatly improved the 

performance of VFR-rated pilots.

6.0 Limitations of Current Study

The current investigation evaluated scenarios in which all systems were functioning nominally over a 

small, but critical, range of conditions and with a limited, but representative, sample of the pilot 

population. Results from this flight test are intended to be applicable to other real-world conditions; 

however, several limitations of the current study dictate that subsequent applications of these results need 

to be performed carefully.
126

Issues related to off-nominal system performance were not addressed, such as ADAHRS failures, 

display failures, and communication failures. One particular off-nominal system issue with SVS display 

technology not addressed in this report was the risk of terrain database errors. Terrain database errors 

could influence a pilot to deviate from the intended flight path due to false SVS terrain cues. It needs to 

be realized that the concept of operations considered herein still involves a visual segment of the approach

and landing, similar to current ILS approaches. Visually verifying the condition of the runway prior to 

landing will still be performed by the pilot, whether they are using SVS or not, under conceivable GA 

operations in the foreseeable future.

Potentially increased heads-down time during marginal VMC operations is not explicitly addressed 

within the current study, although some discussion is provided regarding the presence of terrain heads-up 

and heads-down with SVS displays for VMC operations. While a safety of flight issue, VMC mid-air 

collisions are less prevalent than IMC Controlled Flight Into Terrain (CFIT) accidents. As a result, while 

operations in VMC flight may need to be evaluated with SVS displays, it is proposed that significant 

improvements in IMC flight could potentially more than offset this issue and improve the overall safety 

situation. While not part of this research effort, careful concept of operations development and training

would be needed to effectively deal with issues such as overconfidence and decision making with SVS 

displays. The current work documented herein does reflect the potential benefits from SVS display 

concepts, and other advanced display concepts, and does support the need for further study as well as 

operational evaluations.

A total of 19 pilots participated overall in this flight test, generating 219 research evaluations. More 

evaluation pilots would have helped to add to the statistical power of the subsequent analyses, especially 

those that compared across pilot groups. The use of pilot hours and officially logged training as a control 

variable to establish pilot type could be improved. For example, some low-time VFR pilots 

acknowledged that they frequently used desktop simulation tools to practice IFR approaches. This type of 

auxiliary experience could be mitigated through pre-testing of subject pilots (i.e., to see that they provide 

consistent performance for their group) and also pre-test screening (to screen-out pilots with known 

discrepancies). 

Another more basic limitation of the study was the ability of current HF tools (e.g., SART, TLX, 

DRR) that were employed to accurately differentiate the effects of all of the factors involved. In general, 

these HF parameters are used to compare human performance using one display or operational concept to 

another display or operational concept. Results are usually limited to saying that one display, operational 

concept, etc. is as good, as or better than, another display or operational concept. There is some concern 

about using these metrics for comparisons across largely different scenarios, such as encountered between 

VMC and IMC flight.

7.0 Conclusions

A flight test was performed to compare the use of three advanced primary flight and navigation 

display concepts to a baseline, round-dial concept to assess the potential for advanced operations. The 

displays were evaluated during visual and instrument approach procedures including an advanced 

instrument approach resembling a visual airport traffic pattern. Nineteen pilots from three pilot groups,

reflecting the diverse piloting skills of the General Aviation pilot population, served as evaluation 

subjects. Specifically, three hypotheses characterizing potential transformations were defined. The 

hypotheses can be summarized as follows:

1. Using SVS displays, the workload and SA results of all pilots flying landing approaches 

in IMC will be as good as, or better than, the same pilots flying in VMC using baseline 

round-dial displays. 
127

2. Using SVS displays, low-time, non-instrument-rated pilots flying nominal ILS approaches 

in IMC will achieve flight technical error, workload, and SA results as good as, or better 

than, ratings from instrument-rated pilots conducting the same ILS approach using 

baseline, round-dial displays.

3. Using SVS displays, all pilots, including low-time, non-instrument-rated pilots, will be 

able to conduct IMC approaches with geometries similar to a VFR pattern with flight 

technical error, workload, and SA results as good as, or better than, results from 

instrument rated pilots conducting ILS approaches using baseline round-dial displays.

The flight test had four unique elements to assess these hypotheses. The first was including noninstrument-rated pilots in the evaluation of instrument procedures in simulated IMC. This inclusion 

allowed direct comparison of the performance and ratings of non-IFR-rated pilots using SVS displays and 

IFR-rated pilots using conventional instrumentation. The second element was including operations in 

both VMC and simulated IMC. This aspect of the test enabled direct comparisons of workload and 

situation awareness ratings across these very different operational environments. The third element was 

the inclusion of an instrument approach pattern that overlaid a typical VFR airport traffic pattern. 

Including this element allowed direct comparison of this advanced operation conducted with SVS display 

versus conventional operations and displays. The fourth element was including a progression of 

advanced display concepts. This aids in assessing display elements that are essential towards enabling 

significant operational changes.

Relative to the three hypotheses, the results of the test can be summarized as follows:

Hypothesis 1 is supported by the analyses for SART (situation awareness) and TLX (pilot workload). 

Results for the SVS display concept evaluations for the ILS and VMC-like approaches were significantly 

better than the SCFD and PBG display concepts for these measures. SART and TLX results for the SVS 

display concept IMC evaluations were as good as or better than the VFR evaluation of the BRD display

concept. These results show that using as SVS display, the workload and SART ratings of all the subject 

pilots flying landing approaches in IMC were as good as, or better than, the same pilots flying in VMC 

using baseline round-dial displays. 

Hypothesis 2 is supported by analysis that revealed that the VFR pilots’ results for the SVS display 

concept ILS evaluations were improved for all of the flight technical error variables considered (PTS, 

lateral error, localizer error, and glideslope error), compared to those from the IFR and H-IFR pilots for 

the ILS evaluation of the BRD display concept. Also, VFR-pilot results for the SVS display concept 

during the ILS evaluations were significantly improved for vertical error as well as SART and TLX

results. While a bias in the VFR pilots’ results was encountered that illustrated a tendency of these pilots 

to provide higher SART ratings than the other two pilot groups, the increase in SA provided by SVS 

displays is considered much larger than the bias and more than compensated for this artifact in the data. 

IAS results indicate that VFR pilots’ performance was slightly worse for the SVS display concept 

compared to the IFR and H-IFR pilots’ evaluation of the BRD for the ILS approach, but not significantly. 

Regarding Hypothesis #2, these results show that using an SVS display, low-time, non-instrument-rated 

pilots flying nominal ILS approaches in IMC can achieve flight technical error, workload and SART 

ratings as good as, or better than, ratings from instrument-rated pilots conducting the same ILS approach 

using baseline round-dial displays.

Hypothesis 3 is supported by all pilot types, including low-time VFR-rated pilots, generating similar 

or much better flight-technical- and IAS-error results for the SVS display concept evaluations for the 

VMC-like approach compared to the IFR and H-IFR pilots using the BRD display concept for the ILS 

approach. Analysis also revealed that results for the VMC-like approach using the SVS display concept,

evaluations were improved, but not significantly, for many of the variables considered (PTS, lateral error, 

localizer error, and glideslope error). Vertical error, as well as SART and TLX results, were significantly 
128

improved for the SVS display for the VMC-like approach compared to the IFR and H-IFR pilot 

evaluations of the BRD display concept for the ILS approach. The IAS results indicate that performance 

was slightly worse for the SVS display concept for the VMC-like approach compared to the IFR and HIFR pilots’ evaluation of the BRD for the ILS approach, but again not significantly. With regard to 

hypothesis #3, these results show that using an SVS display, all the subject pilot types, including lowtime, non-instrument rated pilots, were able to conduct IMC approaches with geometries similar to a VFR 

pattern with flight technical error, with workload and SART ratings as good as, or better than ratings from 

instrument-rated pilots conducting ILS approaches using the baseline round-dial display.

Considering all 3 hypotheses, all advanced display concepts were able to satisfy hypothesis #2 as a 

result of the VFR pilots being able to generate flight technical error results as good as or better than the 

IFR and H-IFR pilots’ evaluation of the BRD display concept for the ILS approach. For the VMC-like 

approach only the SVS display concept was able to satisfy hypothesis #3 due to the poor lateral and 

localizer error results for the SCFD, and poor glideslope and vertical error for the PBG. Due to the SVS 

displays’ significantly improved SART results, the SVS display was able to satisfy hypothesis #1 and was 

the only one to be able to satisfy all three hypotheses simultaneously. 

Several other noteworthy observations can be made from the results. While improvements due to the 

moving map ND and integrated PFD were observed compared to the baseline round-dial display concept,

the results indicate additional variability among the advanced guidance concepts (i.e., SCFD and PBG). 

As a result of the improved flight path awareness, the HITS guidance provided improved results for 

qualitative measures, such as SART and TLX, and also quantitative measures, such as lateral flight path

error and localizer error. While the SCFD display concept did provide significantly reduce vertical flight 

path and glideslope error compared to the PBG display concept, overall, the improvements in lateral flight 

path and localizer error provided by the HITS guidance provided superior performance enabling more 

potential applications for future RNP based procedures with curved segments. In addition to the expected 

improvements in situation awareness (including terrain) with SVS terrain displayed, improvements in 

glideslope error were also evident with SVS display concept for the VMC-like approach. This suggests 

that the terrain background improves the ability of pilots (in particular VFR pilots) to perceive and control 

glideslope error as compared to a HITS without terrain. This effect was particularly apparent during 

dynamic approach segments such as the turn to final of the VMC-like approach. Pilots were able to use 

the SVS terrain to better monitor attitude, turn rate, and flight path angle during the turns. On final 

approach the presence of the runway model complemented use of the velocity vector and HITS guidance.

Support for the hypotheses considered in this test suggests that SVS does have a significant effect on 

flight in IMC and may enable new types of operations and significantly alter the training needed to safely 

and comfortably operate in IMC. At the same time, it is important to recognize that this test only

examined nominal approaches and did not include traffic separation, hazardous weather, equipment 

failures, etc. Also, other important rating factors, such as measures of pilots’ decision-making ability or 

possible overconfidence were not considered. It is recommended that future studies build on this study by 

investigating the transformative proprieties of SVS displays in the presence of more comprehensive 

operational considerations and while stressing the importance of effective decision making by pilots of all 

experience levels.
129

8.0 References

[1] Glaab, L.; and Hughes, M.: Terrain Portrayal for Head-Down Displays Flight Test., Proceedings of the 22nd

Digital Avionics Systems Conference, 2003.

[2] Hughes, M.; and Glaab, L.: Terrain Portrayal for Head-Down Displays Simulation Experiment Results, 

Proceedings of the 22nd Digital Avionics Systems Conference, 2003. 

[3] Takallu, M.; Wong, D.; Bartolone, A.; Hughes, M.; and Glaab, L.: Interaction Between Various Terrain 

Portrayals and Guidance/Tunnel Symbology Concepts for General Aviation Synthetic Vision Displays During a 

Low En-Route Scenario, Proceedings of the 23rd Digital Avionics Systems Conference, 2004.

[4] Wong, D.; Takallu, M.; Hughes, M.; Bartolone, A.; and Glaab, L.: Simulation Experiment for Developing the 

Symbology for the Approach and Missed Approach Phases of Flight for Head-Down Synthetic Vision Systems 

Displays. Proceedings of the AIAA Modeling and Simulation Technologies Conference and Exhibit, AIAA-2004-

5440, 2004.

[5] Kramer, L.; Prinzel, L.; Bailey, R.; Arthur, J.: Synthetic Vision Enhances Situation Awareness and RNP 

Capabilities for Terrain-Challenged Approaches. Proceedings of the AIAA Third Aviation Technology, 

Integration, and Operations Technical Forum, AIAA-2003-6814, 2003.

[6] Lemos, K.; and Schnell, T.: Synthetic Vision Systems: Human Performance Assessment of the Influence of 

Terrain Density and Texture. Proceedings of the 22nd Digital Avionics Systems Conference, October 2003.

[7] Schnell, T.; Etherington, T.; Vogl, T.; and Postnikov, A.: Field Evaluation of A Synthetic Vision Information 

System Onboard the NASA ARIES 757 at Eagle County Regional Airport, Proceedings of 21st Digital Avionics 

Systems Conference, October 2002.

[8] Prinzel, L.; Arthur, J.; Kramer, L.; and Bailey, R.: Pathway Concepts Experiment for Head-Down Synthetic 

Vision Displays, Proceedings of the International Society For Optical Engineering, Volume 5424, 2004.

[9] Glaab, L.; Kramer, L.; Arthur, J.; and Parrish, R.: Flight Test Evaluation of Synthetic Vision Display Concepts 

at Dallas/Fort-Worth Airport, NASA/TP-2003-212177, 2003. 

[10] Bailey, R.; Kramer, L.; and Prinzel, L.: Crew and Display Concepts Evaluation for Synthetic / Enhanced 

Vision Systems. Proceedings of the International Society For Optical Engineering, Volume 6226, 2006.

[11] Bartolone, A.; Glaab, L.; Hughes, M.; and Parrish, R.: Initial Development of a Metric to Describe the Level of 

Safety Associated With Piloting An Aircraft With Synthetic Vision Systems (SVS) Displays. Proceedings of the

International Society For Optical Engineering, Volume 5802, May 2005.

[12] Takallu, M.; Glaab, L.; Hughes, M.; Wong, D.; and Bartolone, A.: Piloted Simulation of Various Synthetic 

Vision Systems Terrain Portrayal and Guidance Symbology Concepts for Low Altitude En-Route Scenario. 

NASA/TP-2008-215127, 2008.

[13] General Aviation Manufacturers Association (GAMA), Publication #12: Recommended Practices and 

Guidelines for an Integrated Cockpit/Flight Deck in a 14 CFR Part 23 Certificated Airplane, GAMA, Washington, 

D.C., 2004. 

[14] AC 23-23, Standardization Guide for Integrated Cockpits in Part 23 Airplanes, Staff of the Federal Aviation 

Administration, 9/30/2004.

[15] AC 23-26, Synthetic Vision and Pathway Depictions on the Primary Flight Display, Staff of the Federal 

Aviation Administration, 12/22/2005.

[16] Glover, Howard: Terrain Displays for Enhanced Ground Proximity Warning Systems, AIAA-975557, 1997.

[17] Breen, B.C.. Enhanced situation awareness. In C. Spitzer (Ed.), Avionics Handbook. Boca Raton, FL: CRC 

Press. 2001.

[18] Glaab, Louis J.; Parrish, Russell V.; Hughes, Monica F.; Takallu, Mohammad A.: Effectively Transforming 
130

IMC Flight Into VMC Flight: An SVS Case Study. Presented at the 25th Digital Avionics Systems Conference 

(DASC), Portland, OR., 2008-14170, 2006.

[19] Guide to Human Performance Measurement. American Institute of Aeronautics and Astronautics, Washington 

D.C., 1993.

[20] Hart, S.; and Staveland, L.: Development of the NASA Task Load Index (TLX): Results of Empirical and 

Theoretical Research, In P.A. Hancock & N. Meshkati (Eds.), Human Mental Workload, Elsevier Publishing, 

Amsterdam, Holland, 1988. 

[21] Chiappetti, C.: Evaluation of the Haworth-Newman Avionics Display Readability Scale, Naval Postgraduate 

School Thesis, Monterey, CA, 1994.

[22] Instrument Rating Practical Test Standards for Airplane, Helicopter and Powered Lift, Staff of the Federal 

Aviation Administration, (2004).

[23] DO-236A, Minimum Aviation System Performance Standards: Required Navigation Performance for Area 

Navigation, RTCA, 2000.

[24] FAA Order 8260.3, United States Standard for Terminal Instrument Procedures (TERPS), Staff of the Federal 

Aviation Administration.
131
132

Appendix A: Single-Cue Flight Director Description

The intent of the single-cue flight director (SCFD) system was to provide the general functionality of 

this type of flight guidance within the data set. In general, flight directors of this type provide pitch and 

roll attitude commands for the pilot to follow to achieve acceptable flight path errors. Figure A1 below 

shows what the Primary Flight Display (PFD) looked like for the SCFD display concept. The pilot’s task 

was to pitch and roll the aircraft to get the orange triangle in Figure A-1 to just line-up with the yellow 

triangles. The yellow triangles would move up and down to indicate changes in pitch attitude, and rotate 

left or right to indicate commands in roll attitude. Rate of error information, such as the difference 

between the desired and actual flight path angle, is blended with instantaneous error, such as vertical 

flight path error, to provide pitch attitude commands to follow a specific trajectory. This particular image 

is indicating that the pilot needs to roll the aircraft to the right to eliminate lateral and/or angular errors. 

The SCFD system gains were adjusted during flight-test checkout to achieve what was qualitatively 

considered to be reasonable performance for this type of guidance system. Gains adjusted included the 

Offset, Track, Alt, and gamma parameters. For the flight test checkout, several pilots evaluated the 

system performance and provided inputs regarding gain adjustment. The final values of these parameters 

are provided in Table A-1. 

Figure A- 1: Single Cue Flight Director

Figure A-2 provides the block diagrams used for the roll and pitch attitude commands of the SCFD. 

As can be seen, linear lateral error was combined with angular error to form the roll attitude command. 

The bank angle required to maintain the desired flight path (from the flight plan information) was then 

added in to this signal and compared to the current bank angle. This result was then scaled and used to 

rotate the yellow pitch/roll command triangles with respect to the orange reference triangle.
133

Figure A- 2: Single-Cue Flight Director pitch and roll attitude command block diagram.

Similarly, vertical flight path and flight path-angle errors were combined in the pitch attitude 

command system and added to the normal acceleration required to maintain a coordinated turn with the 

current pitch and roll attitudes. This signal was then compared to the current normal acceleration with the 

difference being scaled and used to the drive the yellow pitch/roll command triangles up/down with

respect to the orange reference triangle. The Offset, Track, Altitude and Gamma gain parameters were

tuned to provide acceptable dynamic responses to linear (e.g., lateral and vertical) and angular (e.g.,

ground-track and flight path-angle) errors.

Table A-1 provides definitions of the various components of the SCFD system along with the actual 

values for the Offset, Track, Altitude and Gamma gains used for this flight test. These gains were 
134

evaluated in flight during checkout operations and adjusted accordingly to provide reasonable qualitative 

performance of the SCFD system. No specific adjustments were made to this system to compare with the 

Highway-In-The-Sky guidance system used for the Pathway-Based Guidance and Synthetic Vision 

System display concepts.

Table A-1. Flight Director Parameters

Symbol Type Description

Unit/

Value

Lateral_error Input Ownship lateral distance closest to the flight path ft

Path_track Input Flight plan track angle deg

Track_angle Input Actual ownship track angle deg

Turning_roll Input Desired roll angle (for curved path only) deg

V_total Var True airspeed ft/sec

G Const Gravitational acceleration 32.2 ft/sec2

Turn_radius Var Turning radius from the flight path Ft

Roll_angle Input Ownship actual roll angle Deg

Offset gain Const Lateral error gain 0.033 deg/ft

Track gain Const Track error gain 2.0 

vertical_error Input Current Ownship vertical flight path error ft

path_gamma Input Commanded vertical flight path angle deg

Gamma Input Actual vertical flight path angle deg

 Input Ownship roll angle rad

 Input Ownship pitch angle rad

Nz_coord Input Normal acceleration of a cordinated turn = 

coscos

g

Nz Input Actual normal acceleration g

Alt gain Const vertical_error gain 0.068deg/ft

gamma gain Const Vertical flight path angle gain 0.3 g/deg

 Const Pitch/roll flight director filter time constant sec
135

Appendix B: Highway-In-The-Sky Guidance System

The intent of the Highway-In-The-Sky (HITS) guidance system was to provide the general 

functionality of this type of flight guidance within the data set. In general, flight path guidance systems of 

this type provide lateral and vertical cues for the pilot to follow to achieve acceptable flight path errors. 

Figure B1 below shows what the Primary Flight Display (PFD) looked like for the Synthetic Vision 

System (SVS) display concept.

For the current study, a concept known as the Crows-Foot tunnel with tadpole guidance was selected. 

This concept provides good situation awareness of the intended flight path without cluttering the display 

and compromising the value of the SVS terrain and is described further in References 3, 4, 8, 9, 11 and 

12. The right-angle magenta triads formed the corner points of the desired flight path tunnel. The tunnel 

was set to 2 dots of localizer error and 1 dot of glideslope error and limited to 600 ft wide and 350 ft tall. 

The pilot’s task was to adjust the aircraft’s flight path and track angles to overlay the velocity vector 

(white circle with left, right, and top white fins) with the magenta guidance cue (tadpole). To provide 

enhanced flight path control, the velocity vector was quickened with pitch rate to eliminate the nonminimum phase response of the C206 to elevator commands. The tadpole guidance cue was located 

using modified pursuit guidance where the cue was located at a point along the desired flight path 

approximately 15 seconds in front of the aircraft’s current location as described in Reference B1. The top 

of the magenta guidance cue (also known as a tadpole) would lean in the direction of the upcoming turn 

to provide additional turn awareness. This particular image is indicating that the pilot needs to roll the 

aircraft to the right to eliminate a lateral error.

Appendix B references:

B1: Merrick, V.K., & Jeske, J.A., “Flight Path Synthesis and HUD Scaling for V/STOL Terminal 

Area Operations,” NASA TM-110348, April 1995.

Figure B- 1: SVS Primary Flight Display showing HITS guidance symbology.
136

Appendix C: Questionnaires

C.1 General

The questionnaires administered in support of this test are included in this appendix. They are listed 

as either post-run or post-test. Post-run questionnaires were those administered immediately after each 

run. Evaluation pilots were provided as much time as they needed to answer the basic block of post-run 

questions. The extended post-run questions were answered whenever time permitted, such as during 

initializations of the ILS approach since it took extra time to perform this operation, or during ATCinduced delays. Post-test questionnaires were those administered following a short break after all testing 

was complete.

C.2 Evaluation Pilot Questionnaire (EPQ) Tool

In order to expedite the acquisition of qualitative pilot rating data during the flight test, a hand-held 

pocket personal computer was used. Evaluation pilots were instructed to read the question aloud and 

then move the response indicator marker using the stylus to the selected response location. Evaluation 

pilots were also asked to provide any additional verbal comments before selecting the next page. Figure 

C-1 shows what the EPQ looked like to the EP for the performance question of the TLX questionnaire 

section. The response scale was similar for most questions with the end descriptors adjusted accordingly 

(i.e., good/poor or high/low etc.). For the DRR question block, the EPs responded yes/no to a series of 

questions and ultimately selected a single numerical score.

Figure C- 1: Example image of the Evaluation Pilot Questionnaire tool.
137

For the DRR questions, EPs first responded to a series of yes/no questions to navigate through the 

decision tree logic. They then selected the most appropriate response from a limited subset of DRRs. For 

example, if EPs answered yes to “Is the Display Readable?” and yes to “Is adequate performance 

attainable with a tolerable workload” and yes to “Is it satisfactory without improvement?”, EPs could then 

select from DRRs of 1, 2, or 3 with the associated Characteristics of Display Parameters and Demands on

Pilot During Scenario phrases.

C.3 Post-Run Questionnaires, basic block:

Instructions: Answer the following questions by moving the slider icon to the appropriate location. 

Press the “Next Question” button when finished.

Situation Awareness Rating Technique (SART)

SART 1: Demand on Attentional Resources: Rate your overall impression of the scenario in terms 

of how much attention and effort was required to successfully perform the task of piloting the aircraft. 

Items to consider include the likelihood of the situation to change suddenly (instability), the degree of 

complication associated with the scenario (complexity), and the number of variables changing during the 

scenario (variability).

SART 2: Supply of Attentional Resources: Rate your overall impression of the scenario in terms 

of how much spare attention was available for secondary tasks after the primary task of piloting the 

aircraft is accounted for. Items to consider include your degree of readiness for the primary task 

(arousal), your level of concentration during the primary task (concentration), the necessity and frequency 

of attention division between the primary task and any sub-tasks (division), and the amount of attention 

left over from the primary task to allocate to new variables of sub-tasks (spare capacity).

SART 3: Understanding of the Situation: Rate your overall understanding of what was occurring 

during the scenario. Items to consider are the amount of information received and understood during the 

scenario (quantity), the value of the information (quality), and the familiarity you may have had with what 

was taking place during the scenario.

The SART result is calculated by: SART 2 + SART 3 – SART 1

Terrain Awareness Question

Terrain Awareness: Rate your overall understanding of the terrain environment you were operating 

within. Items to consider are the quantity and quality of the terrain information available to you and how 

comfortable you were with your terrain awareness during this scenario. 

NASA Task Load Index (TLX) Workload Measurement Questionnaire

Mental Demand: How much mental and perceptual activity was required (e.g. thinking, deciding, 

calculating, remembering, looking, searching, etc.)? Was the task easy or demanding, simple or complex, 

exacting or forgiving?

Physical Demand: How much physical activity was required (e.g. pushing, pulling, turning, 

controlling, activating, etc.)? Was the task easy or demanding, slow or brisk, slack or strenuous, restful or 

laborious? 

Temporal Demand: How much time pressure did you feel due to the rate or pace at which the tasks 

or task elements occurred? Was the pace slow and leisurely or rapid and frantic? 
138

Performance: How successful do you think you were in accomplishing the goals of the task set by 

the experimenter (or yourself)? How satisfied were you with your performance in accomplishing these 

goals?

Effort: How hard did you have to work mentally and physically to accomplish your level of 

performance? 

Frustration Level: How insecure, discouraged, irritated, stressed, and annoyed versus secure, 

gratified, content, relaxed, and complacent did you feel during the task?

The TLX result is calculated by: (Mental Demand + Physical Demand + Temporal Demand + 

Performance + Effort + Frustration Level)/6.0. Since the Performance scale was oriented in the same 

direction as the other scales, that parameter was subtracted from the summation of the other parameters.

Hayworth-Newman Display Readability Rating (DRR) Scale:

The Hayworth-Newman Display Readability Rating scale is shown below is Figure C2. This rating 

scale is also referred to as a Modified Cooper-Harper rating in some literature. Evaluation pilots 

responded to the DRR questions in a decision tree format starting with the first question “Is the display 

readable?”. The DRR scale can be described as having three levels of response. Level 1 corresponds to 

responses 1, 2, or 3 and requires the EP to answer “yes” to the question of “Is it satisfactory without 

improvement?”. Level 2 corresponds to responses 4, 5, or 6 and requires the EP to answer “yes” to the 

question “Is adequate performance attainable with a tolerable workload?” but answer “no” to “Is it 

satisfactory without improvement?”. Level 3 corresponds to 7, 8, 9. A DRR of 10 exceeds Level 3. 

Evaluation Pilots were able to select DRR scores from Level 1, Level 2, Level 3, or DRR=10 depending 

on their responses to the initial decision tree questions.

Post-run questions, extended block:

GEOGRAPHICAL SA - Provide an estimate of your geographical situation awareness while flying 

the approach. Items to consider when marking your response include: knowledge of the location of your 

aircraft; other aircraft; terrain features; airports; cities; waypoints; position relative to obstacles; path 

conflicts; and climb/descent points.

SPATIAL SA - Provide an estimate of your spatial situation awareness while flying the approach. 

Items to consider when marking your response include: knowledge of your aircraft's attitude; altitude; 

heading; velocity; vertical speed; and desired flight path. 

TACTICAL SA - Provide an estimate of your tactical situation awareness while flying the approach. 

Items to consider when marking your response include: identification; location; and flight dynamics of 

other aircraft; own aircraft capabilities in relation to other nearby aircraft; and ATC communications.

PROJECTED SA - Provide an estimate of your ability to accurately predict the future state of your 

aircraft and the states of nearby aircraft while flying the approach. Items to consider when marking your 

response include: the projected status and dynamics of your aircraft; projected status and dynamics of 

other nearby aircraft; anticipation of possible future conflicts with traffic or obstacles; and threat 

prioritization. 

MENTAL WORKLOAD - Provide an estimate of the amount of mental workload required to 

successfully complete the assigned task of flying the approach.

PHYSICAL WORKLOAD - Provide an estimate of the amount of physical workload required to 

successfully complete the assigned task of flying the approach.
139

SPARE CAPACITY - Provide an estimate of the amount of mental resources remaining that could 

be used to process new information after all the necessary mental tasks associated with successfully flying 

the approach have been accounted for.

STRESS - Provide an estimate of the amount of stress (physical and mental) you experienced while 

completing the assigned task of flying the approach.

DATA LOCATION - The location of flight data on the displays critical to the task of flying the 

approach was efficiently organized minimizing the effort required to gather information. 

DATA VALUE- The flight data presented on the displays was complete, valuable, and had a 

significant impact on my ability to successfully fly the approach.

DATA CLARITY - The flight data presented on the displays while flying the approach was clear, 

simple to read, and easy to interpret.

DATA INTEGRATION - The flight data gathered from the displays was easily and rapidly 

integrated into a meaningful mental representation of the current and future operating status of my 

aircraft.

DATA CONTENT - The flight data presented on the displays was excessive and included 

unnecessary information
140

Figure C- 2: Hayworth-Newman Modified Cooper Harper (Display Readability).
141

Appendix D: Individual Subjective Data Plots

This appendix provides individual responses to the pilot questionnaires described in Appendix C and 

reported in the main body of the report. The data presented in this appendix include situation awareness

(SART), subjective workload (TLX), display readability rating (DRR), and terrain awareness (TA). The 

plots are arranged by approach (i.e., VFR, IFR, and VMC-like) with four plots per parameter for a total of 

12 plots. In the main body of the report, results and conclusions regarding the effects of display concept 

and approach are emphasized. The plots in this appendix are designed to show the effect of pilot groups 

and the variability of results across all pilots. The ranges of the SART, TLX, DRR, and TA variables are 

provided in Table D-1.

Table D- 1: Ranges of SART, TLX, DRR, and TA Parameters.

# Variable Max Min

1 SART 200 (great) -100 (none)

2 TLX 100 (high) 0 (low)

3 DRR 10 (very bad) 1 (excellent)

4 TA 100 (high) 0 (none)
142 

Figure D- 1: SART as a function of display concept and pilot type for each evaluation, VFR 

Approach. 

SVS

SVS

SVS

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

PBG

PBG

PBG

SCFD

SCFD

SCFD

BRD

BRD

BRD

SVS

SVS

SVS

PBG

PBG

PBG

SCFD

SCFD

SCFD

BRD

BRD

BRD

Display Concept

-100 -50 0 50 100 150 200

SART

H-IFR Pilots

IFR Pilots

VFR Pilots
143 

Figure D- 2: SART as a function of display concept and pilot type for each evaluation, ILS 

Approach. 

SVS

SVS

SVS

PBG

PBG

PBG

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

PBG

PBG

PBG

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

PBG

PBG

PBG

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

Display Concept

-100 -50 0 50 100 150 200

SART

H-IFR Pilots

IFR Pilots

VFR Pilots
144 

 Figure D- 3: SART as a function of display concept and pilot type, VMC-like Approach. 

SVS

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

Display Concept

-100 -50 0 50 100 150 200

SART

H-IFR Pilots

IFR Pilots

VFR Pilots
145 

Figure D- 4 TLX as a function of display concept and pilot type, VFR Approach. 

SVS

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

Display Concept

0 20 40 60 80 100

TLX

H-IFR Pilots

IFR Pilots

VFR Pilots
146 

Figure D- 5: TLX as a function of display concept and pilot type, ILS Approach. 

SVS

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

Display Concept

0 20 40 60 80 100

TLX

H-IFR Pilots

IFR Pilots

VFR Pilots
147 

Figure D- 6: TLX as a function of display concept and pilot type, VMC-like Approach. 

SVS

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

Display Concept

0 20 40 60 80 100

TLX

H-IFR Pilots

IFR Pilots

VFR Pilots
148 

Figure D- 7: DRR as a function of display concept and pilot type, VFR Approach. 

SVS

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

Display Concept

0 2 4 6 8 10

DRR

H-IFR Pilots

IFR Pilots

VFR Pilots
149 

Figure D- 8: DRR as a function of display concept and pilot type, ILS Approach. 

SVS

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

Display Concept

0 2 4 6 8 10

DRR

H-IFR Pilots

IFR Pilots

VFR Pilots
150 

Figure D- 9: DRR as a function of display concept and pilot type, VMC-like Approach. 

SVS

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

Display Concept

0 2 4 6 8 10

DRR

H-IFR Pilots

IFR Pilots

VFR Pilots
151 

Figure D- 10: Terrain awareness as a function of display concept and pilot type, VFR Approach. 

SVS

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

Display Concept

0 20 40 60 80 100

TA

H-IFR Pilots

IFR Pilots

VFR Pilots
152 

Figure D- 11: Terrain awareness as a function of display concept and pilot type, ILS Approach. 

SVS

SVS

SVS

PBG

PBG

PBG

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

SVS

PBG

PBG

PBG

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

SVS

SVS

SVS

PBG

PBG

PBG

SCFD

SCFD

SCFD

BRD

BRD

BRD

BRD

BRD

BRD

Display Concept

0 20 40 60 80 100

TA

H-IFR Pilots

IFR Pilots

VFR Pilots
153 

Figure D- 12: Terrain awareness as a function of display concept and pilot type, VMC-like 

Approach. 

SVS

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

SVS

SVS

SVS

SVS

SVS

SVS

PBG

PBG

PBG

PBG

PBG

PBG

SCFD

SCFD

SCFD

SCFD

SCFD

SCFD

Display Concept

0 20 40 60 80 100

TA

H-IFR Pilots

IFR Pilots

VFR Pilots
REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

2. REPORT TYPE 

Technical Publication

 4. TITLE AND SUBTITLE

General Aviation Flight Test of Advanced Operations Enabled by 

Synthetic Vision 

5a. CONTRACT NUMBER

 6. AUTHOR(S)

Glaab, Louis J.; Hughes, Monica F.; Parrish, Russell V.; Takallu, 

Mohammad A.

 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

NASA Langley Research Center

Hampton, VA 23681-2199

 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration

Washington, DC 20546-0001

 8. PERFORMING ORGANIZATION

 REPORT NUMBER

L-20203

10. SPONSOR/MONITOR'S ACRONYM(S)

NASA

13. SUPPLEMENTARY NOTES

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified - Unlimited

Subject Category 03

Availability: NASA CASI (443) 757-5802

19a. NAME OF RESPONSIBLE PERSON

STI Help Desk (email: help@sti.nasa.gov)

14. ABSTRACT

A flight test was performed to compare the use of three advanced primary flight and navigation display concepts to a baseline, round-dial concept to assess the 

potential for advanced operations. The displays were evaluated during visual and instrument approach procedures including an advanced instrument approach 

resembling a visual airport traffic pattern. Nineteen pilots from three pilot groups, reflecting the diverse piloting skills of the General Aviation pilot 

population, served as evaluation subjects. The experiment had two thrusts: 1) an examination of the capabilities of low-time (i.e., <400 hours), 

non-instrument-rated pilots to perform nominal instrument approaches, and 2) an exploration of potential advanced Visual Meteorological Conditions 

(VMC)-like approaches in Instrument Meteorological Conditions (IMC). Within this context, advanced display concepts are considered to include integrated 

navigation and primary flight displays with either aircraft attitude flight directors or Highway In The Sky (HITS) guidance with and without a synthetic 

depiction of the external visuals (i.e., synthetic vision). Relative to the first thrust, the results indicate that using an advanced display concept, as tested herein, 

low-time, non-instrument-rated pilots can exhibit flight-technical performance, subjective workload and situation awareness ratings as good as or better than 

high-time Instrument Flight Rules (IFR)-rated pilots using Baseline Round Dials for a nominal IMC approach. For the second thrust, the results indicate 

advanced VMC-like approaches are feasible in IMC, for all pilot groups tested for only the Synthetic Vision System (SVS) advanced display concept. 

15. SUBJECT TERMS

Enhanced vision; Flight tests; General aviation aircraft; Instrument flight rules; Synthetic vision system; Thrust

18. NUMBER

 OF 

 PAGES

158

19b. TELEPHONE NUMBER (Include area code)

(443) 757-5802

a. REPORT

U

c. THIS PAGE

U

b. ABSTRACT

U

17. LIMITATION OF 

 ABSTRACT

UU

Prescribed by ANSI Std. Z39.18

Standard Form 298 (Rev. 8-98)

3. DATES COVERED (From - To)

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

 284848.02.04.07.01 

11. SPONSOR/MONITOR'S REPORT

 NUMBER(S)

NASA/TP-2014-218279

16. SECURITY CLASSIFICATION OF:

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 

gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 

collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and 

Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person 

shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY)

01- 07 - 2014