November 2013

NASA/TP–2013-218054

Enhanced Flight Vision Systems and Synthetic 

Vision Systems for NextGen Approach and 

Landing Operations

Lynda J. Kramer, Randall E. Bailey, Kyle K. E. Ellis, Steven P. Williams, 

Jarvis J. Arthur III, Lawrence J. Prinzel III, and Kevin J. Shelton

Langley Research Center, Hampton, Virginia
NASA STI Program . . . in Profile

Since its founding, NASA has been dedicated to the 

advancement of aeronautics and space science. The 

NASA scientific and technical information (STI) 

program plays a key part in helping NASA maintain 

this important role.

The NASA STI program operates under the 

auspices of the Agency Chief Information Officer. 

It collects, organizes, provides for archiving, and 

disseminates NASA’s STI. The NASA STI 

program provides access to the NASA Aeronautics 

and Space Database and its public interface, the 

NASA Technical Report Server, thus providing one 

of the largest collections of aeronautical and space 

science STI in the world. Results are published in 

both non-NASA channels and by NASA in the 

NASA STI Report Series, which includes the 

following report types:

 TECHNICAL PUBLICATION. Reports of 

completed research or a major significant phase 

of research that present the results of NASA 

Programs and include extensive data or 

theoretical analysis. Includes compilations of 

significant scientific and technical data and 

information deemed to be of continuing 

reference value. NASA counterpart of peerreviewed formal professional papers, but 

having less stringent limitations on manuscript 

length and extent of graphic presentations.

 TECHNICAL MEMORANDUM. Scientific 

and technical findings that are preliminary or of 

specialized interest, e.g., quick release reports, 

working papers, and bibliographies that contain 

minimal annotation. Does not contain extensive 

analysis.

 CONTRACTOR REPORT. Scientific and 

technical findings by NASA-sponsored 

contractors and grantees.

 CONFERENCE PUBLICATION. 

Collected papers from scientific and 

technical conferences, symposia, seminars, 

or other meetings sponsored or cosponsored by NASA.

 SPECIAL PUBLICATION. Scientific, 

technical, or historical information from 

NASA programs, projects, and missions, 

often concerned with subjects having 

substantial public interest.

 TECHNICAL TRANSLATION. 

English-language translations of foreign 

scientific and technical material pertinent to 

NASA’s mission.

Specialized services also include organizing 

and publishing research results, distributing 

specialized research announcements and feeds, 

providing information desk and personal search 

support, and enabling data exchange services.

For more information about the NASA STI 

program, see the following:

 Access the NASA STI program home page 

at http://www.sti.nasa.gov

 E-mail your question to help@sti.nasa.gov

 Fax your question to the NASA STI 

Information Desk at 443-757-5803

 Phone the NASA STI Information Desk at 

443-757-5802

 Write to:

 STI Information Desk

 NASA Center for AeroSpace Information

 7115 Standard Drive

 Hanover, MD 21076-1320
National Aeronautics and 

Space Administration

Langley Research Center 

Hampton, Virginia 23681-2199

November 2013

NASA/TP–2013-218054

Enchanced Flight Vision Systems and Synthetic 

Vision Systems for NextGen Approach and 

Landing Operations

Lynda J. Kramer, Randall E. Bailey, Kyle K. E. Ellis, Steven P. Williams, 

Jarvis J. Arthur III, Lawrence J. Prinzel III, and Kevin J. Shelton

Langley Research Center, Hampton, Virginia
Available from:

NASA Center for AeroSpace Information

7115 Standard Drive

Hanover, MD 21076-1320

443-757-5802

Acknowledgments

This work was jointly sponsored by NASA’s Aviation Safety Program (AvSP), Vehicle 

Systems Safety Technologies project, led by Dr. Steve Young, Project Scientist) and the 

FAA Human Factors R&D Project for NextGen, led by Dr. Tom McCloy, Mr. Dan 

Herschler, and Mr. Stephen Plishka. This cooperative research was formally established 

under an Interagency Agreement between the FAA and NASA (IA1-973, Technical 

Direction 5). Programmatic support and technical advice from these groups was 

gratefully appreciated. The support and advice from Mrs. Terry King and Mr. Dennis 

Mills, FAA, is also greatly appreciated.

The use of trademarks or names of manufacturers in this report is for accurate reporting and does not 

constitute an official endorsement, either expressed or implied, of such products or manufacturers by the 

National Aeronautics and Space Administration.
Table of Contents 

LIST OF FIGURES................................................................................................................................... 3

LIST OF TABLES .................................................................................................................................... 4

SYMBOLS, ACRONYMS AND DEFINITIONS ............................................................................................ 5

1 INTRODUCTION............................................................................................................................. 7

2 BACKGROUND............................................................................................................................... 9

3 METHODOLOGY .......................................................................................................................... 12

3.1 SUBJECTS.........................................................................................................................................12

3.2 SIMULATION FACILITY ........................................................................................................................ 12

3.2.1 Simulator Database............................................................................................................... 13

3.2.2 Audio Effects.......................................................................................................................... 13

3.2.3 Head‐Down Displays ............................................................................................................. 13

3.2.4 Head‐Up Display.................................................................................................................... 14

3.2.5 Eye Tracking System.............................................................................................................. 17

3.3 SV SIMULATIONS .............................................................................................................................. 18

3.4 EV SIMULATION................................................................................................................................18

3.5 NAVIGATIONAL PERFORMANCE VARIATIONS.......................................................................................... 19

3.6 AUTOMATIC DEPENDENT SURVEILLANCE ‐ BROADCAST ........................................................................... 19

3.7 CREW DISPLAY CONCEPTS .................................................................................................................. 19

3.7.1 Head‐Down Flight Display Concepts ..................................................................................... 19

3.7.2 Head‐Up Flight Display Concepts.......................................................................................... 21

3.8 SIMULATOR RECORDING .................................................................................................................... 21

3.9 EVALUATION TASK ............................................................................................................................ 22

3.10 CREW PROCEDURES ....................................................................................................................... 22

3.10.1 Baseline and SVS Procedures................................................................................................. 22

3.10.2 EFVS Procedures.................................................................................................................... 24

3.11 EXPERIMENT MATRIX..................................................................................................................... 25

3.12 MEASURES ...................................................................................................................................28

3.13 TEST CONDUCT ............................................................................................................................. 29

4 RESULTS...................................................................................................................................... 30

4.1 METRICS..........................................................................................................................................31

4.1.1 Flight Performance................................................................................................................ 31

4.1.2 Pilot Workload....................................................................................................................... 33

4.1.3 Eye Gaze ................................................................................................................................33

4.1.4 Display Concept Preferences................................................................................................. 34

4.1.5 CDTI Influences on Situation Awareness and Traffic Awareness .......................................... 35

4.2 EFVS OPERATIONAL CONCEPT COMPARISONS....................................................................................... 35

4.2.1 Approach Performance – Instrument Segment..................................................................... 35

4.2.2 Approach Performance – Visual Segment............................................................................. 37

4.2.3 Landing Performance ............................................................................................................ 40

4.2.4 Rollout Performance.............................................................................................................. 43
2 

4.2.5 Pilot Workload....................................................................................................................... 44

4.2.6 Eye Tracking .......................................................................................................................... 44

4.3 SVS OPERATIONAL CONCEPT COMPARISONS......................................................................................... 47

4.3.1 Approach Performance – Instrument Segment..................................................................... 47

4.3.2 Approach Performance – Visual Segment............................................................................. 52

4.3.3 Landing Performance ............................................................................................................ 56

4.3.4 Rollout Performance.............................................................................................................. 60

4.3.5 Pilot Workload....................................................................................................................... 60

4.3.6 Eye Tracking .......................................................................................................................... 61

4.4 CREW DISPLAY CONCEPT PREFERENCES ................................................................................................ 64

4.5 CDTI AND SEVS INFLUENCES ON TRAFFIC AWARENESS ........................................................................... 65

4.5.1 Traffic Awareness Probe........................................................................................................ 65

4.5.2 Paired Comparisons for Situation Awareness and Traffic Awareness ..................................67

4.5.3 Unexpected Runway Incursion .............................................................................................. 67

4.5.4 Expected Runway Incursions................................................................................................. 70

5 OPERATIONAL CREDIT DISCUSSION OF RESULTS.......................................................................... 71

5.1 EFVS OPERATIONAL CREDIT ............................................................................................................... 71

5.1.1 EFVS Instrument Segment..................................................................................................... 72

5.1.2 EFVS Visual Segment............................................................................................................. 72

5.1.3 EFVS Landing and Roll‐Out.................................................................................................... 74

5.1.4 EFVS Clutter and Obscuration ............................................................................................... 75

5.1.5 EFVS Off‐Nominal Operations............................................................................................... 75

5.2 EFVS SUMMARY............................................................................................................................... 77

5.3 SVS OPERATIONAL CREDIT ................................................................................................................. 77

5.3.1 SVS Instrument Segment....................................................................................................... 78

5.3.2 SVS Runway Visual Acquisition.............................................................................................. 82

5.3.3 SVS Visual Segment............................................................................................................... 83

5.3.4 SVS Clutter and Obscuration ................................................................................................. 84

5.3.5 SVS Off‐Nominal Operations................................................................................................. 85

5.4 SVS SUMMARY.................................................................................................................................87

5.5 TRAFFIC AWARENESS AS INFLUENCED BY CDTI AND SEVS ....................................................................... 88

5.6 PILOT‐MONITORING AND SEVS .......................................................................................................... 88

6 CONCLUSIONS............................................................................................................................. 92

7 REFERENCES................................................................................................................................ 94

8 APPENDIX A ................................................................................................................................ 96

8.1 AIR FORCE FLIGHT TEST CENTER (AFFTC) WORKLOAD ESTIMATE SCALE.................................................... 96

8.2 PAIRED COMPARISON TECHNIQUE ....................................................................................................... 97

8.3 EXAMPLE TRAFFIC PROBE FOR RUNWAY 22R ........................................................................................ 99
3 

List of Figures 

FIGURE 1.  RESEARCH FLIGHT DECK SIMULATOR WITH HUD, HEAD‐DOWN INSTRUMENT PANEL, AND EFB....................13

FIGURE 2.  EFVS HUD (LEFT) AND SVS HUD (RIGHT)........................................................................................... 14

FIGURE 3.  HUD SYMBOLOGY ........................................................................................................................... 16

FIGURE 4.  HUD FLARE CUE.............................................................................................................................. 16

FIGURE 5.  RFD SMART EYE CAMERA AND FLASHER LOCATIONS.............................................................................. 17

FIGURE 6.  AIRBORNE HEAD‐DOWN DISPLAY (HDD) CONCEPTS: CONVENTIONAL HDD ON TOP AND SVS HDD ON BOTTOM

.............................................................................................................................................................20

FIGURE 7.  SURFACE HEAD‐DOWN DISPLAY (HDD) CONCEPTS: CONVENTIONAL HDD ON TOP AND SVS HDD ON BOTTOM

.............................................................................................................................................................21

FIGURE 8.  CDTI FORMATS – NONE (LEFT), MOVING MAP (CENTER), AND MOVING MAP/RUNWAY INSET (RIGHT) ........21

FIGURE 9.  LANDING PERFORMANCE ASSESSMENT................................................................................................. 30

FIGURE 10.  EYE TRACKING ANALYSIS SEGMENTS.................................................................................................. 34

FIGURE 11.  TOUCHDOWN FOOTPRINT FOR EFVS OPERATIONAL CONCEPTS.............................................................. 42

FIGURE 12.  TOUCHDOWN PITCH ANGLE AND BANK ANGLE FOOTPRINT FOR EFVS OPERATIONAL CONCEPTS .................43

FIGURE 13.  EFVS OP CONCEPT, HEAD UP TIME (%)............................................................................................ 45

FIGURE 14.  EFVS OP CONCEPTS, HEAD UP/DOWN TRANSITIONS .......................................................................... 46

FIGURE 15.  TOUCHDOWN FOOTPRINT FOR SVS OPERATIONAL CONCEPTS................................................................ 59

FIGURE 16.  TOUCHDOWN PITCH ANGLE AND BANK ANGLE FOOTPRINT FOR SVS OPERATIONAL CONCEPTS .....................59

FIGURE 17.  SVS OP CONCEPTS, HEAD UP TIME (%) ............................................................................................ 61

FIGURE 18.  SVS OP CONCEPTS, HEAD UP/DOWN TRANSITIONS ............................................................................ 63

FIGURE 19.  PF AND PM DISPLAY CONCEPT PREFERENCE RANKINGS ....................................................................... 65

FIGURE 20.  RUNWAY INCURSION AIRCRAFT AS SEEN ON FLIR ON PM HEAD‐DOWN DISPLAY. ......................................68

FIGURE 21.  AIRCRAFT POSITION AT 100 FT HAT, CONVENTIONAL HDD VS. EFVS .................................................... 73

FIGURE 22.  AIRCRAFT POSITION AT THE THRESHOLD, CONVENTIONAL HDD VS. EFVS ............................................... 74

FIGURE 23.  AIRCRAFT POSITION AT 200 FT HAT, CONVENTIONAL HUD VS. SVS HUD.............................................. 79

FIGURE 24.  AIRCRAFT POSITION AT 200 FT HAT, CONVENTIONAL HUD VS. SVS HDD.............................................. 80

FIGURE 25.  AIRCRAFT POSITION AT 100 FT HAT, CONVENTIONAL HUD VS. SVS HUD.............................................. 81

FIGURE 26.  AIRCRAFT POSITION AT 100 FT HAT, CONVENTIONAL HUD VS. SVS HDD.............................................. 81

FIGURE 27.  TWO‐CREW VS. SINGLE PILOT SVS OPERATIONAL COMPARISON OTW/HDD TRANSITION COUNT .............91
4 

List of Tables 

TABLE 1. BASELINE AND SVS CREW PROCEDURES................................................................................................. 23

TABLE 2. REQUIRED VISUAL REFERENCES............................................................................................................. 24

TABLE 3. EFVS CREW PROCEDURES ................................................................................................................... 25

TABLE 4. EXPERIMENT MATRIX FOR TWO‐CREW OPERATIONS ................................................................................ 26

TABLE 5. EXPERIMENT MATRIX FOR SINGLE PILOT OPERATIONS .............................................................................. 28

TABLE 6. TOUCHDOWN PERFORMANCE CRITERIA ................................................................................................. 30

TABLE 7. QUANTITATIVE APPROACH PERFORMANCE STANDARDS............................................................................. 32

TABLE 8. APPROACH STATISTICS (1000 FT TO DH) FOR EFVS OPERATIONAL CONCEPTS ............................................. 36

TABLE 9. PERCENTAGE APPROACHES SUCCESSFULLY MEETING APPROACH PERFORMANCE STANDARDS. ..........................37

TABLE 10. PROBABILITY OF SUCCESS IN MEETING AWO EXCEEDANCE CRITIERIA......................................................... 37

TABLE 11. LATERAL PATH PERFORMANCE STATISTICS OF EFVS OPERATIONAL CONCEPTS IN THE VISUAL SEGMENT OF THE

APPROACH..............................................................................................................................................38

TABLE 12. VERTICAL PATH PERFORMANCE STATISTICS OF EFVS OPERATIONAL CONCEPTS IN THE VISUAL SEGMENT OF THE

APPROACH..............................................................................................................................................39

TABLE 13. SINK RATE STATISTICS OF EFVS OPERATIONAL CONCEPTS IN THE VISUAL SEGMENT OF THE APPROACH..........39

TABLE 14. BANK ANGLE STATISTICS OF EFVS OPERATIONAL CONCEPTS ................................................................... 40

TABLE 15. LANDING STATISTICS FOR EFVS OPERATIONAL CONCEPTS....................................................................... 41

TABLE 16. ROLLOUT STATISTICS FOR EFVS OPERATIONAL CONCEPTS....................................................................... 43

TABLE 17. EFVS OP CONCEPTS, HEAD UP TIME (%)............................................................................................. 45

TABLE 18. EFVS OP CONCEPTS, HEAD UP/HEAD DOWN TRANSITIONS.................................................................... 47

TABLE 19. APPROACH STATISTICS (1000 FT TO DH) FOR SVS OPERATIONAL CONCEPTS ............................................. 48

TABLE 20. GO‐AROUND EVENT DESCRIPTION WITH SVS OPERATIONAL CONCEPTS .................................................... 49

TABLE 21. PERCENTAGE SVS OPERATONAL CONCEPT APPROACHES SUCCESSFULLY MEETING LOCALIZER PERFORMANCE

STANDARDS .............................................................................................................................................51

TABLE 22. PERCENTAGE SVS OPERATONAL CONCEPT APPROACHES SUCCESSFULLY MEETING GLIDESLOPE PERFORMANCE

STANDARDS .............................................................................................................................................51

TABLE 23. PROBABILITY OF SUCCESS IN MEETING AWO EXCEEDANCE CRITIERIA WITH SVS OPERATIONAL CONCEPTS .......51

TABLE 24. LATERAL PATH PERFORMANCE STATISTICS OF SVS OPERATIONAL CONCEPTS IN THE VISUAL SEGMENT OF THE

APPROACH..............................................................................................................................................53

TABLE 25. VERTICAL PATH PERFORMANCE STATISTICS OF SVS OPERATIONAL CONCEPTS IN THE VISUAL SEGMENT OF THE

APPROACH..............................................................................................................................................54

TABLE 26. SINK RATE STATISITICS OF SVS OPERATIONAL CONCEPTS IN THE VISUAL SEGMENT OF THE APPROACH ...........55

TABLE 27. BANK ANGLE STATISTICS OF SVS OPERATIONAL CONCEPTS ..................................................................... 56

TABLE 28. LANDING STATISTICS FOR SVS OPERATIONAL CONCEPTS......................................................................... 57

TABLE 29. ROLLOUT STATISTICS FOR SVS OPERATIONAL CONCEPTS......................................................................... 60

TABLE 30. SVS OP CONCEPTS, HEAD UP TIME (%)............................................................................................... 62

TABLE 31. SVS OP CONCEPTS, HEAD UP/DOWN TRANSITIONS .............................................................................. 64

TABLE 32. STATISTICS FROM FIRST OCCURRENCE OF TRAFFIC PROBE ....................................................................... 66

TABLE 33. STATISTICS FROM ALL OTHER OCCURRENCES OF TRAFFIC PROBE .............................................................. 66

TABLE 34. UNEXPECTED RUNWAY INCURSION RUN DATA. ..................................................................................... 69

TABLE 35. LATERAL NAVIGATION SYSTEM ERROR TOUCHDOWN PERFORMANCE........................................................ 85

TABLE 36. VERTICAL NAVIGATION SYSTEM ERROR TOUCHDOWN PERFORMANCE ...................................................... 86

TABLE 37. TOUCHDOWN STATISTICS BY CREW COMPLEMENT................................................................................. 89
5 

Symbols, Acronyms and Definitions 

Dimensional quantities are presented in both the International System of Units and U.S. Customary Units. 

Measurements and calculations were made in the U.S. Customary Units. 

ADS-B Automatic Dependent Surveillance – Broadcast 

AFFTC Air Force Flight Test Center 

AFL Above Field Level 

AGL Above Ground Level 

ANOVA Analysis of Variance 

AWO All Weather Operations 

CDTI Cockpit Display of Traffic Information 

CFR Code of Federal Regulations 

DA Decision Altitude 

DH Decision Height 

DEM Digital Elevation Model 

DH Decision Height 

DOT Department of Transportation 

EFB Electronic Flight Bag 

EFVS Enhanced Flight Vision System 

EUROCAE European Organisation for Civil Aviation Equipment 

EV Enhanced Vision 

EVO Equivalent Visual Operations 

EVS Enhanced Vision System 

FAA Federal Aviation Administration 

FAR Federal Aviation Regulation 

FLIR Forward Looking InfraRed 

FMS Flight Management System 

FOV Field of View 

FPM feet per minute 

fps feet per second 

ft feet 

FTE Flight Technical Error 

GA General Aviation 

H Horizontal 

HAT Height Above Threshold 

HDD Head-Down Display 

HGS Head-up Guidance System 

HSI Horizontal Situation Indicator 

HUD Head-Up Display 

IFR Instrument Flight Rules 

ILS Instrument Landing System 

JAR Joint Aviation Regulations 

LaRC Langley Research Center 

MALSR Medium intensity Approach Lighting System with Runway alignment indicator lights 

MDA Minimum Descent Altitude 

ms millisecond 

m/sec meters per second 

NACp Navigation Accuracy Category for Position 
6 

NACv Navigation Accuracy Category for Velocity 

ND Navigation Display 

NextGen Next Generation Air Transportation System 

nm nautical mile 

ORD FAA airport identifier for Chicago O’Hare International Airport 

OTW Out-The-Window 

Part 23 Airworthiness Standards for Normal, Utility, Acrobatic, and Commuter Category 

Airplanes as defined in Title 14 of the Code of Federal Regulations 

Part 25 Airworthiness Standards for Transport Category Airplanes as defined in Title 14 of the 

Code of Federal Regulations 

Part 121 Operating Requirements for Domestic, Flag, and Supplemental Operations as defined in 

Title 14 of the Code of Federal Regulations 

Part 135 Operating Requirements for Commuter and On Demand Operations and Rules Governing 

Persons On Board such Aircraft as defined in Title 14 of the Code of Federal Regulations 

PF Pilot Flying 

PFD Primary Flight Display 

PM Pilot Monitoring 

RFD Research Flight Deck 

RMS root-mean-square 

RVR Runway Visual Range 

RTCA Radio Technical Commission for Aeronautics 

SA Situation Awareness 

SA-SWORD Situation Awareness – Subjective Workload Dominance 

SEVS Synthetic/Enhanced Vision Systems 

SURF IA Airport Surface with Indications and Alerts 

SV Synthetic Vision 

SVS Synthetic Vision System 

T/D Touchdown 

TDZ/CL Touchdown Zone/Centerline 

TDZE Touchdown Zone Elevation 

TOGA Take-Off, Go-Around 

V Vertical 

VFR Visual Flight Rules 

VS Vision Systems 

WAAS Wide Area Augmentation System 
7 

Abstract 

Synthetic Vision Systems and Enhanced Flight Vision System 

(SVS/EFVS) technologies have the potential to provide additional 

margins of safety for aircrew performance and enable operational 

improvements for low visibility operations in the terminal area 

environment with equivalent efficiency as visual operations. To meet this 

potential, research is needed for effective technology development and 

implementation of regulatory standards and design guidance to support 

introduction and use of SVS/EFVS advanced cockpit vision technologies 

in Next Generation Air Transportation System (NextGen) operations. 

A fixed-base pilot-in-the-loop simulation test was conducted at NASA 

Langley Research Center that evaluated the use of SVS/EFVS in NextGen 

low visibility approach and landing operations. Twelve crews flew 

approach and landing operations in a simulated NextGen Chicago 

O’Hare environment. Various scenarios tested the potential for using 

EFVS to conduct approach, landing, and roll-out operations in visibility 

as low as 1000 feet runway visual range (RVR). Also, SVS was tested to 

evaluate the potential for lowering decision heights (DH) on certain 

instrument approach procedures below what can be flown today. 

Expanding the portion of the visual segment in which EFVS can be used 

in lieu of natural vision from 100 feet above the touchdown zone 

elevation to touchdown and rollout in visibilities as low as 1000 feet 

RVR appears to be viable as touchdown performance was acceptable 

without any apparent workload penalties. A lower DH of 150 feet and/or 

possibly reduced visibility minima using SVS appears to be viable when 

implemented on a Head-Up Display, but the landing data suggests 

further study for head-down implementations. 

1 Introduction 

The U.S. air transportation system is undergoing a transformation to accommodate the movement of 

large numbers of people and goods in a safe, efficient, and reliable manner [1]. One of the key 

capabilities envisioned to achieve this Next Generation Air Transportation System (NextGen) is the 

concept of equivalent visual operations (EVO). EVO is the capability to achieve the safety of 

current-day Visual Flight Rules (VFR) operations and maintain the operational tempos of VFR 

irrespective of the weather and visibility conditions. 

One research challenge for EVO is the definition of required equipage on the aircraft and at the 

airport. With today’s equipment and regulations, significant investment is required in on-board 

equipment for navigation, surveillance, and flight control and on the airport for precision guidance 

systems and approach lighting systems for “all-weather” landing capability [2]. The levels of 

equipment redundancy, capability, maintenance, performance and crew training dramatically increase 

as landing visibility minima decrease. Synthetic Vision Systems and Enhanced Flight Vision 

Systems (SVS/EFVS) offer a means of providing EVO capability without significant airport 

infrastructure investment while potentially increasing efficiency and throughput during low visibility 

operations. 
8 

NASA Langley Research Center (NASA LaRC) and the Department of Transportation/Federal 

Aviation Administration (DOT/FAA) are jointly conducting collaborative research to ensure effective 

technology development and implementation of regulatory standards and design guidance to support 

the introduction and use of SVS/EFVS advanced cockpit vision technologies in NextGen operations. 

These technologies have the potential to enable operational improvements that would benefit low 

visibility surface, arrival, and departure operations in the terminal environment with equivalent 

efficiency as visual operations. This work builds from and extends the current operational use and 

certification of existing SVS/EFVS technologies toward all-weather, low visibility operations for 

NextGen. 

In addition, under NASA’s Aviation Safety Program, research is being conducted to evaluate the 

influence of Cockpit Display of Traffic Information (CDTI) and SVS/EFVS technologies on a pilot’s 

situation and traffic awareness during low visibility surface operations. This research is motivated in 

part by the FAA’s 2010 Annual Runway Safety Report [3], which identifies planned mid-term (2012-

2018) NextGen research initiatives that include the use of and integration of CDTI and SVS/EFVS 

technologies. As described in this FAA report, under low visibility operations, “Location information 

of aircraft and vehicles on the airport surface will be displayed on moving maps using Cockpit 

Display of Traffic Information (CDTI) or aided by Enhanced Flight Vision Systems (EFVS), 

Enhanced Vision Systems (EVS), Synthetic Vision Systems (SVS), or other types of advanced vision 

or virtual vision technology.” 
9 

2 Background 

SVS is a computer-generated image of the external scene topography, generated using aircraft 

attitude, high-precision navigation, and data of the terrain, obstacles, cultural features, and other 

required flight information. EFVS is an electronic means to provide a display (typically on a head-up 

display, or HUD) of the external scene by use of an imaging sensor, such as a Forward-Looking 

InfraRed (FLIR) or millimeter wave radar. Both SVS and EFVS are “vision-based” technologies 

intended to create, supplement, or enhance the natural vision of the pilot. 

NASA and others have developed and shown SVS technologies that provide significant 

improvements in terrain awareness and reductions for the potential of Controlled-Flight-Into-Terrain 

incidents/accidents [4-6], improvements in Flight Technical Error (FTE) to meet Required Navigation 

Performance criteria [7,8], and improvements in Situation Awareness (SA) without concomitant 

increases in workload compared to current generation cockpit technologies [9-13]. As such, SVS, 

often displayed on a Head-Down Display (HDD), is emerging as standard equipage for Part 23 and 

Part 25 business and General Aviation (GA) aircraft flight decks even though, to date, no “operational 

credit” is obtained by SVS equipage [14]. Operational credit is a specific benefit afforded the aircraft 

operator from application of FAA Advisory Circulars. 

EFVS capability on a HUD using FLIR sensor technology has garnered a significant share of the 

business aircraft market and is growing in Part 121 and 135 operations [15]. EFVS provides many of 

the same operational benefits as SVS technology, but it uses a real-time view of the external 

environment, independent of the aircraft navigation solution or database. These differences, in part, 

enable operational credit by use of an approved EFVS. In 2004, Title 14 of the Code of Federal 

Regulations (CFR) Section (§) 91.175 was amended to enable operators conducting straight-in 

instrument approach procedures (in other than Category II or Category III operations) to descend 

below the published Decision Altitude (DA), Decision Height (DH) or Minimum Descent Altitude 

(MDA) down to 100 feet (ft) above the touchdown zone elevation (TDZE) using an approved EFVS 

in lieu of natural vision. The enhanced flight visibility provided by the EFVS must meet or exceed 

the published visibility for the approach being flown and the required visual references to descend 

from the DA/DH/MDA to 100 ft above the TDZE must be in view on the EFVS. An approved EFVS 

must meet the requirements of § 91.175(m) and must be presented on a HUD or an equivalent headup display that might be found acceptable to the FAA. In order to descend below 100 feet above the 

touchdown zone elevation, natural vision must be used. 

Synthetic and Enhanced Vision Systems (SEVS) technologies, such as SVS/EFVS in combination 

with HDD/HUD, form the basis for an electronic display of visual flight references (terrain, obstacles, 

and operations-critical navigational and situational references) on electronic cockpit display(s) for the 

flight crew. Integrating these SEVS displays with conformal symbology provides important situation, 

guidance, and/or command information as necessary and/or appropriate to enable all weather 

approach and landing operations. The primary reference for maneuvering the airplane is based on 

what the pilot sees through the SEVS, in lieu of or supplemental to the pilot’s natural vision, in low 

visibility conditions. 

The key concept of the revisions to 14 CFR§ 91.175 is that an EFVS can be used in lieu of natural 

vision from the DA/DH/MDA to 100 ft height above the TDZE provided the enhanced vision image 

in the HUD meets or exceeds the published visibility required for the approach being flown and 

required visual references are in view. Minimum aviation system performance standards are now 

available in Radio Technical Commission for Aeronautics (RTCA) DO-315 [16]. In addition, FAA 

Advisory Circular AC 20-167 [17] provides guidance on certification and installation of EFVS and 
10 

FAA Advisory Circular AC 90-106 [18] provides guidance for obtaining operational approval to use 

EFVS in lieu of natural vision to descend below DA/DH or MDA. RTCA DO-315 also provides 

performance standards for SVS but with no additional operational credit. In other words, installing 

SVS does not change the airplane’s existing operational capability. It can only be used for Situation 

Awareness. 

The emerging challenge for NextGen, and the subject of NASA research, is to develop performancebased standards for SEVS technologies that create EVO. The first part of this challenge is the 

development of performance-based standards that could lead to future operational approvals for 

SVS/EFVS. The second part of this challenge is to evaluate the influence of SEVS technologies and 

other emerging flight-deck information sources such as CDTI and their potential use to achieve EVO. 

The design and use of SEVS technology is integrally tied to the operating rules for landing and takeoff under Instrument Flight Rules (IFR) (14 CFR § 91.175). For EFVS operations, § 91.175 was 

modified such that, to operate below the DA/DH or MDA, the pilot must determine that the enhanced 

flight visibility is not less than that published for the instrument approach being used and certain 

visual references must be seen using the EFVS to continue the descent below the published DA/DH 

or MDA (see 14 CFR § 91.175). No lower than 100 ft height above the TDZE, the visual references 

that positively identify the runway of intended landing must be distinctly visible and identifiable 

using natural vision. 

The FAA started a rulemaking project to expand operational credit for EFVS beyond what is 

currently authorized by the regulations (under 14 CFR § 91.175) [15, 19]. RTCA DO-315A [20] was 

drafted to establish performance standards in concert with this rulemaking project. Minimum system 

performance standards are now published for EFVS operations through the approach to touchdown in 

visibility as low as 1000 ft runway visual range (RVR) by sole use of an approved EFVS in lieu of 

natural vision. Simply stated, (in the RTCA DO-315A MASPS) the visual segment of the approach 

can now be accomplished by using either enhanced flight visibility or natural vision. Past NASA 

research [21] supports the viability of this expanded EFVS operational credit where it was shown that 

using an EFVS to hand fly approaches through touchdown resulted in excellent localizer tracking 

performance (less than 1/3 dot localizer deviation between 300 ft and 100 ft HAT) and an 

improvement in glideslope tracking performance over hand-flown EFVS approaches to touchdown 

where natural vision, not enhanced vision, was required to positively identify the runway of intended 

landing. 

The joint RTCA SC-213 and European Organisation for Civil Aviation Equipment (EUROCAE) 

Working Group 79 committee also drafted RTCA DO-315B [22] to establish minimum performance 

standards for possible operational credit for SVS. Unlike EFVS, the possible path for operational 

credit is not through revision of 14 CFR § 91.175, but is based on FAA Order 8400.13 (“Procedures 

for the Evaluation and Approval of Facilities for Special Authorization Category I Operations and All 

Category II and III Operations”). Specifically, RTCA DO-315B establishes performance standards 

for SVS enabling lower than standard Category I minima or a reduction in the required minimum 

visibility. These RTCA DO-315B performance standards for SVS operational credit do not require 

the use of a HUD. 

A fixed-base experiment was conducted to evaluate selected elements of the proposed performance 

standards for expanded EFVS and SVS operational credits. Specifically, the high-level objectives of 

this simulation test were to: 

 Evaluate the operational feasibility, pilot workload, and pilot acceptability of conducting a 
11 

straight-in instrument approach procedure with published vertical guidance using EFVS (i.e., 

FLIR imagery on a HUD) for the approach, landing, roll-out and turn-off in weather and 

visibility as low as 1000 ft RVR. 

 Evaluate the operational feasibility, pilot workload, and pilot acceptability of conducting a 

straight-in instrument approach to a 150 ft DA/DH procedure with published vertical 

guidance using SVS (displayed either head-down or head-up) and to transition to natural outthe-window (OTW) visual conditions for landing in weather and visibility as low as 1400 ft 

RVR. 

In addition, NASA-specific test objectives included: 

 Evaluation of time required, accuracy, and pilot workload associated with recognizing and 

reacting to potential ground collisions or conflicts with other aircraft, vehicles and 

obstructions across a range of visibility and lighting conditions using an EFVS and SVS. 

 Evaluation of the effect OTW visual cue alerting by the pilot monitoring (PM) has on the 

performance and visual attention of the pilot-flying (PF) during approach and landing low 

visibility operations. This assessment was conducted by modulating the crew complement 

(i.e., single versus dual pilot operations) using SEVS technologies. This objective enables 

correlation and comparison to a follow-on (single pilot) flight test configuration. These runs 

were not conducted to advocate nor imply the possible acceptance of single pilot operations 

for Part 25 aircraft. 

 Examination of CDTI influences on the time required, accuracy, and pilot workload 

associated with recognizing and reacting to potential ground collisions or conflicts with other 

aircraft, vehicles and obstructions when using EFVS and SVS. 
12 

3 Methodology 

3.1 Subjects 

Twenty-four pilots served as test subjects for the research, representing twelve flight crews. Ten 

crews flew for major U.S. air carriers and were paired by airline to ensure crew coordination and 

cohesion with regard to terminal and surface operational procedures. The other crews were business 

aircraft operators, flying Gulfstream G-V or G450 aircraft equipped with EFVS and SVS. All test 

subjects were male. The Captains’ average age was 55 years and the First Officers’ average age was 

47.5 years. The Captains had an average of over 14,661 flight hours with 21 years of commercial 

flying. The First Officers had an average of over 10,648 flight hours with 14 years of commercial 

experience. The Captains were recruited on the basis of HUD experience (at least 100 hours), with 

preference given to pilots with Enhanced Vision (EV)/EFVS experience. All pilots were required to 

hold an Airline Transport Pilot rating. 

The Captain was the designated pilot-flying (PF) throughout all the trials and the First Officer served 

as the pilot-monitoring (PM). 

3.2 Simulation Facility 

This research was conducted in the Research Flight Deck (RFD) simulator at NASA LaRC (Figure 

1). The RFD is configured with four 10.5-inch Vertical (V) by 13.25-inch Horizontal (H), 1280x1024 

pixel resolution color displays, tiled across the instrument panel. Also, the RFD includes a HUD on 

the left side of the cab, Mode Control Panel, Flight Management System (FMS), and two Electronic 

Flight Bags (EFBs) (Figure 1). 

The full-mission RFD simulates a modern twin-jet transport aircraft. The cab is populated with flight 

instrumentation and pilot controls, including the overhead subsystem panels, to replicate the 

simulated aircraft. A NASA-designed sidestick control system is used in this cab instead of 

conventional wheel and column for manual flight control. 

A collimated OTW scene is produced through five Barco model 7120A projectors shown on a 10 foot 

diameter SEOS panorama mirror cell using a 5-channel EP-1000 image generation system providing 

approximately 200H by 40V field-of-view (FOV) at 26 pixels per degree. 

The sidestick inceptor force gradients and deflection characteristics mimic the Airbus 320 aircraft 

[23]. A parabolic shaping gearing was used between the normalized stick deflection of the sidestick 

and the elevator and aileron commands of the simulated aircraft. The parabolic shaping provided 

acceptable handling characteristics for the approach, landing, and departure tasks. The pilot and copilot inceptors are directly linked as if mechanically connected. 

The auto-throttle system backdrives the throttle handles to directly reflect the power setting 

commanded to the engines. Take-off, go-around (TOGA) buttons and autothrottle disconnect buttons 

were located on the throttle handles. 
13 

Figure 1. Research Flight Deck Simulator with HUD, Head-down Instrument Panel, and EFB 

3.2.1 Simulator Database 

Operations were simulated at Chicago O’Hare International Airport (FAA identifier: ORD). The 

simulation was built around FAA source data for ORD, valid from 11 March 2010 to 8 April 2010. 

These data were used to develop all flight plans, scenarios, approach paths, and OTW, synthetic 

vision (SV) and EV databases. 

Day simulations were flown, with the weather tailored to create the desired visibility conditions. 

Approaches were flown only to runways with Medium intensity Approach Lighting System with 

Runway alignment indicator lights (MALSR) installed. Testing included an experimental variation of 

touchdown zone and centerline (TDZ/CL) lights (on and off), where operations with TDZ/CL lights 

were conducted on ORD Runway 9R; otherwise, ORD Runways 4R, 22L, or 22R were used. All 

runways included high intensity runway lights and serviceable centerline and surface markings. 

Airport lighting was drawn using calligraphics. 

3.2.2 Audio Effects 

The RFD simulator included standard audio effects representative of current day air carriers. Of 

particular importance, altitude call-outs were played over the flight deck speakers. The automatic 

altitude calls-out started at “500 feet” when the aircraft was 500 ft above the TDZE. The 

“approaching minimums” and “minimums” call-outs were generated at 100 ft above and at the 

planned DA/DH for a given run. 

Flare “prompts” in the form of additional altitude call-outs were used on all runs (“100,” “50,” “40,” 

“30,” “20,” and “10” at the corresponding radar altitudes in feet). 

3.2.3 Head-Down Displays 

Figure 1 shows the simulator’s four main instrument panel displays on the HDD Panel: a) PF left 

display, including primary flight display (PFD); b) PF right display including navigation display 

HUD

HDD Panel

EFB
(ND);

conten

3.2.4 H

The R

HUD 

from 

HUD 

confo

varied

mode

runwa

above

refere

Glide

indica

under

HUD 

flashe

; c) PM left d

nt of these dis

Head-Up Disp

RFD is equipp

is collimated

a Cathode Ra

was either a 

ormality with 

d by the pilot.

 format, with

ay outline (ed

e field level (A

enced guidanc

slope and loc

ators were pro

rneath the flig

flare cue, con

ed above the f

display, includ

splays were v

play 

ped with a Ro

d and subtend

ay Tube sourc

SV or EV sou

the outside w

. The stroke s

the compass 

dge lines) whi

AFL), a flight

ce cue (Figure

calizer raw da

ovided (i.e., g

ght path mark

nsisting of a f

flight path ma

Figure

ding ND; and

varied experim

ockwell-Collin

s approximat

ce in a strokeurce imagery 

world, the FOV

symbology fo

rose symbol 

ich was confo

t path marker

e 3). The gui

ata indicators 

glideslope and

er when below

flare “prompt

arker (Figure 

e 2. EFVS HU

14 

, d) PM right 

mentally as de

ns Head-up G

ely 26o H by 

-and-raster fo

in an RS-343

V of the HUD

ormat was a m

removed (Fig

ormally drawn

r, a flight path

dance cue wa

which includ

d localizer dev

w 2500 ft abo

t”, was provid

4). 

UD (left) and SV

display, inclu

escribed in Se

Guidance Syst

21o V FOV. 

ormat (Figure 

3 format [24].

D raster image

modified vers

gure 3). The 

n using a 800

h angle refere

as driven by th

ded a deviation

viation). Rad

ove ground le

ded where, at 

VS HUD (righ

uding PFD. T

ection 3.7.1. 

tem (HGSTM)

The HUD pr

2). The raste

. Note that to

e was fixed an

ion of the HG

stroke symbo

0-ft x 200-ft r

ence cue and a

the airplane fl

n scale and an

dar altitude w

evel (AGL). 

50 ft AFL, tw

ht) 

The format an

)-4000 HUD. 

rojects the ima

er input to the

o maintain 

nd could not b

GS-4000 prim

ology include

runway until 

a flight-path 

light director.

ngular deviati

was shown dig

In addition, a

wo “plus” sig

nd 

 The 

agery 

e

be 

mary 

ed a 

50 ft 

 

ion 

gitally 

a

ns 
16 

Figure 3. HUD Symbology 

Figure 4. HUD Flare Cue 

The PF had independent controls to adjust the stroke symbology brightness and the raster imagery 

brightness and contrast. The pilots were trained on how to set the brightness/contrast of the SVS 

image. They were allowed to adjust it at any time during the test, and the principal investigator 

specifically had them set it at the beginning of the day to their personal preferences. The PF also had 

a declutter control, implemented as a four-button castle switch on the pilot’s sidestick. The four 

“declutter” states available to the PF were: (1) Declutter All (no symbology or imagery); (2) 

Symbology (Stroke) Toggle on/off; (3) Imagery (Raster) Toggle on/off; and (4) Display All (both 

symbology and imagery). 

The HUD was stowed when not being used to avoid any confounding from the HUD being in place 

during “non-HUD” runs. 

Flare Cue 

Runway Outline 

Flight Path Angle 

Reference Cue 

Radar Altitude

Wind Speed & Direction 

Flight Path Marker & 

Guidance Cue 

Localizer Deviation 

Glideslope Deviation 

Vertical Speed 
17 

3.2.5 Eye Tracking System 

A 4-camera Smart Eye remote eye tracking system was installed (Figure 5) and optimized to at a 

minimum track the left seat pilot head position with six degrees of freedom at all times. 

The SmartEyeTM remote eye tracking systems first determine head position in all six degrees of freedom. 

This is done by two dimensional image recognition using several key facial characteristics. Points such 

as the eye corners, nostrils, corners of the mouth, ears, etc. are identified and measured in relative pixel 

distance. Combining the located image points using two cameras of known position allows for 3D image 

processing, producing six degree of freedom head position values. Eye tracking is then measured by 

determining the center of the pupil through contrast image processing, relative to a glint reflection, 

provided by infra-red light sources of known location on the iris that indicates the center of the eye itself. 

By calculating the known distance between these two points, trigonometry is used to calculate a vector 

between the two points. A three dimensional eye gaze vector can be calculated in reference to a world 

coordinate system, such as a flight deck. A minimum of two cameras are required to perform three 

dimensional calculations. 

Eye gaze vector tracking was optimized for the HDD instruments and OTW in the pilots’ forward looking 

field of view. The system was not optimized for accurate lateral tracking beyond the ND or to the left of 

the wing panel on the glare shield. 

Figure 5. RFD Smart Eye Camera and Flasher Locations 
18 

3.3 SV Simulations 

A SV database was developed by NASA starting from the OTW database, generally following the 

standards from RTCA DO-315B [22]. The database used a one arc-second Digital Elevation Model 

(DEM) of a 110.25 nautical miles (nm) (East-West) by 145.6 (North-South) nm area centered around 

ORD. The DEM was draped with an elevation-based coloration texturing. 

Each ORD runway was modeled as an asphalt-colored polygon using the threshold data and runway 

widths. Threshold lines, edge lines, and runway numbers were added. 

The intended landing runway (as selected through the FMS prior to run initiation) was denoted on the 

primary display concept being evaluated, either as a conformal magenta outline on the head-down 

PFD depiction, or an 8000 x 200 ft outline (shown as edge lines) on the HUD. 

Because the test was confined to low altitude approach, landing, and surface operations at ORD, 

obstacles would not create a significant visual cue. Therefore, obstacles were not included or marked 

in the SV depiction. 

The SVS-PFD symbology mirrored the HUD using conformal depictions for the flight path marker, 

single cue flight path-referenced guidance symbology, and flight path angle reference cue. Other 

required primary flight reference information was also drawn (e.g., airspeed, altitude, and raw data 

deviations). 

When drawn on the HUD, the SV database terrain texturing and coloration was slightly changed to 

improve its visual perception primarily by specific coloration for conversion into a gray-scale format. 

The SV depiction was always drawn in a heading-up format. Any crosswind was evident by 

conformal lateral positioning of the flight path marker. However, the flight path marker and guidance 

cue were limited and displayed as ghosted representations if their conformal positions exceeded predetermined values. 

3.4 EV Simulation 

The EV real-time simulation is created by the Evans and Sutherland EPXTM physics-based sensor 

simulation. EPX provides rendering of airports, complex terrain, advanced weather, and other highresolution three-dimensional effects for flight simulation. The ORD database was instantiated with 

material code properties. From this database, an IR sensor simulation, interacting with this materialcoded database and the simulated weather conditions, created the desired test experimental 

conditions. 

The EV simulation mimicked the performance of a short-wave/mid-wave FLIR, using an 

approximately 1.0 to 5.0 micron wavelength detector. The nominal enhanced visibility was 

approximately 2400 ft for this experiment. 

The eye point reference for the EV simulation was placed five ft below the pilot design eye reference 

point, but otherwise properly boresighted (i.e., angular alignment) to the aircraft. In the simulated 

airplane, the pilot is approximately 20 ft above the ground during surface operations. This EV eye 

point reference/parallax error generates 2.5 milliradian error to a point located 2000 ft away - 

approximately half of the accuracy budget of the EFVS per current RTCA DO-315 accuracy 

requirements [16]. 
19 

3.5 Navigational Performance Variations 

Variations in navigational accuracy were simulated on each run, bounding 12 ft vertical and 12 ft 

horizontal deviations from the true position. These values were determined by using measured 

performance data found in the Global Positioning System Wide Area Augmentation System (WAAS) 

performance standard document [25]. This effect was added for realism in positioning system 

accuracy. The selected inaccuracies were randomly varied across each subject’s test matrix and were 

held constant during a run. 

3.6 Automatic Dependent Surveillance - Broadcast 

Expected Automatic Dependent Surveillance-Broadcast (ADS-B) inaccuracies were simulated [26]. 

Airborne traffic position and velocity data included Gaussian position and velocity errors about their 

true values representative of RTCA DO-289, Navigation Accuracy Category for Position (NACp) = 9 

(i.e., 95% accuracy bound on horizontal position of 30 m) and Navigation Accuracy Category for 

Velocity (NACv) = 2 values (i.e., 95% accuracy bound on horizontal velocity of 3 m/sec). Surface 

traffic (i.e., aircraft with altitudes less than 100 ft height above threshold, or HAT) included Gaussian 

position and velocity errors about their true values representative of RTCA DO-289, NACp = 11 

values (i.e., 95% accuracy bound on horizontal position of 3 m) and NACv = 4 values (i.e., 95% 

accuracy bound on horizontal velocity of 0.3 m/sec). Traffic data were updated at a one hertz rate to 

emulate ADS-B transmission rates. Between updates, the traffic position data were estimated by 

first-order inter-sample projection of the one hertz data. An ADS-B latency of 0.6 seconds was also 

emulated. 

3.7 Crew Display Concepts 

Two head-down flight display concepts and three head-up flight display concepts were evaluated by 

the crews while flying approaches, landings, and surface operations to Runways 4R, 9R, 22L or 22R 

at ORD. 

3.7.1 Head-Down Flight Display Concepts 

The two HDD concepts (referred to as the Conventional HDD and SVS HDD) are shown in Figures 6 

and 7, differing from each other only in the absence or presence of SVS on the PFD. The HUD was 

stowed during HDD evaluations. 

The SVS on the PFD portrayed a 33o

 V x 44o H field-of-regard. Assuming a 25-inch distance from 

the Design Eye Reference Point to the display, the SVS concept had a minification factor of 

approximately 2.1 for the PF. The PF left display also had a datalink message area and Horizontal 

Situation Indicator (HSI). The PM right display showed a quad-view of flight information: a PFD 

(upper left); HSI (lower left); FLIR repeater or blank area (upper right); and datalink message area 

(lower right). The upper right area of the quad-display was blank for conventional and SVS HDD 

evaluations. 

The PF (right display) and PM (left display) NDs always showed flight traffic and navigational 

information in the airborne mode (see Airborne Modes on Figure 6). The PF and PM NDs 

transitioned to a moving map mode when on the ground and groundspeed less than 100 knots (see 

Surface Modes on Figure 7). The PM ND included a runway inset view in both airborne and moving 

map modes. 
The su

airpor

exper

Runw

variat

identi

collisi

Figure 6

urface movin

rt and intende

rimentally eva

way Inset, see 

tions (e.g., the

ification, and 

ions or confli

6. Airborne He

ng map and th

ed landing run

aluated. The 

Figure 8) we

e presence an

pilot workloa

icts with other

ead-Down Disp

he runway ins

nway, but traf

CDTI configu

ere balanced a

d absence of 

ad associated 

r aircraft. 

play (HDD) Co

20 

et views alwa

ffic icons (i.e.

urations (non

across the exp

CDTI) were c

with recogni

oncepts: Conve

ays showed th

. CDTI) were

ne, on Moving

periment matr

conducted to 

izing and reac

entional HDD 

heir respectiv

e only drawn w

g Map, on Mo

rix. These ex

test the time 

cting to poten

on top and SV

e images of th

when being 

oving Map an

xperimental 

required, acc

ntial ground 

VS HDD on bot

he 

nd 

curate 

ttom
Figure 7

Fig

3.7.2 H

The th

differ

The C

(head

runs a

view)

3.8 Sim

Engin

correl

7. Surface Hea

gure 8. CDTI 

Head-Up Fligh

hree HUD co

ring from each

Conventional 

d-down) right 

and was blank

) shows the EF

mulator Rec

neering unit d

lation and sub

ad-Down Disp

formats – Non

ht Display Co

ncepts (referr

h other only i

HDD (i.e., no

display show

k (black in co

FVS display 

cording

data and audio

bsequent post

lay (HDD) Con

ne (left), Movin

oncepts 

red to as the C

n the absence

o SVS) conce

wed FLIR ima

lor) for Conv

concept with 

o and video da

t-test analysis

21 

ncepts: Conven

ng Map (center

Conventional

e or presence 

ept was displa

agery in the up

ventional and 

symbology a

ata were recor

. 

ntional HDD o

r), and Moving

HUD, SVS H

of imagery (S

ayed during th

pper right cor

SVS HUD ru

and FLIR ima

rded. All dat

on top and SVS

g Map/Runway

HUD and EFV

SV or FLIR) 

hese HUD run

rner of the qu

uns. Figure 2

agery. 

ta were time-s

S HDD on bott

y Inset (right) 

VS) were test

on the PF HU

ns. The PM 

uad-view for E

2 (in the left-s

stamped for 

tom

ted, 

UD. 

EFVS 

ide 
22 

In addition, a SmartEyeTM head and eye-tracking system was installed and used to measure, as a 

minimum, the pilot’s head-position to infer where s/he was looking. Pilot visual behavior was 

quantitatively calculated based upon three-dimensional gaze vector and head position analysis to 

evaluate variation in pilot behavior. 

Video recordings were made of a combination of the following: 

 Pilot and Co-Pilot’s PFD and ND 

 Cockpit area camera; OTW image 

 Smart EyeTM Eyetracking Video Data 

 HUD Camera 

3.9 Evaluation Task 

The evaluation task was a straight-in Instrument Landing System (ILS) approach with a 3-degree 

descent angle to one of four ORD runways (4R, 9R, 22R, or 22L). The approach started three nm 

from the runway threshold for each run. The weather consisted of low to moderate winds with either 

10 knot headwind, 10 knot tailwind, 7.5 knot crosswind, or 15 knot crosswind, light turbulence (rootmean-square (RMS) of 1 ft/sec), and varying OTW visibility levels (1800 ft, 1400 ft, or 1000 ft 

RVR). When used experimentally, the enhanced flight visibility (i.e., visibility provided by the FLIR 

sensor) was 2400 ft RVR. The PF hand-flew the approach from the left seat with the auto-throttle set 

to “speed-hold” at the approach speed of 130 knots indicated airspeed. The auto-throttle 

automatically reduced to idle thrust at 35 ft AGL for landing. The run was terminated once the PF 

completed the landing, roll-out and turnoff or upon go-around initiation. The aircraft was configured 

to land prior to each run (landing gear down and flaps 30 degrees). 

The PFs were instructed to fly the aircraft as if there were passengers aboard, track the approach path, 

and land within the touchdown zone with an acceptable sink rate. After landing, they were to capture 

the centerline and exit at the expected taxiway at a speed of 5 to 15 knots at the 90 degree exits or 30 

knots at the high-speed exits. They were also instructed to initiate a go-around if the landing was not 

safe or there were any safety concerns during the approach. 

3.10 Crew Procedures 

The crew was trained in monitored approach crew procedures. The PF flew the approach using the 

HUD or HDD as the primary flight reference. The PM monitored using the available HDD 

information, including a FLIR repeater (when EFVS was flown), and the OTW scene and assisted the 

PF as appropriate and necessary. There was no transfer of control from the PF to PM (or vice versa). 

To facilitate comparison between baseline conditions (i.e., no SVS or EFVS installed) and 

SVS/EFVS configurations, crew procedures were standardized and trained. 

3.10.1 Baseline and SVS Procedures 

The procedures for the baseline (i.e., no SVS or EFVS installed) and SVS configurations were 

identical and followed normal crew instrument approach procedures (see Table 1). One of the 

intended functions of the SVS is to improve the pilot’s ability to conduct the instrument portion of the 
23 

approach – not to enable descent below the published DA/DH (decision altitude/decision height). 

Therefore, no change in crew procedures would be expected. 

With SVS equipage, the crew (especially, the PM) must monitor the validity of the SVS to continue 

the approach. Prior to reaching the published DA/DH, the crew must verify that the SVS image, 

including the guidance, flight path marker, and runway are functioning properly, without unusual 

behavior statically or dynamically. 

The ability to descend below the DA/DH and to descend below 100 ft HAT was dependent upon the 

pilot being able to recognize and identify, using natural vision, the required visual references, shown 

in the left hand column of Table 2. 

Table 1. Baseline and SVS Crew Procedures 

Altitude‐

Based Events

Radio/Baro

Altitudes (ft)

Automatic

Callouts

PF  

Tasks/Callouts

PM

Tasks/Callouts

500 ft   500 / 1196 “500”   Response:   

“Check”  

Minimums  

+ 100 ft  

DA/DH+100 “Approaching

Minimums”  

Response:  

“Roger”  

Published

Minimums  

DA/DH “Minimums”   With Approach Lights,

Call “Approach Lights, Down to 100”  

With “Out the Window” Cues,

Call “Lights” or “Field in Sight”  

Without Approach or Landing

References,  

Call “Going Around”

Without PF Call of ‘Approach

Lights’,  

Call “Go Around”  

100 ft HAT   100 / 796 “100”   With “Out the Window” Landing

References,  

Call “Field in Sight, Landing”

With “Out the Window” Cues,

Call “Lights” or “Field in Sight”

Without “Out the Window” Landing

References,  

Call “Going Around”

Without PF Call of ‘Landing’,  

Call “Go Around”  
24 

Table 2. Required Visual References 

Required Visual References Using

Natural Vision

(14 CFR § 91.175 (c))

Required Visual References Using an

Enhanced Flight Vision System

(14 CFR § 91.175 (l))

For operation below DA/DH or MDA –

At least one of the following visual references for the intended

runway must be distinctly visible and identifiable:

Approach light system

Threshold

Threshold markings

Threshold lights

Runway end identifier lights

Visual approach slope indicator

Touchdown zone

Touchdown zone markings

Touchdown zone lights

Runway

Runway markings

Runway lights

For operation below DA/DH or MDA –

The following visual references for the intended runway

must be distinctly visible and identifiable:

Approach light system  

OR

Visual references in BOTH paragraphs

91.175(l)(3)(ii)(A) and (B) ‐‐  

(l)(3)(ii)(A) The runway threshold, identified by at

least one of the following –

  ‐‐ beginning of the runway landing surface,

  ‐‐ threshold lights, or

  ‐‐ runway end identifier lights  

AND  

(l)(3)(ii)(B) The touchdown zone, identified by at

least one of the following –

        ‐‐ runway touchdown zone landing surface,

        ‐‐ touchdown zone lights,

        ‐‐ touchdown zone markings, or

        ‐‐ runway lights.  

   

Descent below 100 feet height above TDZE –

At least one of the following visual references for the intended

runway must be distinctly visible and identifiable:

Approach light system, as long as the red terminating bars or

red side row bars are also distinctly visible and identifiable

Threshold

Threshold markings

Threshold lights

Runway end identifier lights

Visual approach slope indicator

Touchdown zone

Touchdown zone markings

Touchdown zone lights

Runway

Runway markings

Runway lights

Descent below 100 feet height above TDZE –

The following visual references for the intended runway

must be distinctly visible and identifiable:

The lights or markings of the threshold

          OR

The lights or markings of the touchdown zone

3.10.2 EFVS Procedures 

The EFVS procedures used for this study (see Table 3) were built around common practice in current 

EFVS operations and FAA requirements (14 CFR § 91.175 (l)) but extended to emphasize that to 

descend below the DA/DH and to descend below 100 ft height above the TDZE depended upon the 

PF being able to recognize and identify the required visual references using EFVS, as shown in the 

right hand column of Table 2. For this test, the crews were briefed that the enhanced flight visibility 

was not less than the visibility prescribed by the instrument approach procedure being used. 
25 

Table 3. EFVS Crew Procedures 

Altitude‐Based

Events

Radar/Baro  

Altitudes (ft)

Automatic

Callouts

PF  

Tasks/Callouts

PM  

Tasks/Callouts

500 ft   500 / 1196   “500”   Response:   

“Systems Normal, EVS Normal”  

Minimums  

+ 100 ft  

DA/DH+100 “Approaching

Minimums”  

Response:  

“Roger”  

Published

Minimums  

DA/DH “Minimums” With EFVS Approach Lights,

Call “EVS Lights”  

With “Out the Window” Cues,

Call “Lights” or “Field in Sight”  

Without EFVS Approach Lights,  

Call “Going Around”  

Without PF Call of ‘EVS Lights’,

Call “Go Around”  

EFVS Decision

Altitude  

(100 ft AFL)

100 / 796 “100”

With EFVS Landing Visual References,

Call “Landing”  

With “Out the Window” Cues,

Call “Lights” or “Field in Sight”  

Without EFVS Landing Visual

References,  

Call “Going Around”  

Without PF Call of ‘Landing’,  

Call “Go Around”  

3.11 Experiment Matrix 

Both single pilot and two-crew evaluations were conducted, but the primary emphasis was placed on 

the two-crew evaluations. 

The primary two-crew experiment test matrix aligned three visibility conditions against five SEVS 

configurations (combinations of displays and vision systems) as shown in Table 4. The visibility 

conditions and SEVS configurations evaluated are indicated by an ‘x’ in Table 4. Each of these ‘x’-

marked conditions was flown twice, once with TDZ/CL lights and once without TDZ/CL lights. 

Two baseline conditions are identified in Table 4: one using a HUD and one using HDDs; both 

without any SVS or EFVS information. These baseline conditions were defined by the FAA for this 

experiment to represent operational concept baselines with which to compare the four SVS 

operational concepts (HUD in 1000 and 1400 ft RVR and HDD in 1400 and 1800 ft RVR) and two 

EFVS operational concepts (1000 and 1400 ft RVR) tested. 

The baseline comparison for SVS operational concepts is the conventional HUD flown in 1400 ft 

RVR without TDZ/CL lights. This configuration mimics the current Special Authorization Category 

I approach specified in FAA Order 8400.13D. This authorization does not require the use of 

centerline or touchdown zone lights. This configuration will be used for comparative analysis in two 

primary performance aspects: 1) the FTE to the DH and at the DH; and, 2) the pilot’s ability to 

transition from instrument flight to visual flight, identify the required visual references, and acquire 

the runway environment. In order for a HDD SVS configuration to be considered for the same 

operational approval as the HUD on a Special Authorization Category I approach, the FTE, pilot 

performance, and touchdown and rollout performance using the HDD SVS configuration should be 

no worse than the performance achieved from an authorized Special Authorization Category I 

operations. The pilot’s ability to transition to head-up visual flight must be smooth and seamless. 

The baseline comparison for EFVS operational concepts is the conventional HDD with 1800 ft RVR 
26 

with TDZ/CL lights. This configuration mimics the lowest visibility currently authorized for certain 

Category I approaches without using a HUD (see FAA Order 8400.13D). This configuration will be 

used for comparative analysis in two primary performance aspects: 1) the ability of the pilot to fly 

below the DA/DH to the runway; and, 2) the pilot’s ability to safely land in the touchdown zone, with 

an acceptable sink rate, and to maintain tracking and alignment with the runway centerline upon 

landing roll-out. For the baseline condition, below the DA/DH, the pilot’s primary reference for 

maneuvering the aircraft will be the OTW visual cues (i.e., natural vision). In the case of an EFVS, 

the pilot’s primary reference for maneuvering the aircraft will be the enhanced flight vision cues. 

For a HUD EFVS configuration to be considered for operational approval to use EFVS in lieu of 

natural vision to descend below the DA/DH and land and roll-out, equivalent performance to an 

approach where natural vision is relied on below the DA/DH must be demonstrated. 

Table 4. Experiment Matrix for Two-Crew Operations 

Visibility

Display

Vision

System 1800 ft RVR 1400 ft RVR 1000 ft RVR

HUD None x

SVS x x 

EFVS x x 

HDD None x* x 

SVS x x 

* 200 ft DA; All others 150 ft DH

EFVS and SVS non-normal runs were injected into the test unbeknownst to the PF and PM to assess 

the crew’s decision-making process when confronted with these non-normal events while flying in 

low-visibility conditions. The crews did not receive any briefing or training on these non-normal 

runs. The non-normal runs included runway incursion scenarios, SVS lateral and vertical navigation 

system error scenarios, EFVS HUD failure scenarios, and for EFVS, insufficient enhanced flight 

visibility to land scenarios. 

To accomplish the runway incursion testing, the experiment was conducted in two testing phases (a 

“non-expectancy” phase and an “expectancy” phase) which were separated by the staging of a 

runway incursion, serving as a rare event collision scenario. In the first phase, experimental 

variations in EFVS and SV equipage and varying weather/visibility conditions were conducted (see 

Table 4), and these runs were considered normal runs. Randomly placed within the normal run 

matrix of the first phase were five runs with non-normal events: two runs with SVS navigation system 

error (one lateral, 1one vertical), two runs with EFVS HUD failure (one in 700 ft RVR and one in 

1000 ft RVR), and one run with insufficient enhanced flight visibility to land. 

The two SVS non-normal events involved unannunciated navigation system inaccuracies while flying 

the SVS HDD. These deviations were: a) lateral deviation of +/-131 ft (left/right) or b) vertical 

deviation of +/-115 ft (high/low). The lateral and vertical inaccuracies chosen are the WAAS 

horizontal and vertical alert limits, respectively. The lateral navigation system error was always 

flown to runway 22R without TDZ/CL lights in a 10 knot headwind and 1400 ft RVR OTW test 

condition. The vertical navigation system error run was always flown to runway 22L without 

TDZ/CL lights in a 10 knot headwind and 1400 ft RVR OTW test condition. These configurations 

were used to assess the flight crew’s reaction to a situation where the navigation positioning system 

provides an unannunciated large error. 
27 

The two EFVS non-normal events gauged the crew’s reaction and consequences of a failure of the 

EFVS at 50 ft AFL. The nominal EFVS (FLIR) visibility was 2400 ft. The OTW weather and 

visibility were either 1000 ft RVR or 700 ft RVR when a failure of the HUD caused the loss of all

HUD information (i.e., loss of HUD symbology and EFVS) at 50 ft AFL. These two non-normal 

runs were flown without TDZ/CL lights. The 1000 ft RVR EFVS failure runs were flown in a 7.5 

knot left crosswind and the 700 ft RVR EFVS failure runs were flown in a 7.5 knot right crosswind. 

The criticality of this failure is that the pilot loses both the enhanced vision view of which s/he is 

reliant and any guidance information, causing the pilot to rely solely on the available OTW visual 

cues to complete the landing and roll-out or go-around. The crews were not briefed or trained on this 

failure event. 

An additional EFVS run with the enhanced flight visibility set to 1000 ft (instead of the nominal 2400 

ft) was flown during the first phase of testing. For this HUD EFVS run, the OTW visibility was set to 

1000 ft RVR and flown to a runway without TDZ/CL lights. This run was added to evaluate the 

tendency of the PF to continue an approach to landing even though s/he did not necessarily have the 

enhanced flight visibility sufficient to conduct the operation. The crews were not briefed that the 

enhanced flight visibility had been reduced to 1000 ft. 

The last run of the first phase was a “rare event” runway incursion scenario which was flown to test 

pilot/crew recognition and reaction in a non-normal situation without expectancy of the flight crew. 

The “unexpected” runway incursion was flown using one of four display configurations: 1) EFVS 

without CDTI, 2) EFVS with CDTI on moving map and runway inset, 3) SVS HDD without CDTI, 

or 4) SVS HDD with CDTI on moving map and runway inset. So, there were 12 total unexpected 

runway incursion runs for the simulation experiment, with three samples in each of the four display 

configurations tested. This run was always flown to runway 22R without TDZ/CL light in a 10 knot 

headwind and 1400 ft RVR OTW test condition. Because of the severity of the rare event runway 

incursion, it was anticipated that the subsequent behavior of the crew (pilots) would be altered and 

more attuned to potential traffic incursion events. Hence, this run was always the final one in the first 

phase testing. 

The second phase followed the rare event runway incursion scenarios. In this second phase, repeated 

incursions/object detection scenarios were flown and tested using EFVS and one of three CDTI 

combinations (none, Moving Map, Moving Map and Runway Inset). Each crew was exposed to nine 

EFVS CDTI evaluations of “expected” traffic incursion on either a runway or taxiway: three 

incursion events occurred on runways without TDZ/CL lights (4R, 22L, and 22R) and six taxiway 

incursion events. It was assumed that these runs would not be without expectancy on the part of the 

pilots. The purpose of the second phase runs was to test the effects of CDTI on the time required, 

accuracy of identification, and pilot workload associated with potential ground collisions or conflicts 

with other aircraft. 

The single pilot conditions (see Table 5) focused primarily on collecting data for correlation or 

comparison with planned follow-on flight test configurations and for comparison of the influence of 

two-crew operations. As such, the comparisons between the two-crew and single pilot operations 

contained within this test were valid and indicative of the influence of crewed operations. These 

results will also provide data for comparison to the planned (single pilot) flight test. These runs were 

not conducted to advocate nor imply the possible acceptance of single pilot operations for Part 25 

aircraft. In addition, the general applicability of the single pilot results with respect to operations and 

equipage may not be representative of Part 23-type aircraft. 

Only the subject trained as PF flew the single pilot evaluations. The six single pilot runs (see Table 
28 

5) were blocked together and were conducted within the first phase of experimental testing. TDZ/CL 

lights were present for each of these six runs. 

Table 5. Experiment Matrix for Single Pilot Operations 

Operational Concept

CDTI

None Moving Map

Only

Moving Map

and Runway

Inset

EFVS HUD in 1000 ft RVR x x x

SVS HDD in 1400 ft RVR x x x 

Wind variations were balanced across the experiment matrix for each crew/pilot to evenly distribute 

the conditions across the configurations. Thus, wind effects were tested but not in a within-subjects 

design. It was assumed that left and right crosswinds could be interchanged without affecting any 

experimental results. 

3.12 Measures 

During each approach and landing run, path error, pilot control inputs, and touchdown performance 

(sink rate and speed at touchdown, distance fore or aft of touchdown zone, and distance left or right 

of centerline) were measured for analysis. During taxi operations, centerline tracking was measured. 

After each run, pilots completed the Air Force Flight Test Center (AFFTC) Workload Estimate Scale 

[27]. After data collection was completed, pilots were administered two paired comparison tests: the 

Situation Awareness – Subjective Workload Dominance (SA-SWORD) [28] technique and one on 

Traffic Awareness evaluating CDTI formats tested. These subjective measures are provided in 

Appendix A. 

At six times during the two-crew testing, a traffic awareness probe, modeled after a Situation 

Awareness Global Assessment Test (SAGAT) [29] was administered. The data were used to quantify 

the flight crew’s awareness (PF and PM) of traffic and the influence of CDTI. 

On these runs, twelve aircraft were located in close proximity to and on the intersecting and adjacent 

taxiways of the active runway, along the entire length. Immediately after clearing the runway, the 

displays and OTW scene were blanked. The pilots were given a paper diagram of the active runway 

(see Appendix A, Section 8.3), including the intersecting and adjacent runways and were asked to 

recall the location of all traffic in proximity to the runway they just used. They identified this by 

circling the location of the aircraft (i.e., traffic) that they recalled on the paper chart. The pilots were 

asked to complete this probe without consulting or discussing with each other. 

The probe was administered for the following runs: 

1. EFVS in 1400 ft RVR flown to a runway without TDZ/CL lights 

2. EFVS in 1400 ft RVR flown to a runway with TDZ/CL lights 

3. Conventional PFD in 1800 ft RVR flown to a runway with TDZ/CL lights (i.e., Operational 

Baseline for EFVS concept comparisons) 
29 

4. SVS PFD in 1400 ft RVR flown to a runway without TDZ/CL lights 

5. SVS PFD in 1400 ft RVR flown to a runway with TDZ/CL lights 

6. Conventional HUD in 1400 ft RVR flown to a runway without TDZ/CL lights (i.e., 

Operational Baseline for SVS concept comparisons) 

CDTI (None, On Moving Map, or On Moving Map and Runway Inset) were balanced across the test 

matrix and order of occurrence for the traffic probe. 

3.13 Test Conduct 

The subjects were given a one-hour briefing describing the experiment, HUD and HDD concepts, 

crew procedures, and evaluation tasks. The test purpose was described to the test subjects as 

“evaluating the potential use of EFVS and SVS for reduced landing weather minima and the influence 

of CDTI for NextGen operations.” 

After the briefing, a 1.5 hour training session in the RFD was conducted to familiarize the subjects 

with the aircraft handling qualities, display symbologies, pilot procedures, and controls. In particular, 

in-simulator training highlighted the crew procedures for EFVS and SVS operations and landing 

performance. Landing performance was planned as one of the performance parameters used to assess 

the efficacy of the SEVS experimental variations. However, none of the pilots were familiar with the 

handling characteristics of the RFD simulator. To accommodate this disparity, each PF was trained to 

an acceptable standard of approach and landing performance. 

In Table 6, touchdown performance criteria are shown. After each training run, a landing 

performance assessment was displayed for feedback (Figure 9). The value and rating for the 

touchdown performance assessment were color-coded. Touchdown performance parameters were 

depicted in green text if they met the “Desired” criteria, yellow text if they met the “Adequate” 

criteria, and red text if the met the “Not Adequate” criteria listed in Table 6. The pilots were asked to 

meet the desired performance criteria listed in Table 6. Training concluded once the pilots 

demonstrated repeatable desired landing performance, with only an occasional adequate performance 

score. If the adequate performance criteria were met, they landed within the touchdown zone with 

acceptable sink rates. 

The training was flown in varying OTW visibility from visual conditions down to 1000 ft RVR. 

Similarly, enhanced flight visibility (i.e., visibility provided by the FLIR sensor) ranged from 

unlimited down to 1000 ft. The training emphasized that they must always remain safe and if they 

felt unsafe conditions exist, the necessary precautions, including a go-around, should be executed 

immediately. 
30 

Table 6. Touchdown Performance Criteria 

Performance Value Desired Adequate Not Adequate

Lateral Distance from

Centerline

Within +/‐ 27 ft Between +27 and +58 ft or

Between ‐27 and ‐58 ft

> +/‐58 ft

Longitudinal Distance from

Threshold  

Between 750 to 2250 ft Between 200 & 750 ft or

Between 2250 & 2700 ft

< 200 or >2700 ft

Sink rate Between 0 to 6 ft/sec Between 6 to 10 ft/sec >10 ft/sec

Airspeed (knots) Between Vref‐5 to Vref+5 Between Vref‐5 to Vref‐15 < Vref‐15 or > Vref+5  

Vref + 5 is the approach speed in table above

Figure 9. Landing performance assessment 

4 Results 

Flight performance (approach, landing, and rollout), pilot workload, and eye-tracking (where 

applicable) data were analyzed for two operational domains: EFVS Operations and SVS Operations. 

In the first domain (referred to as EFVS Operational Concept Comparisons in Section 4.2), the 

operational feasibility, pilot workload, and pilot acceptability of conducting a straight-in instrument 

approach procedure with published vertical guidance using EFVS (i.e., FLIR imagery on a HUD) for 

the approach, landing, roll-out and turn-off in weather and visibility as low as 1000 ft RVR were 

evaluated. The baseline comparison for EFVS operational concepts is the Conventional HDD flown 

in 1800 ft RVR with TDZ/CL lights. This baseline condition creates the direct comparison of “visual 

segment” performance (from DA/DH to touchdown and roll-out) in the lowest visibility (1800 ft 

RVR) allowable using natural vision under today’s regulations against an EFVS “visual segment.” 

Similarly, in the second domain (referred to as SVS Operational Concept Comparisons in Section 

4.3), the operational feasibility, pilot workload, and pilot acceptability of conducting a straight-in 

instrument approach to a 150 ft DA/DH procedure with published vertical guidance using SVS 
31 

(displayed either head-down or head-up) and to transition to natural OTW visual conditions for 

landing in weather and visibility as low as 1400 ft RVR were evaluated. The baseline comparison for 

SVS operational concepts is the Conventional HUD flown in 1400 ft RVR without TDZ/CL lights. 

This baseline condition creates the direct comparison of “instrument segment and visual transition” 

performance (descent to the DA/DH with transition to the visual segment) in the lowest visibility 

(1400 ft RVR) allowable using natural vision under today’s regulations (i.e., Special Authorization 

Cat. I, under FAA Order 8400.13D) against an SVS “instrument segment and visual transition.” 

EFVS and SVS non-normal runs were injected into the test unbeknownst to the PF and PM. These 

situations stressed the crew’s decision-making process when confronted with non-normal events and 

are analyzed separately and discussed in the next section. 

Single pilot evaluations were also conducted for the EFVS and the SVS PFD with the three variations 

of CDTI (none, Moving Map, Moving Map and Runway Inset). Statistical analyses were conducted 

on approach and touchdown performance with crew complement (single, dual) and CDTI as the main 

factors. 

4.1 Metrics 

4.1.1 Flight Performance 

Flight performance was evaluated using different metrics and measures to explore specific 

assessments of interest. 

Approach performance during the “instrument” segment was analyzed using RMS localizer deviation 

(in dots), RMS glideslope deviation (in dots), and RMS sink rate deviation (in feet per minute, or 

fpm) where this value is difference or deviation from the sink rate required to perfectly track the 

glideslope in the given wind conditions. These parameters correspond intuitively to how well a 

stabilized approach to landing – an important safety measure – was established and maintained. The 

approach data were analyzed from 1000 ft to DA/DH for the normal runs. The beginning altitude 

value of 1000 ft was the start of each run. 

Approach performance was also analyzed using existing FAA [30, 31] and Joint Aviation Regulations 

(JAR) All Weather Operations (AWO) [32] performance-based approach standards for glideslope and 

localizer tracking. These standards were drawn from numerous sources pertaining to the general 

concept of low-visibility approach and landings. However, none of these existing standards were 

written specifically as quantitative performance standards for advanced vision systems (such as SVS 

and EFVS) operations but are applied herein for comparative purposes. 

A synopsis of these existing quantitative performance requirements are shown in Table 7. In the 

Practical Test Standard [30], glideslope and localizer performance requirements are expressed in 

microamps, with 150 microamps equal to full scale deflection on the ILS. In Table 7, glideslope and 

localizer deviations are expressed in dots deflection by assuming +/- 2 dots full scale deflection 

corresponds to +/-150 microamps deviation from on-course. Note that +/- 2 dots full scale deflection 

of the glideslope deviation indicator is 1.4 degrees (+/- 0.7 degrees from center of glideslope beam) 

and that +/- 2 dots full scale deflection of the localizer deviation scale is 5.0 degrees (+/- 2.5 degrees 

from runway centerline) [33]. 

This synopsis emphasizes “performance” parameters of interest that are relevant to this experiment 

and does not include many important regulation facets and nuances for the sake of brevity. Of 
32 

particular note, airspeed, sink rate, and bank angle control are typically evaluated but are not reported 

herein. 

For instance, AC120-29A [31] documents an acceptable means for obtaining approval of operations 

in Category I/II landing weather minima (i.e., a DH lower than 200 ft but not lower than 100 ft and a 

runway visual range not less than 1200 ft). Part of the operation includes a “visual segment using 

natural vision;” that is, below the DH, the primary reference for maneuvering the airplane is based on 

what the pilot sees visually. AC120-28D [34] describes an acceptable means for obtaining approval 

of operations in Category III landing weather minima (i.e., DHs below 100 ft, or no DH and a runway 

visual range not less than 700 ft). Unlike AC120-29A, AC120-28D does not include a natural visual 

segment. Consequently, with a goal of operating in Category III landing weather minima by use of 

SVS and EFVS technology, a conundrum is created. Further operating credit for vision systems (VS) 

technology pushes into the AC120-28D weather and visibility arena but AC120-28D does not give 

consideration for a visual segment for the pilot. The analyses that follow examine the applicability of 

existing FAA and JAR approach standards for VS operations in all-weather operations. JAR AWO231 documents an acceptable means for obtaining approval of operations for approaches with 

decision heights below 200 ft and down to 100 ft. Like AC120-29A, the JAR standard relies on the 

pilot’s natural vision for maneuvering the airplane below the DH. 

Table 7. Quantitative approach performance standards. 

Localizer Tracking Glideslope Tracking

Practical Test Standard  

Reference 30  

<3/4 Full Scale Deflection (i.e., 1.5

dots), Between Final Approach Fix and

Decision Height  

<3/4 Full Scale Deflection, (i.e., 1.5 dots)

Between Final Approach Fix and Decision

Height  

AC 120‐29,  

Appendix 2, Paragraph

6.2.1.

Reference 31

< 1/3 (i.e., 2/3 dots) Full Scale

Deflection from 1000 ft Height Above

Touchdown (HAT) to 200 ft HAT  

< 1/2 Full Scale (i.e., 1 dot) Deflection from

700 ft HAT to 200 ft HAT  

“Cat 2, Successful

Approach”

FAR Part 91, Appendix A,

Section 3, Subsection e2

At 100 ft DH, cockpit is within and

tracking so as to remain within, the

lateral confines of the runway

extended.  

Deviation from glideslope after leaving the

outer marker does not exceed 50% Full

Scale Deflection (i.e., 1 dot down to 100 ft

DH)   

Joint Aviation Regulations

–All Weather Operations

AMC AWO 231

Reference 32

No more than 5% of approaches with

>1/3 dot between 300 ft and 100 ft

HAT  

No more than 5% of approaches with >1

dot between 300 ft and 100 ft HAT  

Flight path performance data (lateral position, vertical position, and sink rate) were used to evaluate 

how effectively the pilots could use the different EFVS and SVS concepts during the visual segment 

of the approach to position the aircraft for landing. The absolute values for maximum bank angle 

from threshold crossing to touchdown and the bank angle at touchdown were also used to evaluate the 

effectiveness of the different EFVS and SVS concepts. 

Touchdown statistics were used to evaluate how effectively the pilots could land with the different 

SEVS display concepts and how well the crews met the touchdown performance criteria to which 

they had been trained (see Table 6). Existing FAA AC120-28D [34, Appendix 3] and JAR AWO 
33 

[32] performance-based “auto-land” standards for touchdown (T/D) longitudinal position, lateral 

position from centerline, and sink rate were applied in the objective landing data analysis. 

Specifically, the standards require no longitudinal touchdown earlier than a point on the runway 200 

ft from the threshold or beyond 2700 ft from the threshold, no lateral touchdown with the outboard 

landing gear more than 70 ft from the runway centerline, and no touchdown sink rate greater than -10 

feet/second to a probability of 1 x 10-6. These standards pertain to the general concept of lowvisibility approach and landings using guidance systems technologies, but were not written 

specifically for operations with advanced vision systems such as EFVS and SVS. This experiment 

used an aim point located 1000 ft from runway threshold. For the simulated aircraft, the outboard 

landing gear would be 70 ft from the centerline when the fuselage is at 58 ft lateral deviation from 

centerline, assuming no crab angle at touchdown. 

Lateral deviation from centerline statistics (maximum value and RMS) were used to evaluate how 

effectively the pilots could maintain centerline during rollout with the different EFVS and SVS 

display concepts. 

4.1.2 Pilot Workload 

Workload was assessed after each experimental run, independently for the PF and PM, using the 

AFFTC Workload Estimate Technique [27]. Workload ratings were evaluated by conducting 

separate Analysis of Variance (ANOVA) tests for the EFVS operational concepts and for the SVS 

operational concepts with operational concept as the main factor. If a significant F-value was 

obtained in an ANOVA, then Student-Newman-Keuls (SNK) post-hoc tests (α set at 0.05) were 

performed. SNK post hoc tests provide specific information on which means are different from each 

other when a significant F-test result is found on a main factor consisting of three or more levels. 

4.1.3 Eye Gaze 

Several metrics were applied to the eye gaze tracking data to evaluate the PF visual behavior, 

including; head-up percentage and transition count between OTW and head-down. For HDD 

concepts, the height above threshold and pilot gaze direction at the point of visual transition OTW 

was also determined to evaluate variance in pilots’ behavior when attempting to acquire the runway 

environment. ANOVAs were conducted to determine significant variance in visual behavior across 

EFVS and SVS operational concepts. If a significant F-value was obtained in an ANOVA, then, 

Tukey Honestly Significant Difference (HSD) pair-wise comparison tests (α set at 0.05) were 

performed. HSD post hoc tests provide specific information on which means are different from each 

other when a significant F-test result is found on a main factor consisting of three or more levels. 

To further inspect visual behavior, data was divided into instrument, instrument-to-visual, visual, 

flare, and landing segments, shown below in Figure 10. Segments were chosen based upon standard 

pilot visual behavior at various heights while on short final in low visibility conditions. During each 

segment under manual flight control and auto-throttles, the pilot visual behavior is driven by known 

task loading. The instrument segment, from the initial start of 1000 ft to 50 ft above the DA/DH, 

requires the pilot to maintain attention on the primary flight display, maintaining flight on course and 

glideslope. Outside visual references are not available so attention is likely maintained inside the 

cockpit when only the HDDs are available. The instrument-to-visual segment is an altitude driven 

window, 50 feet above the DA/DH to the DA/DH, during which pilots would transition from the 

HDD to OTW to acquire the required visual references to continue to land. The visual segment 

begins from the DH to 50 feet HAT, with expected task loading to drive pilot attention out the 

window to complete the visual approach. From 50 feet HAT until all-wheels were in continuous 
contac

attent

pilot v

the ap

Analy

visual

locati

in the

evalua

headfull tr

neces

head u

once p

Data q

system

averag

system

suffic

4.1.4 D

Crew 

PF an

HDD

prefer

used t

signif

Wilco

level 

ct represented

tion OTW to c

visual behavi

pproach to lan

ysis to determ

l transitions: i

on specific ey

e same locatio

ate when a tra

up/head-dow

ransition OTW

sarily followe

up for at least

pilot visual at

quality was a

m utilized in t

ge data collec

m indicated av

cient for gross

Display Conce

display conc

nd PM were in

, Conventiona

rred display (r

to test for diff

ficant differen

oxon post hoc

of significanc

d the flare seg

complete the 

or from all-w

nd and rollout

Fig

mine visual tra

initial glance 

ye fixation du

on for a minim

ansition occur

wn visual beha

W. An initial 

ed by sequent

t 200 ms at an

ttention rema

ssessed postthe study. So

ction rate of 8

verage spatia

s measuremen

ept Preferenc

ept preferenc

ndependently 

al HUD, SVS

rank=5) for fl

ferences amon

nce was found

c tests, a Bonf

ce [36]. 

gment, during

flare based up

wheel contact u

t is considered

gure 10. Eye T

ansition altitud

and full trans

uration and co

mum of 200 m

rred, data wer

avior to determ

glance was d

tial fixations 

ny point witho

ined continuo

analysis to de

ome data drop

87% across al

al accuracy of

nt of head up/

ces 

ces for flying 

asked to rank

S HUD, and E

flying with in 

ng the display

d, Wilcoxon n

ferroni correc

34 

g which the ta

pon visual ref

until the grou

d complete an

Tracking Analy

de and directi

sition. The fa

ontinuance of

milliseconds (m

re analyzed u

mine the diffe

determined to 

OTW, or in o

out continued

ously OTW. 

etermine the o

pouts did occu

l pilots for all

f the eye gaze 

/head down an

with in low-v

k order the di

EFVS) from th

low-visibility

y concepts for

non-parametri

ction was appl

ask of landing

ferences. The

und speed reac

nd taxi operat

ysis Segments 

ion were brok

actors separat

f fixations in t

ms) were def

using a movin

erence betwee

be the first si

other words, p

d attention OT

overall effecti

ur while in tes

l analysis seg

to be approx

nalysis. 

visibility cond

isplay concep

heir most pre

y conditions.

r the pilot dis

ic post hoc te

lied and all ef

g is predicted 

e landing seg

ches 60 knots

tions begin. 

ken into two s

ting these tran

that location. 

fined as a fixa

ng two second

en an initial g

ingle fixation

pilot attention

TW. A full tr

iveness of the

st production

gments. Calib

ximately three

ditions were a

pts (Conventio

ferred display

 Friedman’s A

splay preferen

ests were used

ffects are repo

to drive pilot

gment evaluat

s, at which po

specific types

nsitions are 

 Eye gaze ve

ation [35]. To

d gated averag

glance OTW a

n OTW, not 

n concentrated

ransition occu

e eye tracking

, resulting in

bration of the 

e degrees, 

also assessed.

onal HDD, SV

y (rank=1) to 

ANOVA [36]

nce rankings. 

d. For the 

orted at a 0.0

t

tes 

oint 

s of 

ectors 

o

ge of 

and a 

d

urred 

g

an 

 The 

VS 

least 

] was 

 If a 

1 
35 

4.1.5 CDTI Influences on Situation Awareness and Traffic Awareness 

ANOVAs on paired-comparisons technique responses were employed to assess Situation Awareness 

and Traffic Awareness differences for the three CDTI formats tested. If a significant F-value was 

obtained, Student-Newman-Keuls (SNK) post-hoc tests (α set at 0.05) were performed. 

4.2 EFVS Operational Concept Comparisons 

4.2.1 Approach Performance – Instrument Segment 

The three concepts used for EFVS operational concept comparisons for approach performance during 

the instrument segment (from 1000 ft to DH) were: 

1) Conventional HDD with TDZ/CL lights (1800 ft RVR/200 ft DH) 

2) EFVS (1400 ft RVR/150 ft DH) 

3) EFVS (1000 ft RVR/150 ft DH) 

The Conventional HDD flown in 1800 ft RVR to a 200 ft DH and a runway with TDZ/CL lights was 

considered the operational baseline concept for the EFVS operational concept comparisons. 

Throughout the remainder of this report, it will be referred to as one of the EFVS operational 

concepts tested in this experiment. 

The EFVS display runs evaluated for the 1000 ft and 1400 ft RVR conditions included the runs made 

with and without TDZ/CL lights. In the visibility conditions tested, the TDZ/CL lights were not 

visible to the crew during the instrument segment of the approach. 

Approach Statistics 

In Table 8, the approach statistics (mean, standard deviation, minimum value, and maximum value) 

are shown for the EFVS operational concepts, including the baseline condition. Also provided in 

Table 8 are the number of runs that resulted in a go-around and the total number of runs for the EFVS 

operational concepts tested. 
36 

Table 8. Approach Statistics (1000 ft to DH) for EFVS Operational Concepts 

Conventional HDD  

With TDZ/CL lights   EFVS HUD  

1800 ft RVR/200 ft DH   1400 ft RVR/150 ft DH   1000 ft RVR/150 ft DH

Number of Go‐Around / Number of Runs 2/12   0/24   0/24  

RMS Localizer Deviation

(dots)

Mean 0.05 0.06   0.05  

Std Dev 0.02 0.05   0.03  

Min 0.01 0.02   0.01  

Max 0.07 0.25   0.16  

RMS Glideslope Deviation

(dots)

Mean 0.28 0.09   0.11  

Std Dev 0.09 0.05   0.06  

Min 0.15 0.03   0.05  

Max 0.46 0.24   0.34  

RMS Sink Rate Deviation

(fpm)

Mean 177.44 72.77   94.05  

Std Dev 52.16 50.30   81.82  

Min 116.23 32.31   41.09  

Max 287.93 294.17   436.99  

All EFVS display concept approaches concluded in a landing; while, two of the EFVS operational 

baseline concept (Conventional HDD with TDZ/CL Lights in 1800 ft RVR/200 ft DH) approaches 

resulted in a go-around. Video review of these two baseline runs indicated that the PF did not have 

the required visual cues to continue the landing at the 200 ft DH. Therefore, the crew properly 

followed the EFVS crew procedures and performed the go-around. 

ANOVA Analyses 

ANOVA analyses revealed that there were no significant (p>0.05) differences for RMS localizer 

deviation among the three concepts used for the EFVS operational concept comparisons. However, 

RMS glideslope deviation (F(2,53)=32.243, p<0.0001) and RMS sink rate deviation (from a nominal 

3-degree glideslope value) (F(2,53)=9.321, p<0.0001) were significantly less for the EFVS display 

concepts compared to the Baseline concept (Conventional HDD with TDZ/CL Lights in 1800 ft 

RVR/200 ft DH). SNK post hoc tests revealed that there were no significant differences between the 

1400 ft and 1000 ft RVR EFVS display concepts for either the RMS glideslope deviation or RMS 

glideslope deviation measures. For RMS glideslope deviation, the EFVS concepts had a mean=0.10 

dots with a standard deviation () = 0.05 dots, and the Conventional HDD concept had a mean=0.28 

dots with =0.09 dots. Similarly for RMS sink rate deviation, the EFVS concepts had a mean=84.24 

fpm with =56.27 fpm, and the Conventional HDD concept had a mean=177.44 fpm with =52.16 

fpm. 

Objective Approach Standards Analysis 

Existing performance-based approach standards were also applied in the objective data analysis. 

These standards emphasize the maximum glideslope and localizer deviations, instead of RMS 

deviation as shown in Table 8. The approach data were analyzed only for those approaches that were 

flown to touchdown. 

The percentage of approaches which met the localizer and glideslope criteria of the four existing 
37 

approach performance standards are shown, broken down by EFVS operational concept in Table 9. 

Note that in Table 9, the JAR AWO values were based on the percentage of runs flown and did not 

include the statistical analysis to ensure a 95% bound as per JAR-AWO. To apply this additional 

constraint on the data, the Continuous Method [32] technique was used to calculate the probability of 

success, P(α), of meeting the AWO exceedance criteria (1/3 dot localizer, 1 dot glideslope) 95% of 

the time with required levels of confidence for the different EFVS operational concepts flown. The 

calculated P(α) values are shown in Table 10, broken down by EFVS operational concept. The 

influence of DA is shown in Tables 9 and 10, where, if the criteria window included “DA” (e.g., in 

PTS and JAR AWO), then the analysis used a corresponding altitude window down to 200 ft or to 

150 ft as appropriate. 

Table 9. Percentage approaches successfully meeting approach performance standards. 

Localizer Glideslope

Conventional

HDD with TDZ/CL

lights

EFVS HUD EFVS HUD Conventional HDD

with TDZ/CL lights

EFVS HUD EFVS HUD

1800 ft RVR/

200 ft DH

1400 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

1800 ft RVR/

200 ft DH

1400 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

FAA‐S‐8081‐4D 100% 100% 100% 100% 100% 100%

AC 120‐29 100% 100% 100% 90% 100% 95%

FAR 91 100% 100% 100% 40% 100% 91%

JAR‐AWO 100% 100% 100% 40% 100% 95%

Table 10. Probability of Success in meeting AWO exceedance critieria 

Display Concept RVR/DH (ft) Localizer P() Glideslope P() Number of Runs Number of Go‐Around

Conventional HDD 1800/200 100 29 12 2

EFVS 1000/150 100 95 22 0

EFVS 1400/150 100 97 24 0

The data indicate that: 

 Localizer tracking was never an issue, irrespective of the criteria. 

 Glideslope tracking performance appears to be affected by display location (head-up versus 

head-down). 

 EFVS display concepts were able to meet the JAR-AWO localizer and glideslope criteria 

from 300 ft to 100 ft HAT without any go-arounds being performed. 

4.2.2 Approach Performance – Visual Segment 

The five concepts used for EFVS operational concept comparisons for approach performance during 

the visual segment (from DH to threshold crossing) were: 

1) Conventional HDD with TDZ/CL lights (1800 ft RVR/200 ft DH) 

2) EFVS with TDZ/CL lights (1400 ft RVR/150 ft DH) 
38 

3) EFVS without TDZ/CL lights (1400 ft RVR/150 ft DH) 

4) EFVS with TDZ/CL lights (1000 ft RVR/150 ft DH) 

5) EFVS without TDZ/CL lights (1000 ft RVR/150 ft DH) 

The Conventional HDD flown in 1800 ft RVR to a 200 ft DH and a runway with TDZ/CL lights was 

considered the operational baseline concept for the EFVS operational concept comparisons. 

In Tables 11 - 13, statistics of lateral path performance, vertical path performance, and sink rate 

deviation at 100 ft HAT and at threshold crossing are provided to evaluate how effectively the pilots 

could use the different EFVS operational concepts during the visual portion of the approach segment. 

The maximum bank angle from threshold crossing to touchdown and bank angle at touchdown were 

also used to evaluate the effectiveness of the different EFVS operational concepts and are presented 

in Table 14. 

Table 11. Lateral Path Performance Statistics of EFVS Operational Concepts in the Visual Segment of the 

Approach 

Conventional

HDD

With

TDZ/CL Lights

EFVS HUD

With

TDZ/CL

Lights

EFVS HUD

Without

TDZ/CL Lights

EFVS HUD

With

TDZ/CL Lights

EFVS HUD

Without

TDZ/CL Lights

1800 ft RVR/

200 ft DH

1400 ft RVR/

150 ft DH

1400 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

Number of Go‐Around / Number of Runs 2/12 0/12 0/12 0/10 0/12

Lateral Path Deviation

at 100 ft HAT (ft)

Mean  ‐7.7  ‐9.4  ‐4.7 3.2 1.7

Std Dev 15.7 13.5 14.7 20.6 13.4

Max Left  ‐39.1  ‐31.5  ‐29.3  ‐33.1  ‐29.0

Max Right 16.8 11.5 17.7 31.0 18.2

Lateral Path Deviation

at Threshold Crossing

(ft)

Mean  ‐6.7  ‐7.9  ‐2.6 0.3  ‐1.0

Std Dev 17.5 9.7 19.1 16.7 12.9

Max Left  ‐37.2  ‐24.1  ‐36.7  ‐30.4  ‐31.4

Max Right 22.1 10.3 21.6 26.0 14.7
39 

Table 12. Vertical Path Performance Statistics of EFVS Operational Concepts in the Visual Segment of 

the Approach 

Conventional

HDD

With

TDZ/CL Lights

EFVS HUD

With

TDZ/CL

Lights

EFVS HUD

Without

TDZ/CL Lights

EFVS HUD

With

TDZ/CL Lights

EFVS HUD

Without

TDZ/CL Lights

1800 ft RVR/

200 ft DH

1400 ft RVR/

150 ft DH

1400 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

Number of Go‐Around / Number of Runs 2/12 0/12 0/12 0/10 0/12

Vertical Path

Deviation at 100 ft

HAT (ft)

Mean 2.4  ‐2.6  ‐5.1  ‐0.8  ‐4.9

Std Dev 18.3 3.6 3.5 6.0 4.0

Max Below  ‐27.5  ‐6.8  ‐9.7  ‐11.1  ‐11.8

Max Above 27.2 3.4 1.0 7.5 3.6

Altitude at Threshold

Crossing (ft)

Mean 58.6 53.5 50.8 54.7 53.3

Std Dev 14.2 6.1 8.4 4.8 7.2

Min 28.4 40.3 43.0 47.1 45.6

Max 79.4 64.6 68.9 62.1 68.4

Table 13. Sink Rate Statistics of EFVS Operational Concepts in the Visual Segment of the Approach 

Conventional

HDD

With

TDZ/CL Lights

EFVS HUD

With

TDZ/CL

Lights

EFVS HUD

Without

TDZ/CL Lights

EFVS HUD

With

TDZ/CL Lights

EFVS HUD

Without

TDZ/CL Lights

1800 ft RVR/

200 ft DH

1400 ft RVR/

150 ft DH

1400 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

Number of Go‐Around / Number of Runs 2/12 0/12 0/12 0/10 0/12

Sink Rate at 100 ft HAT

(ft/min)

Mean  ‐688.3  ‐699.4  ‐704.2  ‐666.4  ‐689.3

Std Dev 165.6 79.5 93.2 75.0 72.9

Max  ‐1059.9  ‐823.1  ‐876.0  ‐762.9  ‐820.9

Sink Rate at Threshold

Crossing (ft/min)

Mean  ‐663.7  ‐602.4  ‐562.3  ‐636.8  ‐549.2

Std Dev 241.0 130.8 162.0 130.9 125.3

Max  ‐922.7  ‐731.6  ‐850.1  ‐785.9  ‐815.4

ANOVA analyses indicated no significant differences (p>0.05) for lateral deviation from centerline, 

vertical deviation from path, or sink rate among the five EFVS operational concepts tested at both the 

100 ft HAT and Threshold Crossing Locations. 
40 

Table 14. Bank Angle Statistics of EFVS Operational Concepts 

Conventional

HDD

With

TDZ/CL Lights

EFVS HUD

With

TDZ/CL Lights

EFVS HUD

Without

TDZ/CL Lights

EFVS HUD

With

TDZ/CL Lights

EFVS HUD

Without

TDZ/CL Lights

1800 ft RVR/

200 ft DH

1400 ft RVR/

150 ft DH

1400 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

Max Bank Angle

from Threshold

Crossing to

Touchdown (deg)

Mean 3.5 3.4 3.4 5.2 4.0

Std Dev 2.5 2.6 2.4 4.8 3.1

Max 9.2 9.8 8.8 14.2 11.0

Bank Angle at

Touchdown (deg)

Mean 1.9 2.1 1.6 3.1 3.3

Std Dev 1.9 2.7 0.9 3.6 3.3

Max 6.0 9.8 3.2 12.4 11.1

ANOVA analyses indicated no significant differences (p>0.05) among the five EFVS operational 

concepts for the maximum bank angle from threshold to touchdown or for the bank angle at 

touchdown measures. 

4.2.3 Landing Performance 

The five concepts used for EFVS operational concept comparisons for landing performance were: 

1) Conventional HDD with TDZ/CL lights (1800 ft RVR/200 ft DH) 

2) EFVS with TDZ/CL lights (1400 ft RVR/150 ft DH) 

3) EFVS without TDZ/CL lights (1400 ft RVR/150 ft DH) 

4) EFVS with TDZ/CL lights (1000 ft RVR/150 ft DH) 

5) EFVS without TDZ/CL lights (1000 ft RVR/150 ft DH) 

The Conventional HDD flown in 1800 ft RVR to a 200 ft DH and a runway with TDZ/CL lights was 

considered the operational baseline concept for the EFVS operational concept comparisons. 

Landing Statistics 

In Table 15, the touchdown (T/D) statistics (mean, standard deviation, minimum value, and maximum 

value) are shown, broken out by RVR and TDZ/CL light configuration, for the EFVS operational 

concepts. The T/D measures (lateral position, longitudinal position and sink rate) means were used to 

determine which touchdown performance rating level (Desired, Adequate, or Not Adequate) as 

defined in Table 6 was achieved for the EFVS operational concepts. Also provided in Table 15 are 

the number of runs that resulted in a go-around and the total number of runs for the EFVS operational 

concepts tested. Note that for touchdown lateral position data in Table 15, the “min” value equates to 

the maximum deviation to the left of centerline and the “max” value equates to the maximum 

deviation to the right of centerline. 
41 

Table 15. Landing Statistics for EFVS Operational Concepts 

Conventional

HDD

With

TDZ/CL Lights

EFVS HUD

Without

TDZ/CL Lights

EFVS HUD

With

TDZ/CL Lights

EFVS HUD

Without

TDZ/CL Lights

EFVS HUD

With

TDZ/CL Lights

1800 ft RVR/

200 ft DH

1400 ft RVR/

150 ft DH

1400 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

Number of  

Go‐Around /

Number of Runs

2/12 0/12 0/12 0/12 0/11

T/D Longitudinal

Position (ft)

Mean 923.7 1016.3 1097.6 798.6 1026.3

Std Dev 324.7 401.9 431.1 358.5 288.9

Min 194.1 396.4 452.9 430.0 706.9

Max 1289.8 1682.6 1942.5 1662.7 1635.4

Rating Desired Desired Desired Desired Desired

T/D Lateral

Position (ft)

Mean  ‐0.8 2.4  ‐3.7  ‐1.2 0.6

Std Dev 8.2 19.2 11.4 14.7 12.7

Min  ‐10.9  ‐40.5  ‐31.8  ‐30.8  ‐16.0

Max 14.9 26.3 11.5 25.9 19.7

Rating Desired Desired Desired Desired Desired

T/D Sink Rate

(fps)

Mean  ‐8.1  ‐6.6  ‐6.7  ‐7.7  ‐7.6

Std Dev 2.7 3.2 1.9 3.7 3.0

Min  ‐4.0  ‐0.9  ‐3.5  ‐3.4  ‐4.5

Max  ‐11.9  ‐11.9  ‐9.6  ‐15.8  ‐12.7

Rating Adequate Adequate Adequate Adequate Adequate

The data show that, on average, all five EFVS operational concepts: 

 Were within the “Desired” landing position performance criteria (laterally within 27 ft of 

centerline and longitudinally between 750 to 2250 ft from threshold) as defined in Table 6. 

 Had higher than expected touchdown sink rates with only “Adequate” sink rate performance 

criteria (between six to 10 feet/second) being met as defined in Table 6. 

ANOVA Analyses 

ANOVAs revealed no significant differences (p>0.05) for touchdown longitudinal position 

(mean=973 ft, =370 ft), lateral position (mean=-0.55 ft, =13.6 ft), and sink rate (mean=-7.3 ft/sec, 

=2.9 ft/sec) for the five EFVS operational concepts tested. 

Objective Landing Standards Analysis 

In Figure 11, the touchdown data are shown, for the four EFVS display concepts and the EFVS 

operational baseline concept. Included on this plot is the touchdown aim point (0 ft lateral distance, 

1000 ft longitudinal distance) indicated by the axes origin and the landing box (laterally within 58 ft 

of centerline and longitudinally between 200 to 2700 ft from threshold) indicated by the dashed 

rectangle. In Figure 12, the pitch angle and bank angle at touchdown are shown. Included on this 

plot are the normal landing ground contact angles for the simulated aircraft indicated by the solid-
lined 

Visua

define

200 ft

conce

Opera

were n

None 

criteri

[34, A

contour on th

al inspection o

ed in AC-120

t to 2700 ft lo

ept (EFVS op

ational concep

no wing tip o

of the five EF

ia for T/D lon

Appendix 3]. 

he plot. 

Figure 11. To

of the data sho

0-28D and JA

ongitudinal di

erational base

pts, including

or tail strikes. 

FVS operatio

ngitudinal pos

ouchdown Foo

ows that 1) al

R AWO (i.e.,

stance from t

eline concept)

g EFVS opera

onal concepts 

sition, lateral 

42 

otprint for EFV

ll EFVS displ

, within 58 f

hreshold); 2) 

) landings wa

ational baselin

tested met th

position from

VS Operational 

lay concepts w

ft lateral dista

all but one o

as within the l

ne concept, la

he 10-6 probab

m centerline, o

concepts 

were within t

ance of center

f the Convent

landing box 3

anded on the r

bility of occur

or sink rate as

the landing bo

rline and betw

tional HDD 

3) all EFVS 

runway; 4) th

rrence auto-la

s per AC120-

ox 

ween 

here 

and 

28D 
43 

Figure 12. Touchdown Pitch angle and Bank Angle Footprint for EFVS Operational concepts 

4.2.4 Rollout Performance 

Rollout statistics (RMS lateral deviation from centerline and maximum lateral deviation from 

centerline) for the five EFVS operational concepts tested are provided in Table 16. 

Table 16. Rollout Statistics for EFVS Operational Concepts 

Conventional

HDD  

With

TDZ/CL

Lights  

EFVS HUD  

Without

TDZ/CL

Lights  

EFVS HUD

With

TDZ/CL

Lights  

EFVS HUD  

Without

TDZ/CL

Lights  

EFVS HUD

With

TDZ/CL

Lights  

1800 ft RVR/

200 ft DH  

1400 ft RVR/

150 ft DH  

1400 ft RVR/

150 ft DH  

1000 ft RVR/

150 ft DH  

1000 ft RVR/

150 ft DH  

RMS Lateral

Deviation from

Centerline (feet)

Mean 4.4 7.8 6.7 7.3 9.2

Std Dev 1.9 4.1 4.2 4.0 3.3

Maximum Lateral

Deviation from

Centerline (feet)

Mean 10.1 18.3 13.5 14.4 17.9

Std Dev 4.1 11.7 8.8 7.9 8.5

An ANOVA revealed no significant differences (p>0.05) among the five EFVS operational concepts 

tested for RMS lateral deviation from centerline (mean=7.1 ft, =3.5 ft) or maximum lateral deviation 

from centerline (mean=14.9 ft, =8.2 ft) during rollout operations. 
44 

4.2.5 Pilot Workload 

An ANOVA revealed significant differences (F(4,52)=3.912, p=0.008) between the EFVS operational 

concepts for the PF post-run workload ratings. SNK post hoc tests revealed that the Conventional 

HDD concept (mean=3.6) was rated as having significantly higher workload than the four EFVS 

display concepts (mean=2.8). There were no appreciable PF workload differences among the four 

EFVS display concepts tested. 

An ANOVA revealed no significant differences (p>0.05) in PM post-run workload ratings 

(mean=2.6) among the five EFVS operational concepts tested. 

Crews rated their workload as being “moderate, easily managed and having considerable spare time” 

while using the EFVS display concept on approach through landing in visibilities as low as 1000 ft 

RVR. 

4.2.6 Eye Tracking 

Eye tracking analysis for the EFVS operational concepts are broken into two main quantitative 

metrics: Head-up percent and transition count from instruments to OTW by segment of approach and 

landing. EFVS operational concepts utilized both the HUD and HDD display locations. Therefore, 

analysis of OTW transition height above threshold and gaze direction was not performed. 

Statistically significant effects are circled in red in Figures 13 and 14. 

Comparing the visual behavior across the EFVS operational concept display conditions, Tukey HSD 

post hoc tests revealed there was a predictable significant effect (Figure 13) for head up percentage 

time between the Conventional HDD concept (EFVS operational baseline concept) and all other 

EFVS HUD concepts for all in-flight segments (Instrument, Transition, and Visual). Of note, during 

the visual segment when pilot attention is expected to be OTW to make the visual approach (DH to 

50 ft HAT) pilots operating with the Conventional HDD condition remained inside the flight deck 

looking at instrumentation over 35% of the time (Figure 13). There was no significant difference 

between any of the conditions for the flare or landing segments. During the flare segment (50 ft HAT 

to touchdown) the runway environment and threshold lights were visible, drawing pilot attention 

nearly completely OTW. Pilot head up time varied between 77% and 84% during the landing rollout, 

indicating pilot attention remained inside the flight deck approximately 16-23% of the time, 

referencing airspeed and the moving map display available on the ND. Table 17 shows the results for 

head up percentage of the three EFVS operational concepts by eye tracking analysis segments. 
There

Conv

Conv

with a

OTW

thoug

pilots

segme

e was a signifi

entional HDD

entional HDD

an average ne

W during the vi

gh the approac

 make at mos

ents, with all 

  

Instrument

Segment

Transition

Segment

Visual

Segment

Flare

Segment

Landing

Segment

Figure

Table 1

ficant differen

D and all of th

D visual trans

ear four, verify

isual segment

ch lights had b

st one or no tr

information n

C

Mean  

Std Dev  

Mean  

Std Dev

Mean  

Std Dev

Mean  

Std Dev  

Mean  

Std Dev  

e 13. EFVS Op

7. EFVS Op C

nce (Figure 18

he EFVS cond

itions during 

fying pilots’ a

t (Figure 18) 

been acquired

ransitions insi

necessary to c

onventional

HDD  

1800 RVR/

200 DH  

24.3

16.1

44.9

24.8

65.5

29.4

92.4

12.8

83.4

14.9

45 

p Concept, Hea

Concepts, He

8) in the numb

ditions for all

the transition

attention conti

to check airsp

d to continue 

ide the flight 

complete the 

EFVS HUD   E

1400 RVR/

150 DH  

1

95.0

4.1

96.3

5.3

94.2

14.3

91.4

15.8

77.8

17.3

ad Up Time (%

ead Up Time 

ber of visual t

l segments of

n and visual s

inues to trans

peed, guidanc

the approach

deck during t

approach to l

EFVS HUD  

1000 RVR/

150 DH  

95.6

3.6 F(

96.8

3.6 F(

98.7

1.9 F(

95.5

10.6

83.4

13.5

%) 

(%) 

transitions be

f flight. The n

egments were

sition between

ce and attitud

h. When equi

the transition 

land available

ANOVA

  

Display/RVR Co

2,51) = 298.13,

Display/RVR Co

(2,51) = 73.46, p

Display/RVR Co

(2,51) = 14.81, p

No Significa

  

No Significa

  

etween the 

number of 

e nearly the s

n instruments

de indicators e

pped with a H

and visual 

e on the HUD

ndition

p < 0.000

ndition

p < 0.000

ndition

p < 0.000

nce

nce

ame 

 and 

even 

HUD, 

D. 
Durin

reduc

Durin

behav

and C

inform

runwa

differ

RVR 

percen

inform

the th

ng the flare se

es significant

ng the landing

vior observed 

Conventional H

mation availab

ay relative to 

rence across E

below 1400 f

ntages were s

mation availab

hree EFVS op

egment, transi

tly to one or l

g and rollout, 

with head up

HDD during 

ble on the mo

the runway e

EFVS operatio

feet limits pilo

similar across

ble head dow

erational con

Figure 14. 

ition count for

ess, with near

pilots began t

p time. Trans

the landing se

oving map dis

exit taxiway. 

onal concepts

ot transitions 

s all condition

wn. Table 18 s

cepts by eye t

 EFVS Op Con

46 

r pilots opera

rly all attentio

to transition b

itions were ob

egment. This

splay aiding th

Tukey HSD p

s is due to var

between head

ns suggesting 

shows the res

tracking analy

ncepts, Head U

ating under th

on focused O

back inside th

bserved with 

s behavior is l

the pilot awar

post hoc tests

riance in OTW

d up and head

these transiti

sults for head 

ysis segments

Up/Down Tran

he Convention

OTW. 

he flight deck

the use of bo

likely due to 

reness of aircr

s indicate the 

W RVR, sugg

d down. How

ions were for 

up/head dow

s. 

nsitions 

nal HDD conc

k, confirming

oth the EFVS 

additional 

raft position o

significant 

gesting reduce

wever, head u

short glances

wn transitions

cept 

the 

HUD 

on the 

ed 

up 

s to 

of 
47 

Table 18. EFVS Op Concepts, Head Up/Head Down Transitions 

  

Conventional

HDD   EFVS HUD   EFVS HUD   ANOVA

1800 RVR/

200 DH  

1400 RVR/

150 DH  

1000 RVR/

150 DH     

Instrument

Segment

Mean   11.6 3.8 4.1 Display/RVR Condition

Std Dev   9.4 4.0 4.1 F(2,51) = 73.46, p < 0.000

Transition

Segment

Mean   3.3 0.5 0.6 Display/RVR Condition

Std Dev   2.6 1.0 1.1 F(2,51) = 12.97, p < 0.000

Visual

Segment

Mean   4.2 0.4 0.3 Display/RVR Condition

Std Dev   2.6 0.8 0.7 F(2,51) = 31.41, p < 0.000

Flare

Segment

Mean   1.0 0.1 0.2 No Significance

Std Dev   1.8 0.3 0.5    

Landing

Segment

Mean   5.3 6.0 2.9 No Significance

Std Dev   3.8 4.4 2.5    

4.3 SVS Operational Concept Comparisons 

4.3.1 Approach Performance – Instrument Segment 

The five concepts used for SVS operational concept comparisons for approach performance during 

the instrument segment (from 1000 ft to DH) were: 

1) Conventional HUD without TDZ/CL lights (1400 ft RVR/150 ft DH) 

2) SVS HUD (1400 ft RVR/150 ft DH) 

3) SVS HUD (1000 ft RVR/150 ft DH) 

4) SVS HDD (1800 ft RVR/150 ft DH) 

5) SVS HDD (1400 ft RVR/150 ft DH) 

The Conventional HUD flown in 1400 ft RVR to a 150 ft DH and a runway without TDZ/CL lights 

was considered the operational baseline concept for the SVS operational concept comparisons. 

Throughout the remainder of this report, it will be referred to as one of the SVS operational concepts 

tested in this experiment. 

The SVS display runs evaluated for the 1000, 1400, and 1800 ft RVR conditions included the runs 

made with and without TDZ/CL lights. In the visibility conditions tested, the TDZ/CL lights were 

not visible to the crew during the instrument segment of the approach. 

Approach Statistics 

In Table 19, the approach statistics (mean, standard deviation, minimum value, and maximum value) 
48 

are shown for the five SVS operational concepts. Also provided in Table 19 are the number of runs 

that resulted in a go-around and the total number of runs for the SVS operational concepts tested. 

Table 20 provides event description data gathered from video review of the go-around runs for the 

five SVS Operational Concepts tested. 

Table 19. Approach Statistics (1000 ft to DH) for SVS Operational Concepts 

Conventional

HUD Without

TDZ/CL Lights

SVS HUD SVS HDD

1400 ft RVR/

150 ft DH

1400 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

1800 ft RVR/

150 ft DH

1400 ft RVR/

150 ft DH

Number of Go‐Around /

Number of Runs 1/12   4/24   7/24   1/24   4/24  

RMS Localizer

Deviation (dots)

Mean 0.05 0.05 0.04 0.05 0.04

Std Dev 0.02 0.02 0.02 0.03 0.02

Min 0.03 0.02 0.02 0.01 0.01

Max 0.08 0.09 0.08 0.12 0.10

RMS Glideslope

Deviation (dots)

Mean 0.11 0.08 0.09 0.26 0.25

Std Dev 0.10 0.04 0.04 0.13 0.16

Min 0.04 0.03 0.03 0.08 0.09

Max 0.38 0.19 0.20 0.64 0.68

RMS Sink Rate

Deviation (fpm)

Mean 93.2 61.1 67.1 141.8 134.1

Std Dev 104.4 16.4 15.6 66.8 82.8

Min 39.7 28.4 31.4 56.9 58.7

Max 399.4 85.6 95.8 358.9 393.9
49 

Table 20. Go-Around Event Description with SVS Operational Concepts 

Display Concept TDZ/CL Lights OTW RVR (ft) Event Description

Conventional

HUD   OFF 1400

PM called 'lights' and then PF called 'lights.'  PF did not see runway environment

at minimums so he performed a go‐around.  PM could see runway enviornment

and said 'you are right down the centerline' as the PF called for go‐around.

SVS HUD OFF 1400

PF turned imagery off shortly before ‘minimums’ call.    PF called ‘rabbits’, but

then called ‘go‐around’.   Post‐run PF commented that he had threshold lights

but did not feel comfortable continuing landing with just threshold lights.

SVS HUD ON 1400

PF turned imagery OFF/ON near ‘approaching minimums’ and ‘minimums’ calls.

PM called ‘lights’, followed by PF calling ‘lights.’  PF called ‘going around’ as he

didn’t have threshold lights.

SVS HUD OFF 1400

PF never turned imagery off.  PF called ‘nothing, going around’ as he did not see

threshold lights.

SVS HUD ON 1400

PF never turned imagery off.  PM called ‘lights’, but PF did not.  PF called ‘going

around.’

SVS HUD ON 1000

PF turned imagery off shortly before ‘minimums’ call.    PF saw nothing and

performed go‐around at minimums.

SVS HUD OFF 100

PF turned imagery off shortly before ‘approaching minimums’ call.   PM called

‘rabbits’, followed by PF calling ‘rabbits’, and then PM called ‘lights’. PF called

‘go around’ when he didn’t see lights by ‘minimums’ call.

SVS HUD OFF 1000

PF turned imagery off at 500 ft AGL.    PM called ‘lights’ and then PF called

‘lights’.  PM saw threshold lights but PF did not so called go‐around.   

SVS HUD OFF 1000

After descending through 400 ft AGL, PF turned imagery on/off every 20 feet or

so.  PM called ‘lights’.  With crosswind and not seeing lights, PF decided to go‐

around at 200 ft AGL.

SVS HUD ON 1000

PF never turned imagery off as he was trying to look through SV image.    PM

called ‘lights’.  PF did not see lights so he performed a go‐around.

SVS HUD OFF 1000

PF never turned imagery off as he was trying to see lights in blank area of SV

image.  PM called ‘lights.’ PF did not see lights so he performed a go‐around.

SVS HUD ON 1000

PF never turned imagery off.  PM called ‘lights’, and then PF called ‘lights.’  PM

called ‘field in sight’, but PF did not see threshold lights at minimums so

performed go‐around.

SVS HDD OFF 1800

PF called ‘lights, landing‘.  PF went around at ~35 ft AGL. [Video review showed

aircraft 2 dots high prior to go‐around call.]

SVS HDD ON 1400

PF called ‘rabbits.’ PM called ‘you are high’ and PF called ‘going around.’  Post

run PF commented that he didn’t see enough to land.

SVS HDD OFF 1400

PM called 'lights' and then PF called 'lights.'  PM called 'runway' and PF called

'landing'.  [Video review showed aircraft was 1 dot low and unstable in roll]. PF

called go‐around at 5 ft AGL.

SVS HDD ON 1400

PM called 'lights' and then PF called 'lights.'  PM called 'runway' and PF called

'go around' at minimums because he didn’t have threshold lights.

SVS HDD ON 1400

PM called ‘lights’ and then PF called ‘lights’.  PF called ‘landing’, but head‐down

to head‐up transition and pilot‐induced oscillations in roll caused instability.  PF

called ‘go‐around’ at 25 ft AGL.

The data appears to indicate that the SVS imagery brightness setting may have occluded the approach 

lights for some of the pilots, but not all. The pilots were trained on how to best set the 

brightness/contrast of the SVS image. They were allowed to adjust it at any time during the test and 
50 

the principal investigator specifically had them set it at the beginning of the day to their personal 

preferences. Comparing the SVS HUD go-around numbers in Table 19 to the EFVS go-around 

numbers in Table 8, the EFVS (FLIR imagery) appears to have enhanced the approach lights for the 

pilot enabling him to line up and see the required visual cues for landing; while the SVS imagery may 

have occluded the approach lights. 

RVR appears to have had an influence of the PF’s decision to land or go-around while using the SVS 

HUD. In the 1000 ft RVR condition, seven of the possible 24 SVS HUD runs resulted in a go-around 

while in the 1400 ft RVR condition four of the possible 24 SVS HUD runs resulted in a go-around. 

RVR also appears to have had an influence of the PF’s decision to land or go-around while using the 

SVS HDD. In the 1800 ft RVR condition, one of the possible 24 SVS HDD runs resulted in a goaround while in the 1400 ft RVR condition four of the possible 24 SVS HDD runs resulted in a goaround. 

ANOVA Analyses 

An ANOVA revealed that there were no significant (p>0.05) differences for RMS localizer deviation 

(mean=0.05 dots with =0.02 dots) among the five SVS operational concepts. 

An ANOVA revealed significant differences (F(4,86)=13.446, p<0.0001) between the SVS 

operational concepts for RMS glideslope deviation. SNK post hoc tests indicated the RMS glideslope 

deviation was significantly less for the HUD concepts (Conventional and SVS) compared to the HDD 

SVS concepts. For RMS glideslope deviation, the HUD concepts had a mean=0.09 dots with = 0.05 

dots and the HDD concepts had a mean=0.26 dots with =0.15 dots. 

An ANOVA revealed significant differences (F(4,86)=6.9445, p<0.0001) between the SVS 

operational concepts for RMS sink rate deviation (from a nominal 3-degree glideslope value). SNK 

post hoc tests showed the RMS sink rate deviation was significantly less for the SVS HUD concepts 

(mean=64 fpm and =16 fpm) compared to SVS HDD concepts (mean=138 fpm and =75 fpm). 

There were no significant differences between the Conventional HUD concept (mean= 93 fpm and 

=104 fpm) and the SVS HUD concepts or the Conventional HUD concept and the SVS HDD 

concepts. 

Objective Approach Standards Analysis 

Existing performance-based approach standards were also applied in the objective data analysis for 

those SVS operational concept approaches that were flown to touchdown. 

The percentage of approaches which met the localizer and glideslope criteria of the four existing 

approach performance standards are shown, broken down by SVS operational concept in Tables 21 

and 22. Note that in Tables 21 and 22 the JAR AWO values were based on the percentage of runs 

flown which met the 1/3 dot localizer and 1 dot glideslope criteria between 300 ft and 100 ft AFL and 

did not include the statistical analysis to ensure a 95% bound as per JAR-AWO. To apply this 

additional constraint on the data, the Continuous Method [32] technique was used to calculate the 

probability of success, P(α), of meeting the AWO exceedance criteria (1/3 dot localizer, 1 dot 

glideslope) 95% of the time with required levels of confidence for the SVS operational concepts 

flown. The calculated P(α) values are shown in Table 23, broken down by SVS operational concept. 

The influence of DA is shown in Tables 21-23, where, if the criteria window included “DA” (e.g., in 

PTS and JAR AWO), the analysis used a corresponding altitude window down to 150 ft. 
51 

Table 21. Percentage SVS operatonal concept approaches successfully meeting localizer performance 

standards 

Localizer

Conventional HUD

without TDZ/CL

lights

SVS HUD SVS HDD

1400 ft RVR/

150 ft DH

1400 ft RVR/

150 ftDH

1000 ft RVR/

150 ft DH

1800 ft RVR/

150 ft DH

1400 ft RVR/

150 ft DH

FAA‐S‐8081‐4D 100% 100% 100% 100% 100%

AC 120‐29 100% 100% 100% 100% 100%

FAR 91 100% 100% 100% 100% 100%

JAR‐AWO 100% 100% 100% 100% 100%

Table 22. Percentage SVS operatonal concept approaches successfully meeting glideslope performance 

standards 

Glideslope

Conventional

HUD without

TDZ/CL lights

SVS HUD SVS HDD

1400 ft RVR/

150 ft DH

1400 ft RVR/

150 ft DH

1000 ft RVR/

150 ft DH

1800 ft RVR/

150 ft DH

1400 ft RVR/

150 ft DH

FAA‐S‐8081‐4D 100% 100% 100% 87% 90%

AC 120‐29 91% 100% 100% 91% 100%

FAR 91 73% 90% 94% 48% 55%

JAR‐AWO 82% 90% 94% 48% 55%

Table 23. Probability of Success in meeting AWO exceedance critieria with SVS operational concepts 

Display Concept RVR/DH

(ft)

Localizer

P()

Glideslope

P()

Number

of Runs

Number of  

Go‐Around

Conventional HUD without TDZ/CL Lights 1400/150 100 66 12 1

SVS HDD 1400/150 100   39   24   4  

SVS HDD 1800/150 99   30   24   1

SVS HUD 1000/150 100   96   24   7  

SVS HUD 1400/150 100   88   24   4  

The data indicate that: 

 Localizer tracking was never an issue, irrespective of the criteria. 

 Glideslope tracking performance appears to be affected by display location (head-up versus 

head-down) and the presence of SVS imagery. 

 SVS HUD in 1000 ft RVR condition was able to meet the JAR-AWO localizer and glideslope 

criteria but with a 29% (7 out of 24 runs) missed approach rate. 
52 

4.3.2 Approach Performance – Visual Segment 

The nine concepts used for SVS operational concept comparisons for approach performance during 

the visual segment (from DH to threshold crossing) were: 

1) Conventional HUD without TDZ/CL lights (1400 ft RVR/150 ft DH) 

2) SVS HUD with TDZ/CL lights (1400 ft RVR/150 ft DH) 

3) SVS HUD without TDZ/CL lights (1400 ft RVR/150 ft DH) 

4) SVS HUD with TDZ/CL lights (1000 ft RVR/150 ft DH) 

5) SVS HUD without TDZ/CL lights (1000 ft RVR/150 ft DH) 

6) SVS HDD with TDZ/CL lights (1800 ft RVR/150 ft DH) 

7) SVS HDD without TDZ/CL lights (1800 ft RVR/150 ft DH) 

8) SVS HDD with TDZ/CL lights and (1400 ft RVR/150 ft DH) 

9) SVS HDD without TDZ/CL lights (1400 ft RVR/150 ft DH) 

The Conventional HUD flown in 1400 ft RVR to a 150 ft DH and a runway without TDZ/CL lights 

was considered the operational baseline concept for the SVS operational concept comparisons. 

In Tables 24-26, statistics of lateral path performance, vertical path performance, and sink rate 

deviation at 100 ft HAT and at threshold crossing are provided to evaluate how effectively the pilots 

could use the different SVS operational concepts during the visual portion of the approach segment. 

The maximum bank angle from threshold crossing to touchdown and the bank angle at touchdown 

were also used to evaluate the effectiveness of the different SVS operational concepts, including the 

operational baseline, and are presented in Table 27. 

 
53 

Table 24. Lateral Path Performance Statistics of SVS Operational Concepts in the Visual Segment of the 

Approach 

Conventional

HUD  

Without

TDZ/CL Lights

SVS HUD

With

TDZ/CL

Lights

SVS HUD

Without

TDZ/CL

Lights

SVS HUD

With

TDZ/CL

Lights

SVS HUD

Without

TDZ/CL

Lights

SVS HDD

With

TDZ/CL

Lights

SVS HDD

Without

TDZ/CL

Lights

SVS HDD

With

TDZ/CL

Lights

SVS HDD

Without

TDZ/CL

Lights

1400 ft RVR

1400 ft

RVR

1400 ft

RVR

1000 ft

RVR

1000 ft

RVR

1800 ft

RVR

1800 ft  

RVR

1400 ft

RVR

1400 ft

RVR

No. Go‐Around /

No. Runs 1/12 2/12 2/12 3/12 4/12 1/12 0/12 3/12 1/12

Lat Path Dev at

100 ft HAT (ft)

Mean  ‐2.6  ‐1.2  ‐13.1 3.5  ‐5.3  ‐7.7  ‐4.1  ‐1.7  ‐3.5

Std

Dev

11.5 18.0 18.9 8.7 13.3 23.7 20.2 13.0 19.4

Max

Left ‐16.8  ‐22.0  ‐54.6  ‐12.5  ‐27.5  ‐58.2  ‐35.2  ‐24.4  ‐34.9

Max

Right 23.2 24.2 14.4 11.2 14.4 28.5 33.3 15.4 32.4

Lat Path Dev at Threshold

Crossing (ft)

Mean  ‐3.0 0.5  ‐10.4 4.4  ‐4.8  ‐0.4  ‐6.8  ‐9.9 0.7

Std

Dev

16.3 16.0 20.0 12.3 15.0 16.1 16.5 14.5 20.4

Max

Left ‐26.5  ‐24.4  ‐47.2  ‐12.1  ‐29.1  ‐36.1  ‐45.1  ‐39.9  ‐25.2

Max

Right 25.8 25.0 23.5 31.0 13.9 21.1 12.7 9.7 52.7
54 

Table 25. Vertical Path Performance Statistics of SVS Operational Concepts in the Visual Segment of the 

Approach 

Conventional

HUD  

Without

TDZ/CL Lights  

SVS HUD

With

TDZ/CL

Lights  

SVS HUD

Without

TDZ/CL

Lights  

SVS HUD

With

TDZ/CL

Lights  

SVS HUD

Without

TDZ/CL

Lights  

SVS HDD

With

TDZ/CL

Lights  

SVS HDD

Without

TDZ/CL

Lights  

SVS HDD

With

TDZ/CL

Lights

SVS HDD

Without

TDZ/CL

Lights

1400 ft RVR 1400 ft

RVR

1400 ft

RVR  

1000 ft

RVR  

1000 ft

RVR

1800 ft

RVR

1800 ft  

RVR

1400 ft

RVR  

1400 ft

RVR  

No. Go‐Around/

No. Runs 1/12   2/12   2/12   3/12   4/12   1/12   0/12   3/12   1/12  

Vert Path Dev at

100 ft HAT (ft)

Mean  ‐3.8  ‐1.4  ‐2.7  ‐3.6  ‐8.7  ‐1.1 6.7  ‐3.3 1.3

Std Dev 7.1 5.1 9.0 4.0 5.6 16.5 14.7 13.7 22.0

Max

Below ‐11.6  ‐7.2  ‐9.6  ‐10.3  ‐13.6  ‐30.6  ‐13.8  ‐29.0  ‐22.0

Max

Above 13.7 9.2 22.4 3.4 3.5 23.5 26.8 15.7 39.1

Altitude at Threshold

Crossing (ft)

Mean 51.5 52.6 50.8 45.4 51.3 52.9 52.9 52.2 54.9

Std Dev 10.2 5.0 8.8 4.0 12.8 16.7 16.7 17.0 21.9

Min 40.8 46.3 43.3 35.9 37.5 19.0 19.0 25.3 26.4

Max 69.9 61.6 72.3 49.1 75.4 74.0 74.0 76.1 90.2
55 

Table 26. Sink Rate Statisitics of SVS Operational Concepts in the Visual Segment of the Approach 

 

Conventional

HUD  

Without

TDZ/CL Lights

SVS HUD

With

TDZ/CL

Lights

SVS HUD

Without

TDZ/CL

Lights

SVS HUD

With

TDZ/CL

Lights

SVS HUD

Without

TDZ/CL

Lights

SVS HDD

With

TDZ/CL

Lights

SVS HDD

Without

TDZ/CL

Lights

SVS HDD

With

TDZ/CL

Lights

SVS HDD

Without

TDZ/CL

Lights

1400 ft RVR 1400 ft

RVR

1400 ft

RVR

1000 ft

RVR

1000 ft

RVR

1800 ft

RVR

1800 ft

RVR

1400 ft

RVR  

1400 ft

RVR  

No. Go‐Around/

No. Runs 1/12 2/12 2/12 3/12 4/12 1/12 0/12 3/12 1/12

Sink Rate at 100 ft HAT

(ft/min)

Mean ‐683.6  ‐753.9  ‐711.6  ‐782.2  ‐675.1  ‐743.4  ‐647.3  ‐721.5  ‐749.4

Std

Dev

85.4 93.1 79.0 28.5 113.7 218.5 118.7 232.3 182.2

Max    ‐815.6  ‐952.7  ‐806.4  ‐829.0  ‐869.5  ‐1168.2  ‐871.6  ‐1159.3 ‐1046.0

Sink Rate at Threshold

Crossing (ft/min)

Mean ‐576.7  ‐562.0  ‐632.9  ‐676.3  ‐545.1  ‐600.0  ‐673.2  ‐535.9  ‐555.1

Std

Dev

124.1 188.1 121.5 112.7 249.9 253.0 165.6 190.7 297.7

Max ‐735.8  ‐764.9  ‐829.6  ‐871.6  ‐788.3  ‐896.1  ‐915.2  ‐793.3  ‐1014.6

ANOVA analyses indicate no significant differences (p>0.05) for lateral deviation from centerline, 

vertical deviation from path, or sink rate among the nine SVS operational concepts tested at both the 

100 ft HAT and Threshold Crossing Locations. 
56 

Table 27. Bank Angle Statistics of SVS Operational Concepts 

Conventi

onal HUD

Without

TDZ/CL

Lights

SVS HUD

With

TDZ/CL

Lights

SVS HUD

Without

TDZ/CL

Lights

SVS HUD

With

TDZ/CL

Lights

SVS HUD

Without

TDZ/CL

Lights

SVS HDD

With

TDZ/CL

Lights

SVS HDD

Without

TDZ/CL

Lights

SVS HDD

With

TDZ/CL

Lights

SVS HDD

Without

TDZ/CL

Lights

1400 ft

RVR

1400 ft

RVR

1400 ft

RVR

1000 ft

RVR

1000 ft

RVR

1800 ft

RVR

1800 ft

RVR

1400 ft

RVR  

1400 ft

RVR  

Max Bank Angle from

Threshold Crossing to

Touchdown (deg)Mean 3.9 3.5 3.7 3.0 3.1 4.5 4.8 5.2 3.8

Std

Dev

2.8 3.1 3.1 1.9 3.5 2.6 3.0 2.9 2.6

Max    7.1 10.0 9.8 6.3 11.2 9.0 10.4 11.0 7.3

Bank Angle at

Touchdown (deg)

Mean 2.4 2.6 2.6 1.4 2.7 2.6 2.4 3.2 2.0

Std

Dev

2.8 3.2 2.7 0.9 3.7 2.6 2.2 2.4 2.7

Max 7.2 10.1 7.8 2.8 11.1 9.0 8.2 7.7 7.5

ANOVA analyses indicate no significant differences (p>0.05) among the nine SVS operational 

concepts for the maximum bank angle from threshold to touchdown or for the bank angle at 

touchdown measures. 

4.3.3 Landing Performance 

The nine concepts used for SVS operational concept comparisons for landing performance were: 

1) Conventional HUD without TDZ/CL lights (1400 ft RVR/150 ft DH) 

2) SVS HUD with TDZ/CL lights (1400 ft RVR/150 ft DH) 

3) SVS HUD without TDZ/CL lights (1400 ft RVR/150 ft DH) 

4) SVS HUD with TDZ/CL lights (1000 ft RVR/150 ft DH) 

5) SVS HUD without TDZ/CL lights (1000 ft RVR/150 ft DH) 

6) SVS HDD with TDZ/CL lights (1800 ft RVR/150 ft DH) 

7) SVS HDD without TDZ/CL lights (1800 ft RVR/150 ft DH) 

8) SVS HDD with TDZ/CL lights and (1400 ft RVR/150 ft DH) 

9) SVS HDD without TDZ/CL lights (1400 ft RVR/150 ft DH) 
57 

The Conventional HUD flown in 1400 ft RVR to a 150 ft DH and a runway without TDZ/CL lights 

was considered the operational baseline concept for the SVS operational concept comparisons. 

Landing Statistics 

In Table 28, the T/D statistics (mean, standard deviation, minimum value, and maximum value) are 

shown, broken out by RVR and TDZ/CL light configuration, for the nine SVS operational concepts. 

The T/D measures (lateral position, longitudinal position and sink rate) means were used to determine 

which touchdown performance rating level (Desired, Adequate, or Not Adequate) as defined in Table 

6 was achieved for the SVS operational concepts. Also provided in Table 28 are the number of runs 

that resulted in a go-around and the total number of runs for the SVS operational concepts tested. 

Note that for touchdown lateral position statistics in Table 28, the “min” value equates to the 

maximum deviation to the left of centerline and the “max” value equates to the maximum deviation to 

the right of centerline. 

Table 28. Landing Statistics for SVS Operational Concepts 

Conventional

HUD Without

TDZ/CL Lights

SVS  

HUD

With

TDZ/CL

Lights

SVS

HUD

Without

TDZ/CL

Lights

SVS

HUD

With

TDZ/CL

Lights

SVS

HUD

Without

TDZ/CL

Lights

SVS

HDD

With

TDZ/CL

Lights

SVS  

HDD

Without

TDZ/CL

Lights

SVS  

HDD

With

TDZ/CL

Lights

SVS

HDD

Without

TDZ/CL

Lights

1400 ft  

RVR  

1400 ft

RVR  

1400 ft

RVR  

1000 ft

RVR  

1000 ft

RVR  

1800 ft

RVR  

1800 ft

RVR  

1400 ft

RVR  

1400 ft

RVR  

No.Go‐Around

/ No. Runs 1/12 2/12 2/12 3/12 4/12 1/12 0/12 3/12 1/12 T/D Longitudinal Position (ft)

Mean 639.1   1183.8   840.9   770.8   742.0   944.0   835.1   1003.3   685.0  

Std

Dev 275.9   450.1   283.8   157.7   496.6   546.1   397.2   328.3   389.8  

Min 369.7   605.9   457.7   462.5   229.9   ‐50.0   341.1   452.5   26.8  

Max 1243.6   2022.1   1330.3   948.9   1814.2   2146.5   1711.7   1477.8   1244.0  

Rating Adequate   Desired   Desired   Desired   Adequate   Desired   Desired   Desired   Adequate  

T/D Lateral Position

(ft)

Mean 0.5   2.0   ‐3.3   5.4   ‐3.3   5.2   ‐3.5   ‐12.5   2.5  

Std

Dev 13.8   21.0   16.6   11.6   18.2   11.0   19.1   21.6   21.8  

Min  ‐14.1   ‐27.9   ‐27.1   ‐15.2   ‐32.4   ‐16.6  ‐33.4   ‐52.4   ‐33.5  

Max 29.5   43.7   19.1   20.3   19.2   46.9 26.8   18.9   50.7  

Rating Desired   Desired   Desired   Desired   Desired   Desired   Desired   Desired   Desired  

T/D Sink Rate

(fps)

Mean  ‐8.2   ‐7.4   ‐7.2   ‐6.4   ‐6.8   ‐6.8   ‐7.0   ‐6.6   ‐6.9  

Std

Dev 1.8   2.1   4.2   2.0   2.4   3.1   3.2   3.7   2.8  

Min  ‐5.7   ‐3.9   ‐0.6   ‐4.3   ‐3.4   ‐2.6   ‐1.8   ‐0.7   ‐2.4  

Max  ‐10.6   ‐10.7   ‐13.5   ‐9.9   ‐10.3   ‐11.4   ‐12.5   ‐11.6   ‐11.7  

Rating Adequate   Adequate   Adequate   Adequate   Adequate   Adequate Adequate   Adequate   Adequate  

The data show that: 

 The absence or presence of TDZ/CL lights appears to not have influenced the PF’s decision 

to land or go-around while using the SVS HUD. 
58 

 All HUD (Conventional and SVS) operational concept landings were within the touchdown 

box defined in AC-120-28D [34] and JAR AWO [32] (i.e., within 58 ft lateral distance of 

centerline and between 200 ft to 2700 ft longitudinal distance from threshold). 

 All SVS operational concept comparison runs met the lateral touchdown criteria (within 57 ft 

of centerline) 

 Unbeknownst to them, one crew landed 50 feet SHORT of the runway while flying with the 

SVS HDD display concept with TDZ/CL lights in the 1800 ft RVR condition. 

 One SVS HDD without TDZ/CL lights in the 1400 ft RVR condition run (one out of 11 

landings) did not meet the longitudinal touchdown criteria (between 200 to 2700 ft from 

threshold). The other 10 landings for this operational concept met the criteria. 

 On average, all SVS operational concept comparison runs had higher than expected 

touchdown sink rates with only “Adequate” sink rate performance criteria (between six to 10 

feet/second) being met as defined in Table 6. 

ANOVA Analyses 

ANOVAs revealed no significant differences (p>0.05) for touchdown longitudinal position 

(mean=847 ft, =404 ft), lateral position (mean=-0.61 ft, =18.4 ft), or sink rate (mean=-7.1 ft/sec, 

=2.8 ft/sec) for the nine SVS operational concepts tested. 

Objective Landing Standards Analysis 

In Figure 15, the touchdown data are shown for the eight SVS display concepts and the SVS 

operational baseline concept. Included on this plot is the touchdown aim point (0 ft lateral distance, 

1000 ft longitudinal distance) indicated by the axes origin and the landing box (laterally within 58 ft 

of centerline and longitudinally between 200 to 2700 ft from threshold) indicated by the dashed 

rectangle. In Figure 16, the pitch angle and bank angle at touchdown are shown. Included on this 

plot are the normal landing ground contact angles for the simulated aircraft indicated by the solidlined contour on the plot. 
Figure 16. 

Figure 15. T

Touchdown pi

ouchdown Foo

tch angle and b

59 

tprint for SVS

ank angle foo

 Operational c

tprint for SVS

oncepts 

 operational concepts 
60 

Visual inspection of the data shows that 1) all HUD (SVS and Conventional) display concepts were 

within the landing box defined in AC-120-28D and JAR AWO (i.e., within 58 ft lateral distance of 

centerline and between 200 ft to 2700 ft longitudinal distance from threshold); 2) three SVS HDD 

concept landings were outside the landing box, with one (of the three) landing short of the runway 3) 

there were no wing tip or tail strikes, and 4) all SVS Operational concepts, including baseline, had 

higher than expected touchdown sink rates. 

None of the SVS operational concepts tested met the 10-6 probability of occurrence auto-land criteria 

for T/D longitudinal position, lateral position from centerline, or sink rate as per AC120-28D [34, 

Appendix 3]. 

4.3.4 Rollout Performance 

Rollout statistics (RMS lateral deviation from centerline and maximum lateral deviation from 

centerline) for the nine SVS operational concepts tested are provided in Table 29. 

Separate ANOVAs revealed no significant differences (p>0.05) in PF RMS lateral deviation from 

centerline (mean= 8.6 ft, =4.4) or maximum lateral deviation from centerline (mean= 19.8 ft, 

=10.0) during rollout operations among the nine SVS operational concepts tested. 

Table 29. Rollout Statistics for SVS Operational Concepts 

Conventional

HUD

Without

TDZ/CL Lights

SVS HUD

With

TDZ/CL

Lights

SVS

HUD

Without

TDZ/CL

Lights

SVS

HUD

With

TDZ/CL

Lights

SVS

HUD

Without

TDZ/CL

Lights

SVS

HDD

With

TDZ/CL

Lights

SVS  

HDD

Without

TDZ/CL

Lights

SVS  

HDD

With

TDZ/CL

Lights

SVS

HDD

Without

TDZ/CL

Lights

1400 ft  

RVR  

1400 ft

RVR  

1400 ft

RVR  

1000 ft

RVR  

1000 ft

RVR  

1800 ft

RVR  

1800 ft

RVR  

1400 ft

RVR  

1400 ft

RVR  

RMS Lat Dist

from C/L (ft)

Mean 8.2 8.4 8.8 8.2 9.4 8.7 8.6 8.6 8.8

Std

Dev 4.1 5.5 4.3 3.8 4.1 4.7 3.4 6.1 4.0

Max Lat Dist

from C/L (ft)

Mean 18.1 19.6 20.2 15.8 18.5 20.2 21.5 22.6 20.9

Std

Dev 9.3 10.5 7.6 6.9 8.5 11.0 8.0 15.4 12.5

4.3.5 Pilot Workload 

Separate ANOVAs revealed no significant differences (p>0.05) in PF post-run workload ratings 

(mean=3.3) or PM post-run workload ratings (mean=2.7) among the nine SVS operational concepts 

tested. 

Crews rated their workload as being “moderate, easily managed and having considerable spare time” 

while using the SVS HUD concepts on approach through landing in visibilities as low as 1000 ft 

RVR and SVS HDD concepts in visibilities as low as 1400 ft RVR. 
4.3.6 E

Eye tr

Head 

SVS o

transi

findin

ANOV

indica

flight 

percen

Simila

appro

condi

Conv

Tukey

presen

signif

Eye Tracking

racking analy

up percent an

operational co

tion height ab

ngs are circled

VAs perform

ated significan

segments of 

ntage across c

ar behavior to

oach, revealin

tions (Table 3

entional HDD

y HSD pair-w

nce of SVS o

ficant effects w

ysis for the SV

nd transition c

oncepts utiliz

bove threshol

d in red in Fig

med to assess v

nt variance in

the approach 

conditions du

o EFVS opera

g approximat

30). Howeve

D condition (c

wise comparis

n the HUD co

were observe

Figure 1

VS operationa

count from in

ed both HUD

d and gaze di

gures 17 and 

visual behavio

n head up time

(Figure 17).

uring the flare

ational concep

tely 23% head

er, these result

compare Figu

son tests of he

ontrasted to th

ed due to RVR

17. SVS Op C

61 

al concepts ar

nstruments to 

D and HDD di

irection was n

18. 

or differences

e between HU

There were n

e and landing 

pts was obser

d down time d

ts indicate an

ure 13 and Fig

ead up percen

he Conventio

R or TDZ/CL

Concepts, He

re broken into

OTW by seg

isplay locatio

not performed

s across the S

UD and HDD

no significant 

segments. 

rved during th

during the vis

n increase of 1

gure 17) durin

ntage revealed

onal HUD con

L lighting cond

ead Up Time 

o two main qu

gment of appr

ons; therefore,

d. Statisticall

SVS operation

D display conc

differences in

he in-flight se

sual segment 

10% head up 

ng the visual s

d no significan

ndition across

ditions. 

(%) 

uantitative me

oach and land

, analysis of O

ly significant 

nal concepts 

cepts during a

n head up 

egments of the

with SVS HD

time over the

segment. 

nt effects due

s all segments

etrics; 

ding. 

OTW 

all ine

DD 

e

e to 

s. No 
62 

Table 30. SVS Op Concepts, Head Up Time (%) 

Conventional

HUD SVS HUD SVS HUD SVS HDD SVS HDD ANOVA

1400 RVR/

150 DH

1400 RVR/

150 DH

1000 RVR/

150 DH

1800 RVR/

150 DH

1400 RVR/

150 DH

Instrument

Segment

Mean 95.2 94.1 94.1 18.6 24.1 Display/RVR Condition

Std

Dev 4.3 8.3 7.1 15.7 20.4 F(4,92) = 184.75,  p < 0.000

Transition

Segment

Mean 98.9 93.0 94.9 43.2 42.9 Display/RVR Condition

Std

Dev 1.3 11.2 7.1 19.2 25.3 F(4,91) = 60.70,  p < 0.000

Visual

Segment

Mean 99.5 93.0 94.3 77.5 75.9 Display/RVR Condition

Std

Dev 0.9 14.4 13.9 24.8 24.2 F(4,89) = 6.01,  p < 0.000

Flare

Segment

Mean 99.2 91.8 98.8 94.8 98.2 No Significance

Std

Dev 1.4 17.2 2.4 15.2 4.0

Landing

Segment

Mean 74.3 85.3 85.8 89.7 88.9 No Significance

Std

Dev 16.8 8.8 13.8 9.5 4.0

Analysis of head up/head down transitions indicated a significant difference across conditions during 

all in flight segments of the approach to land with the exception of the flare segment. Tukey HSD 

post hoc tests revealed SVS HDD conditions had significantly higher numbers of transitions than 

SVS and Conventional HUD conditions (Figure 31). Analysis showed no significant effect across 

display conditions during the flare segment, averaging nearly zero transitions, indicating pilot 

attention is strictly OTW, further confirmed with results of the head up time analysis (Figure 17). 

ANOVA results of the landing segment transitions showed significant differences across display 

conditions. Tukey HSD post hoc tests indicate these differences are due not to the display location of 

HUD vs. HDD, but the OTW RVR. RVR levels at 1000 ft significantly reduced the number of 

transitions between head up and head down. Reasoning behind pilot visual transitions during the 

landing rollout segment is the information available inside the flight deck, relevant to ownship and 

runway exit ramp information being available on the moving map display. This additional 

information not available on the HUD encourages a division of pilot attention between flight deck 

HDDs and OTW. Data suggests this divided attention is increasingly limited at the lower 1000 RVR 

level with a reduction in visual transitions to the moving map display. This is reasonably due to 

decreased range in OTW visibility that reduces the time to visually acquire any dynamic changes in 

the OTW scene. With the reduction in visual transitions at 1000 RVR, pilots maintained a relatively 

equal (within 10%) amount of time head up as with all other conditions, with the least amount of time 

head up shown with the conventional HUD condition (Figure 17). Although not statistically 

significant, these results suggest the additional information available through the use of SVS aided 

pilots in maintaining attention OTW during the landing segment, even when divided attention was 

limited at lower RVR levels (Figure 18). Table 31 shows specific average transition values across all 

tested conditions. 
Figure 18. SVS Op Con

63 

cepts, Head Up/Down Transitions 
64 

Table 31. SVS Op Concepts, Head Up/Down Transitions 

  

Conventional

HUD SVS HUD SVS HUD   SVS HDD   SVS HDD   ANOVA

1400 RVR/

150 DH  

1400 RVR/

150 DH  

1000 RVR/

150 DH  

1800 RVR/

150 DH  

1400 RVR/

150 DH     

Instrument

Segment

Mean   3.2 3.2 4.3 9.1 10.3 Display/RVR

Condition

F(4,92) = 5.78,  

p < 0.000 Std Dev 3.4 3.6 6.3 6.2 9.6

Transition

Segment

Mean   0.3 0.5 0.5 5.1 3.7 Display/RVR

Condition

F(4,91) = 31.40,  

p < 0.000 Std Dev 0.7 0.8 0.9 2.7 2.4

Visual

Segment

Mean   0.1 0.1 0.1 2.1 2.0 Display/RVR

Condition

F(4,91) = 18.78,  

p < 0.000 Std Dev 0.3 0.3 0.4 1.1 1.8

Flare

Segment

Mean   0.1 0.6 0.2 0.2 0.4

No Significance.  

Std Dev 0.3 0.5 0.5 0.5 0.7

Landing

Segment

Mean   5.8 3.6 2.0 5.4 4.4 Display/RVR

Condition

F(4,91) = 4.08,  

p < 0.004 Std Dev   3.6 3.3 2.6 3.6 3.6

4.4 Crew Display Concept Preferences 

The PF and PM were independently asked to rank order the display concepts (Conventional HDD, 

SVS HDD, Conventional HUD, SVS HUD, and EFVS) from most preferred display (rank=1) to least 

preferred display (rank=5) for flying with in low-visibility conditions. Both the PF and PM pilots 

ranked their low-visibility operations display concept preferences as follows: 1) EFVS (most 

preferred), 2) SV HUD, 3) Conventional HUD, 4) SVS HDD, and 5) Conventional HDD (least 

preferred). 

Friedman’s test statistic showed there were significant differences among the displays for both the PF 

(2

(4)=36.145, p<0.001) and PM (2 (4)=41.818, p<0.001) preference rankings. Wilcoxon tests were 

used to follow up this finding. A Bonferroni correction was applied and all effects are reported at a 

0.01 level of significance. 

In Figure 19, the inverse mean rankings for the PF and PM display concept preferences are shown. 

By using the inverse of the preference rankings, the most preferred display would have a value of 1 

and the least preferred display would have a value of 0.2 (i.e., 1/5=0.2) in Figure 19. 
65 

The PF preferences were significantly better for: 1) EFVS over all other concepts, 2) SVS HUD than 

SVS HDD, and 3) SVS HDD than Conventional HDD. There were no significant PF preference 

differences between: 1) SVS HUD and Conventional HUD or 2) Conventional HUD and SVS HDD. 

The PM preferences were significantly better for: 1) EFVS over all other concepts, 2) HUD concepts 

than HDD concepts, 3) SVS HDD than Conventional HDD. There were no significant PM preference 

differences between SVS HUD and Conventional HUD. 

Figure 19. PF and PM Display Concept Preference Rankings 

4.5 CDTI and SEVS Influences on Traffic Awareness 

Traffic awareness was measured using several methods, both objective and subjective, to evaluate the 

influence of SEVS, CDTI, and their interaction. 

4.5.1 Traffic Awareness Probe 

A traffic awareness probe, modeled after a SAGAT test, was administered (see Section 3.12 for 

ConventionalHDD

ConventionalHDD

SVS HDD

SVS HDD

ConventionalHUD

ConventionalHUD

SVS HUD

SVS HUD

EFVS

EFVS

0 0.2 0.4 0.6 0.8 1

PMInverse Mean

DisplayPreference Rankings

PF Inverse Mean

DisplayPreference Rankings

MostPreferred

Concept

LeastPreferred

Concept
66 

experimental method and protocol used for this measure). The data quantifies the flight crew’s 

awareness (PF and PM) of traffic and the influence of CDTI. 

The data from the traffic probe are shown in Table 32 and Table 33. In Table 32, the data from the 

first time that the pilots responded to the probe are given. In Table 33, the data from all other queries 

are given. The first and remaining probes are separated because the flight crews were not aware that 

this probe was going to be conducted. Therefore, the first occurrence indicates unexpected behavior 

whereas the other data reflects modified behavior, albeit the probes were not very frequent. 

The data are presented for crew position (PF or PM) and whether CDTI was present or absent. The 

CDTI condition is collapsed across the two different CDTI types (i.e., the moving map with ground 

traffic and moving map with runway inset of ground traffic are treated as not being significantly 

different). 

Table 32. Statistics from First Occurrence of Traffic Probe

    Number

of

Responses

Total

Number

of

Traffic

%

Responses

Min.

Responses

per Run

Max.

Responses

per Run

% Correct

(Of

Responses)

PF No CDTI 3 out of 60 5 0 2 100

CDTI 11 out of 84 13 0 4 91

PM No CDTI 7 out of 60 12 0 4 100

CDTI 12 out of 84 14 0 3 100

Table 33. Statistics from All Other Occurrences of Traffic Probe

    Number

of

Responses

Total

Number

of

Traffic

%

Responses

Min.

Responses

per Run

Max.

Responses

per Run

% Correct

(Of

Responses)

PF No CDTI 34 out of 204 17 0 5 100

CDTI 74 out of 432 17 0 5 93

PM No CDTI 30 out of 204 15 0 4 93

CDTI 147 out of 432 34 0 11 95

These data were not tested for statistically significant differences due to the lack of sufficient 

statistical power. The statistics in Tables 32 and 33 do show several trends, as noted in the following: 

 The percent of responses that were correct when they answered was very nearly 100% for all 

pilots, on all occasions. This indicates that the pilots were not guessing in response to the 

probe. 

 The “first occurrence” data showed that, nominally, pilots were not aware of traffic around 

the active runway. The data show that only 5 to 14% of the traffic was noticed by the pilots 

in the initial administration of the traffic probe. In many cases, the pilots had no awareness of 

the traffic (i.e., as indicated by the “min. responses per run” data, on at least one run within 

each category, the PF or PM could not recall the presence or location of any traffic around the 

active runway). 
67 

 The data showed that when pilots had expectations that a traffic probe was being 

administered (Table 33) CDTI can aid the PM for traffic awareness. The PM responses with 

CDTI show that the PM was twice as aware of the traffic when CDTI was available 

compared to when it was not. On the other hand, the presence of CDTI did not appear to 

increase the PF’s awareness of traffic. Because this result occurs with “expectation,” it 

implies that pilots must be trained in the use of CDTI. This training should emphasize to the 

pilots to include the CDTI information found on their displays in their visual scan for traffic 

there and OTW. 

 Unfortunately, even with expectation and CDTI, the PMs were only aware of one-third of the 

traffic on average around the active runway and on at least one run, the PM could not recall 

the presence or location of any traffic around the active runway. 

4.5.2 Paired Comparisons for Situation Awareness and Traffic Awareness 

Post-test SA was assessed using the SA-SWORD paired-comparison technique which provided 

relative SA ratings across the three CDTI formats tested. For these comparisons, SA was defined as 

“The pilot’s awareness and understanding of all factors that will contribute to the safe flying of their 

aircraft under normal and non-normal conditions.” The PF and PM independently completed the 

SA-SWORD technique for the CDTI formats tested. 

Similarly, a post-test paired-comparison technique was used to assess traffic awareness for the CDTI 

formats tested. For these comparisons, traffic awareness was defined as “The pilot’s awareness and 

understanding of significant traffic that will affect his/her aircraft under normal and non-normal 

operating conditions.” The PF and PM independently completed the traffic awareness paired 

comparison technique. 

Situation Awareness 

Independent ANOVA analyses revealed the presentation of CDTI was significant for PF SA-SWORD 

ratings (F(2,30)=216.273, p<0.0001) and PM SA-SWORD ratings (F(2,30)=227.619, p<0.0001). 

Post-hoc tests (SNK using =0.05) showed the same three unique subsets for the PF and PM SASWORD ratings: 1) No CDTI (lowest SA), 2) CDTI on Moving Map, and 3) CDTI on Moving Map 

and Runway Inset (highest SA). 

Traffic Awareness 

Independent ANOVA analyses revealed the presentation of CDTI was significant for PF traffic 

awareness ratings (F(2,30)=44.740, p<0.0001) and PM traffic awareness ratings (F(2,30)=5819.857, 

p<0.0001). Post-hoc tests (SNK using =0.05) showed the same three unique subsets for the PF and 

PM traffic awareness ratings: 1) No CDTI (lowest traffic awareness), 2) CDTI on Moving Map, and 

3) CDTI on Moving Map and Runway Inset (highest traffic awareness). 

4.5.3 Unexpected Runway Incursion 

Non-normal runs were injected into the test unbeknownst to the PF and PM. Of the non-normal runs 

one was an unexpected runway incursion scenario, followed by nine runway incursion scenarios 

which were not identified to the flight crew. However, after experiencing the first unexpected runway 

incursion, the crew may have expected another type of incursion or non-normal scenario. These 

incursion scenarios provide an operationally oriented quantification of traffic awareness using SEVS 
68 

and CDTI. 

The unexpected runway incursion scenario was flown once by each crew. This run was always the 

30th trial out of 39 trials flown by each crew. 

The runway incursion run was always flown to Runway 22R without TDZ/CL lights in a 10 knot 

headwind and 1400 ft RVR OTW test condition with one of four display configurations 

1) EFVS without CDTI, 

2) EFVS with CDTI on moving map and runway inset, 

3) SVS HDD without CDTI, or 

4) SVS HDD with CDTI on moving map and runway inset. 

The PF could identify the incurring aircraft OTW, with the FLIR on HUD (for EFVS runs), or with 

CDTI (if available HDD). The PM could identify the incurring aircraft OTW, with FLIR on HDD 

(for EFVS runs), or with CDTI (if available HDD). In Figure 20, the FLIR signature from the 

incurring vehicle is shown after the aircraft has turned onto the runway. 

Figure 20. Runway Incursion aircraft as seen on FLIR on PM head-down display. 

Table 34 shows the four display configurations used for the unexpected runway incursion, if the 

incursion was detected, the detection altitude, and PF eye location when the incursion was detected. 

Also included in Table 34 are event description data gathered from video review of the unexpected 

runway incursion runs. 
69 

Table 34. Unexpected Runway Incursion Run Data. 

Display

Concept

CDTI

Incursion

Detected

AGL Altitude

when

Incursion

Detected (ft)

PF Eye

Location when

Incursion

Detection

Event Description

EFVS Moving Map +

Runway Inset yes 26 OTW PF called go‐around at 26 ft AGL

EFVS Moving Map +

Runway Inset yes 230 OTW

Using CDTI on runway inset, PM noted another

aircraft approaching runway at ~340 ft AGL.  PM

called “go around” at 230 ft.  

EFVS Moving Map +

Runway Inset no N/A No data Neither pilot saw incurring vehicle until after they

landed.  (No eye tracking data available.)  

EFVS None yes 4 OTW

PF noticed aircraft at 4 ft AGL and called go‐

around.  PM had head in cockpit and didn’t see

incursion.  PF felt like he was looking through

incurring aircraft on the runway.

EFVS None yes 22 No data

PM called go‐around at about 22 ft AGL.  PF saw

incurring aircraft about the same time. (No eye

tracking data available.)  

EFVS None yes 40 OTW

PF saw engines and hit TOGA button.  No audible

recognition of incursion prior to TOGA press.  PM

didn’t see incursion.  

SVS PFD Moving Map +

Runway Inset yes 8 OTW

Neither pilot had moving map or runway inset in

visual scan. PM called traffic on runway/go‐around

at 8 ft AGL  

SVS PFD Moving Map +

Runway Inset yes 32 OTW

PM called go‐around. Noted on CDTI at first, then

looked away.  Finally saw on moving map display.

SVS PFD Moving Map +

Runway Inset yes 240 Head‐Down

PM commented on traffic going onto runway at

270 ft AGL and called go‐around at 240 ft AGL

SVS PFD None yes 0 OTW PM called for go‐around at 0 ft.

SVS PFD None yes 2 OTW

PF saw just before landing at 2 ft AGL and called

go‐around.  

SVS PFD None yes 4 No data

PM saw incurring aircraft just prior to touchdown.

Go‐around called at 4 ft AGL.  (No eye tracking

data available.)  

The data show that: 

 CDTI was beneficial in only two of six runs (when the incurring vehicle was recognized by 

the crew above 200 ft AGL). 

 For one of the three EFVS runs with CDTI, the crew did not notice the runway incursion until 

landing on it. 

 For nine of 12 runs, crews did not notice incurring aircraft until under 40 ft AGL, with five of 
70 

them performing a go-around under 20 ft AGL. 

4.5.4 Expected Runway Incursions 

The “expected” runway incursion runs were always given to the pilots in the second phase of 

experimental testing, immediately following the unexpected runway incursion. These runs were 

flown to runways without TDZ/CL lights (4R, 22L, and 22R) using the EFVS with one of three HDD 

CDTI configurations (none, Moving Map, or Moving Map and Runway Inset). As was expected, the 

crews were more tuned to potential runway incursion events after being exposed to the unexpected 

runway incursion event. As such, all crews noticed the expected runway incursions and performed a 

go-around. However, one crew did not notice the three expected runway incursions until under 30 ft 

AGL, with two of these three runs under 20 ft AGL. On average, the other crews noticed the 

expected runway incursions and performed a go-around at 186 ft AGL (with standard deviation of 61 

ft). 

There were no significant (p>0.05) CDTI effects on incursion detection time from beginning of run 

(mean =75 sec, =6.6 sec) or detection altitude (mean=71 ft, =75 ft) for the expected runway 

incursion runs. The mean AGL altitude when incursion was detected was slightly higher (i.e., earlier 

detection) with the Moving Map and Runway Inset CDTI format, however, it was not significantly 

different than Moving Map or No CDTI formats. 
71 

5 Operational Credit Discussion of Results 

The results of a pilot-in-the-loop simulation of SEVS technologies were presented using an 

operational comparative baseline in the preceding section. 

In the following, these results are discussed, again, using an operational comparative baseline, 

focusing on performance and safety to identify if consideration for operation credit is warranted in the 

following conditions: 

 To enable descent below the published DA/DH on a straight-in instrument approach 

procedure, landing, and roll-out in visibilities as low as 1000 ft RVR by use of an EFVS 

without natural visibility references 

 To enable descent to a DA/DH as low as 150 ft HAT on an ILS approach by use of SVS 

5.1 EFVS Operational Credit 

The EFVS simulation results, in terms of performance and safety, are discussed with respect to the 

following five items to identify if consideration for operation credit is warranted for EFVS. 

1. Did EFVS operations affect the existing instrument segment? 

For consideration of operational credit, the use of EFVS should “do no harm” in the 

instrument segment. 

2. Did EFVS provide equivalent levels of safety and performance in the visual segment to the 

operational comparative baseline?

For consideration of operational credit, the use of EFVS should provide equivalent, if not 

better, levels of performance and safety in the visual segment (from DH to 100 ft) as the 

operational comparative baseline. This comparison is identical to the previous certification 

basis used for EFVS under the existing § 91.175 (l) and (m) rule. 

3. Could pilots safely land and roll-out using EFVS?

For consideration of operational credit, the use of EFVS should provide equivalent, if not 

better, levels of performance and safety in the flare, landing, and roll-out as the operational 

comparative baseline. This comparison expands the previous certification basis under § 

91.175 (l) and (m) rule and if successful, essentially eliminates the requirement for natural 

vision to conduct approach, landing, and roll-out. 

4. Any effect of clutter / obscuration using EFVS?

The combination of what the pilot can see in the FLIR image and HUD, and what can be seen 

through and around the HUD, must be as safe and effective as the view without the EFVS 

image and HUD. This element primarily, but not exclusively, considers the influence of 

HUD clutter and potential obscuration of the PF’s view out of the window. 

5. Influence of EFVS design factors and off-nominal operations 

Off-nominal performance and non-normal scenarios should not create unacceptable or unsafe 

situations or conditions. Not all possible off-nominal situations were simulated in the test, 

but a few pertinent ones were tested and are reviewed as they may possibly influence the 

acceptability and utility of EFVS and thus, consideration for operation credit. 
72 

The discussion collapses the comparison between the operational baseline (traditional head-down 

Primary Flight Display without synthetic or enhanced vision) flown in 1800 ft RVR to a 200 ft DH 

against a EFVS HUD evaluated in 1400 ft or 1000 ft RVR conditions (i.e., collapsed across the 1400 

ft and 1000 ft RVR and runway lighting conditions). 

5.1.1 EFVS Instrument Segment 

In accordance with accepted FAA approach standards (FAA-S-8081-4D and AC120-29), the 

glideslope tracking (~90 to 95% of runs meeting the standards) and localizer tracking (100% of runs 

meeting the standards) for the baseline and EFVS HUD conditions were equivalent. Using RMS 

deviation performance statistics in the instrument segment (from 1000 ft to DH), the EFVS HUD 

condition provided better glideslope tracking and lower sink rate deviation over the operational 

baseline. 

Equivalent and improved performance for the EFVS HUD was felt to be due to two considerations. 

First, the HUD gives a conformal presentation of flight path and guidance which improves flight 

technical performance in comparison to a minified PFD. Even though the symbology and guidance 

algorithms are the same, the increased sensitivity of the HUD symbolic scaling improves the pilot’s 

ability to track the flight path and maintain a stabilized instrument approach. Secondly, as the aircraft 

descends toward the DH, a pilot using a head-down display tends to look out the forward windscreen 

for the emergence of visual flight references. Although technically in the instrument segment, this 

natural behavior detracts from the pilot’s full attention to flight technical performance. As one pilot 

said, “look up, fly up” noting a natural tendency of a pilot to shallow their flight path when initially 

transitioning to head-up flight conditions. Performance also suffers due to the time spent in visual 

accommodation transitioning between the OTW and head-down instruments. In comparison, the 

EFVS HUD allows nearly simultaneous attention to flight technical performance as well as 

emergence of visual flight references, inherent to the HUD. These performance improvements are 

especially notable since, in EFVS HUD case, worse performance might be expected in sink rate and 

glideslope tracking performance because the EFVS used a lower DH. With a lower DH, the 

increased sensitivity of glideslope should make glideslope tracking more difficult and the emergence 

of enhanced visual references should divide the attention of the pilot-flying. 

For consideration of operational credit, the use of EFVS should “do no harm” in the instrument 

segment. These results show that EFVS clearly does no harm, and in fact, improves the ability of the 

pilot to fly in the instrument segment compared to the operational baseline. 

5.1.2 EFVS Visual Segment 

Flight technical performance in the visual segment is best quantified by the JAR-AWO results, 

identifying the probability of success in the instrument to visual segment. Glideslope tracking must 

be within 1 dot and localizer tracking within 1/3 dot from 300 ft to 100 ft HAT. The results show that 

localizer tracking for the EFVS operational baseline condition (Conventional HDD) and EFVS HUD 

conditions were equivalent. However, glideslope tracking was much better with the EFVS HUD 

conditions than the operational baseline. The probability of success was 95% to 97% for the 

HUD/EFVS condition, but only 29% for the Conventional HDD condition. This performance is also 

reflected in the two go-arounds being conducted by pilots using the Conventional HDD condition 

whereas none of the EFVS HUD approaches ended in a go-around. 

Another way of quantifying FTE within the visual segment is to evaluate how well the aircraft was in 

position to land at 100 ft and upon crossing the threshold. In Figure 21, the aircraft position upon 
73 

reaching 100 ft HAT is plotted for both the EFVS HUD and Conventional HDD conditions. This 

figure shows the vertical and lateral deviation of the aircraft from the three degree descent path to the 

glidepath intercept point. The figure illustrates the superior vertical positioning when flying the 

EFVS HUD. The vertical deviation is much less, indicating that the aircraft is more closely aligned 

with the glidepath to landing. Lateral deviation of the EFVS HUD configurations is less than the 

operational baseline condition, but not significantly so. 

Figure 21. Aircraft Position at 100 ft HAT, Conventional HDD vs. EFVS 

In Figure 22, the aircraft position upon crossing the threshold is plotted for both the EFVS HUD and 

Conventional HDD conditions. This figure shows lateral deviation of the aircraft from the runway 

centerline and the altitude deviation from a 50 ft threshold crossing height. Again, the figure 

illustrates the superior vertical positioning when flying the EFVS HUD. The vertical deviation is 

much less, indicating that the aircraft is in a much better position to land within the touchdown zone. 

Also, performance is much more repeatable. Lateral deviation of the EFVS HUD configurations is 

less than the operational baseline condition, but not significantly so. 

-100 -80 -60 -40 -20 0 20 40 60 80 100

-100

-80

-60

-40

-20

0

20

40

60

80

100

() p

 Vertical Deviation from Path (ft)

 Lateral Deviation from Path (ft)

Base (HDD)

EFVS
74 

Figure 22. Aircraft Position at the Threshold, Conventional HDD vs. EFVS 

The superior performance with the EFVS HUD condition can largely be traced to the display type 

difference. Although technically in the visual segment, the eye gaze tracking data showed 

statistically significant differences between the Conventional HDD and EFVS HUD conditions. 

Whereas almost no head-up/head-down visual transitions were made by pilot’s flying the EFVS HUD 

condition, an average of four head-up-to-head-down (and vice versa) visual transitions were made 

when flying the Conventional HDD condition. Approximately 35% of the time was spent head-down 

by the pilot flying the Conventional HDD condition in the visual segment. This head-down time was 

time well-spent, checking flight instrumentation, but flight technical performance is comparatively 

better when using the HUD. The pilot flying a HUD was able to simultaneously attend to the visual 

references and the instrumentation, without shifting attention. The HDD condition also was impacted 

by a minified flight path and guidance portrayal versus conformal, non-minified HUD flight path and 

guidance. 

For consideration of operational credit, the use of EFVS should provide equivalent, if not better, 

levels of performance and safety in the visual segment (from DH to 100 ft) as the operational 

comparative baseline. This comparison shows that performance in the visual segment is significantly 

better for the EFVS HUD condition than the operational baseline. Further, since the Conventional 

HDD condition still tends to draw the pilot head-down, inside the cockpit, even though they are in the 

visual segment, one can logically argue that the EFVS HUD condition improves the safety of flight 

by increasing the time that a pilot is head-up, looking outside the aircraft. 

5.1.3 EFVS Landing and Roll-Out 

The landing and touchdown data shows no statistically significantly differences in touchdown 

statistics (longitudinal and lateral position, sink rate) between the EFVS HUD and Conventional 

HDD condition. The data showed that there were no wing/empennage strikes of the ground in either 

condition. All touchdowns, for both the EFVS HUD and Conventional HDD conditions, resulted in a 

touchdown within the touchdown zone (200 ft to 2700 ft from the threshold), with the gear within the 

-100 -80 -60 -40 -20 0 20 40 60 80 100

-100

-80

-60

-40

-20

0

20

40

60

80

100

 Vertical Deviation from 50 ft Threshold Xing Height (ft)

 Lateral Deviation from Path (ft)

Base (HDD)

EFVS
75 

lateral confines of the runway, and with an acceptable sink rate. 

The eye gaze tracking data showed statistically significant differences in the number of headdown/head-up eye transitions in the flare and landing segment. In the flare segment, pilots still had 

on average one head-down/head-up transition when flying with the Conventional HDD condition. (In 

the landing and roll-out segment, the visibility condition triggered changes in the number of headdown/head-up eye transitions.) 

For consideration of operational credit, the use of EFVS should provide equivalent, if not better, 

levels of performance and safety in the flare, landing, and roll-out as the operational comparative 

baseline. The simulation results show that landing performance and roll-out performance is 

equivalent to the operational baseline condition. The simulations included headwinds, tailwinds, and 

direct crosswinds up to 15 knots. The EFVS HUD used a flare prompt, but did not include any form 

of flare guidance. 

Two other caveats are important in understanding these data. Overall, the data showed higher sink 

rates at touchdown than what would normally be expected. These sink rates did not differ by display 

configuration. There was a general simulation trend across all experimental variations. Two 

contributing causes were identified. First, the simulation did not include motion. Fixed-base 

simulations have been found to yield higher sink rates at landing than identical simulations conducted 

using motion-base cueing [37]. Second, the simulation used sidestick controllers on an aircraft 

normally equipped with a wheel and column. Fly-by-wire control laws were not implemented to 

improve the handling characteristics of the vehicle. So the combination of unique handling qualities 

and the fact that no real-world training or experience with sidestick controllers on the simulated 

aircraft was available to the flight crews before this experiment contributed to the higher than normal 

sink rates at touchdown. 

5.1.4 EFVS Clutter and Obscuration 

Non-normal runs were injected into the test unbeknownst to the PF and PM. Of the non-normal runs 

one was an unexpected runway incursion scenario, followed by nine expected runway incursion 

scenarios. These incursion scenarios provide an operationally oriented, quantification of traffic 

awareness using SEVS and CDTI. 

The unexpected runway incursion suggests that the use of EFVS improves the PF’s ability to detect 

runway incursions. When the data are collapsed across the CDTI condition, in three out of six runs 

with the EFVS, the PF was the first pilot to detect the runway incursion. In only one out of six runs 

without EFVS (i.e., flying the SVS PFD condition), the PF was the first pilot to detect the runway 

incursion. The EFVS provided an enhanced view of the incurring traffic in the HUD and this 

imagery aided detection. 

The data shows that the EFVS imagery does not clutter or obscure the PF’s view out of the window 

but instead, enhances the PF’s view out of the window. The test did not, however, test this hypothesis 

in weather conditions where the natural outside visibility and the enhanced visibility conditions were 

nearly equivalent. 

5.1.5 EFVS Off-Nominal Operations 

Three off-nominal conditions using the EFVS were given unexpectedly to the flight crew. On two 

runs, the HUD and EFVS failed with the RVR at 700 ft and at 1000 ft upon descending below 50 ft 
76 

HAT. On another run, the FLIR provided only 700 ft of visibility, instead of the normal 2400 ft 

visibility. The crew’s decision-making process when confronted with these non-normal events while 

flying in low-visibility conditions with the EFVS was assessed through descriptive statistics and 

(where appropriate) ANOVA analyses on run type (normal and non-normal) differences for flight 

performance. 

EFVS and HUD Failure 

Two EFVS failure runs (one in 1000 ft RVR and one in 700 ft RVR) were flown by each crew. The 

crews were not briefed or trained on this event. 

For the 1000 ft RVR EFVS failure condition, 11 of the 12 crews made the decision to land despite the 

complete loss of the HUD symbology and FLIR imagery. One crew made the decision to go-around 

at 38 ft AGL. 

For the 700 ft RVR EFVS failure condition, 10 of the 12 crews made the decision to land despite the 

complete loss of the HUD symbology and FLIR imagery. Two crews made the decision to go-around 

at 30 ft and 34 ft AGL for this test condition. 

ANOVA analyses on run type (F(2,41)=11.044, p<0.0001) with subsequent SNK post-hoc tests 

revealed that the lateral touchdown performance was significantly worse for the 700 ft RVR EFVS 

Failure runs (mean=-19.5 ft) compared to the 1000 ft RVR EFVS Failure runs (mean=5 ft) and EFVS 

normal runs (mean=-0.3 ft). There were no significant (p>0.5) run type (EFVS normal, 700 ft RVR 

EFVS Fail, 1000 ft RVR EFVS Fail) differences for the other touchdown measures of longitudinal 

position, sink rate, or airspeed. 

The EFVS failures occurred at 50 ft HAT. At this position, the data shows that the aircraft is in 

position to land, having just crossed the runway threshold, on speed and configured for landing due to 

the benefits of EFVS and guidance. When the EFVS HUD failure occurs, the data shows that most 

pilots just continued the operation since, in their opinion, there was sufficient outside visibility to 

safely complete the landing and roll-out. Pilot commentary remarked that even 700 ft RVR was 

sufficient outside visibility to complete the landing and roll-out, but 700 ft RVR was about the limit. 

Any less visibility and they would have been forced to go-around at this low altitude because they 

would not have had sufficient visibility to complete the operation. 

Insufficient Enhanced Flight Visibility to Land 

One insufficient enhanced flight visibility (700 ft RVR) to land run was flown by each crew. This run 

was flown to Runway 22L without TDZ/CL lights in a 7.5 knot right crosswind. 10 of the 12 crews 

made the decision to continue the approach and land even though the EFVS HUD did not provide 

sufficient visual flight references at the DH and 100 ft HAT points. Two of the 12 crews correctly 

followed EFVS crew procedures and initiated a go-around for this test condition. It should be noted 

that approximately 0.5 seconds after the 100 ft altitude aural callout the FLIR did provide sufficient 

visual flight references to complete the approach. This experiment set-up may have unintentionally 

encouraged crews to continue the approach since the FLIR visibility became sufficient to land so 

close to the 100 ft HAT point. 

ANOVA analyses revealed that there were no significant (p>0.05) run type (700 ft RVR FLIR or 

2400 ft RVR FLIR) differences for the touchdown measures of longitudinal position, lateral position, 

sink rate, or airspeed. 
77 

Off-nominal performance and non-normal scenarios should not create unacceptable or unsafe 

situations or conditions. Not all possible off-nominal situations were simulated in the test, but a few 

pertinent ones were tested and are reviewed as they may possibly influence the acceptability and 

utility of EFVS and thus, consideration for operation credit. 

5.2 EFVS Summary 

The data suggests that operational consideration for the use of EFVS HUD to enable descent below 

the published DA/DH on a straight-in instrument approach procedure, landing, and roll-out in 

visibilities as low as 1000 ft RVR without natural visibility references is warranted. 

5.3 SVS Operational Credit 

The SVS simulation results, in terms of performance and safety, are discussed with respect to the 

following five items to identify if consideration for operation credit is warranted for SVS. 

1. Did SVS provide equivalent levels of performance and safety in the Instrument Segment 

compared to the Operational Baseline? 

For consideration of operational credit, the use of SVS should provide equivalent, if not 

better, levels of performance and safety in the instrument segment – to a DH of 150 ft - as the 

operational comparative baseline. 

2. Did SVS positively influence the pilot’s visual search/runway acquisition at or before 

DA/DH? 

For consideration of operational credit, the use of SVS should positively contribute toward 

the pilot’s ability to transition from the instrument to the visual segment for awareness of the 

landing runway location, positively influencing the identification/verification of the landing 

runway, and acquiring the natural vision landing references. 

3. Did SVS affect performance and safety within the Visual Segment, including the ability to 

safety land and roll-out? 

For consideration of operational credit, the use of SVS should “do no harm” once established 

within the visual segment and for landing and roll-out operations. 

4. Any Effect of Clutter / Obscuration Using SVS? 

In the case of the SV-HUD, the combination of what the pilot can see in the SVS image, and 

what can be seen through and around the HUD, must be as safe and effective as the view 

without the HUD. This performance primarily but not exclusively considers the influence of 

HUD clutter and obscuration on the pilot-flying. 

5. Influence of SVS Design Factors and Off-nominal Operations 

Off-nominal performance conditions and scenarios should not create an unacceptable or 

unsafe situation. Not all possible off-nominal situations were simulated in the test, but a few 

pertinent ones were. 

The discussion collapses the comparison between the operational baseline (HUD without Synthetic or 

Enhanced Vision) flown in 1800 ft RVR to a 150 ft DH against SVS shown on a HUD (collapsed 

across evaluations in 1400 ft or 1000 ft RVR conditions) or on a HDD (collapsed across the 1800 ft 

and 1400 ft RVR). Runway lighting conditions were also collapsed across these conditions. 
78 

5.3.1 SVS Instrument Segment 

Using RMS deviation performance statistics in the instrument segment (from 1000 ft to DH), the SVS 

HUD condition showed no statistically significant differences in performance compared to the 

operational baseline (HUD/No SVS). Conversely, the SVS HDD condition showed degraded 

glideslope tracking and higher sink rate deviation compared to the operational baseline. Localizer 

tracking in all conditions was equivalent. When these data are analyzed in accordance with accepted 

FAA approach standards (FAA-S-8081-4D and AC120-29) which emphasize the maximum 

glideslope and localizer deviations instead of RMS deviation, the glideslope (~90% to 100% of runs 

meeting standards) and localizer (100% of runs meeting standards) tracking were essentially 

equivalent. These results suggest that the statistically significant differences found when analyzing 

RMS deviation may not be operationally relevant, at least to the 200 ft HAT point, used as the lowest 

altitude in AC120-29. 

As one might logically assume, the eye gaze tracking data showed that the SVS HDD concept had 

significantly more head-down time during the instrument segment. The pilots were head-up greater 

than 94% of the time with the HUD concepts and less than 22% of the time with the SVS HDD 

concepts. The data did show that the pilot-flying transitioned more, between head-up and head-down, 

when flying with the SVS HDD than with the HUD conditions (with and without SVS). But the data 

also shows that when flying with a HUD, the pilots were not exclusively head-up. The pilots 

averaged between three and five head-up/head-down transitions when flying the HUD. This result 

suggests that the pilots when flying the HUD were not exclusively using the HUD, but did transition 

into the cockpit, yet they did not stay inside for long. 

Another way of quantifying FTE within the visual segment is to evaluate how well the aircraft could 

be ‘delivered’ during the instrument segment to the runway. 

In Figure 23, the aircraft position upon reaching 200 ft HAT is plotted for the Conventional HUD and 

the SVS HUD conditions. This figure shows the vertical and lateral deviation of the aircraft from the 

three degree descent path. The figure illustrates almost identical positioning when flying either the 

Conventional HUD or SVS HUD. The presence of SVS neither improved nor degraded aircraft 

positioning to 200 ft HAT. 
79 

Figure 23. Aircraft Position at 200 ft HAT, Conventional HUD vs. SVS HUD 

In Figure 24, the aircraft position upon reaching 200 ft HAT is plotted for the Conventional HUD and 

the SVS HDD conditions. This figure shows the vertical and lateral deviation of the aircraft from the 

three degree descent path. The figure illustrates almost identical positioning laterally when flying 

either the Conventional HUD or SVS HDD. Slightly better vertical path tracking is shown for the 

Conventional HUD versus the SVS HDD. The minified SVS display and some attention sharing by 

the PF (on occasion) looking outside for emerging visual references are probably the cause of the 

degraded vertical tracking with the SVS HDD condition. 

-100 -80 -60 -40 -20 0 20 40 60 80 100

-100

-80

-60

-40

-20

0

20

40

60

80

100

 Vertical Deviation from Path (ft)

 Lateral Deviation from Path (ft)

Base (HUD)

SVS-HUD
80 

Figure 24. Aircraft Position at 200 ft HAT, Conventional HUD vs. SVS HDD 

In Figure 25, the aircraft position upon reaching 100 ft HAT – transitioning through the DH and into 

the visual segment – is plotted for the Conventional HUD and the SVS HUD conditions. This figure 

shows the vertical and lateral deviation of the aircraft from the three degree descent path. The figure 

illustrates almost identical positioning when flying either the Conventional HUD or SVS HUD. The 

presence of SVS neither improved nor degraded aircraft positioning to 100 ft HAT. 

In Figure 26, the aircraft position upon reaching 100 ft HAT is plotted for the Conventional HUD and 

the SVS HDD conditions. This figure shows the vertical and lateral deviation of the aircraft from the 

three degree descent path. The figure illustrates almost identical positioning laterally when flying 

either the Conventional HUD or SVS HDD. Slightly better vertical path tracking is shown for the 

Conventional HUD versus the SVS HDD. The minified SVS display and some attention sharing by 

the PF (on occasion) looking outside for emerging visual references are probably the cause of the 

degraded vertical tracking with the SVS HDD condition. 

 

-100 -80 -60 -40 -20 0 20 40 60 80 100

-100

-80

-60

-40

-20

0

20

40

60

80

100

() p

 Vertical Deviation from Path (ft)

 Lateral Deviation from Path (ft)

Base (HUD)

SVS-HDD
81 

Figure 25. Aircraft Position at 100 ft HAT, Conventional HUD vs. SVS HUD 

Figure 26. Aircraft Position at 100 ft HAT, Conventional HUD vs. SVS HDD 

For consideration of operational credit, the use of SVS should provide equivalent, if not better, levels 

of performance and safety in the instrument segment – to a DH of 150 ft – as the operational 

comparative baseline. The results show that equivalent performance is provided for the HUD 

condition and, while statistically significant performance differences were shown, acceptable 

-100 -80 -60 -40 -20 0 20 40 60 80 100

-100

-80

-60

-40

-20

0

20

40

60

80

100

 Vertical Deviation from Path (ft)

 Lateral Deviation from Path (ft)

Base (HUD)

SVS-HUD

-100 -80 -60 -40 -20 0 20 40 60 80 100

-100

-80

-60

-40

-20

0

20

40

60

80

100

 Vertical Deviation from Path (ft)

 Lateral Deviation from Path (ft)

Base (HUD)

SVS-HDD
82 

performance when using published FAA approach criteria has been shown for the SVS HDD 

condition tested in this experiment. 

5.3.2 SVS Runway Visual Acquisition 

The eye gaze tracking data was used to examine the influence of SVS on the pilot’s visual 

search/runway acquisition at or before DA/DH. 

The data shows no performance differences with and without SVS for the HUD conditions. 

Intuitively, no visual attention performance differences were expected. A pilot’s attention in both 

cases would principally be focused around the HUD flight path marker and guidance cue information. 

In the absence of errors, the guidance and flight path will be directed toward the synthetic and real 

runways; thus, the pilot’s attention would be directed, by design, in the generally correct 

direction/orientation with and without SVS. 

In comparison of the SVS HDD concept to the operational baseline (HUD/No SVS), the eye gaze 

tracking data is greatly biased by the differences in the display media (HDD vs. HUD) rather than the 

presence or absence of SVS. Pilots are head-down flying the SVS HDD condition approximately 

57% of the time in the instrument-to-visual transition segment versus only approximately 1% of the 

time for the HUD operational baseline. Pilots flying the SVS HDD concepts perform on average 

three to five head-down/head-up transitions, compared to less than one on average for the HUD 

baseline. These data suggest that the display media are extremely influential in this comparison. The 

pilot flying the HUD, with and without SVS, is visually oriented and directed toward the runway. 

Expecting equivalent performance in terms of head-up time, the number of head-up/head-down 

transitions, and the direction of eye gaze when transitioning to visual flight references when 

comparing a HUD to a HDD configuration is just not reasonable. 

Instead of using the operational baseline comparison for the HDD, the eye gaze tracking data were 

analyzed for the HDD condition with and without SVS information [38]. When comparing a pilot’s 

first transition to OTW to find the visual references/landing runway, the data show no statistically 

significant differences due to the presence or absence of SVS on the HDDs. However, the data does 

trend toward better performance for SVS (82% of the transitions were in the correct direction to the 

runway vs. 73% correct without SVS). For a full transition to visual flight (the time when the pilot 

goes head-out and stays predominately head-out for landing), the presence of SVS did support a 

better transition. 87% of the time, the pilot using a HDD with SVS correctly looked in the proper 

direction for the runway versus only 66% of the time without SVS. In the instrument-to-visual 

transition segment, no statistically significant differences in head-up time or in head-up/head-down 

transitions were found. 

Performance data were also used to examine the influence of SVS on the pilot’s ability to transition 

from the instrument segment to the visual segment. 

The JAR-AWO approach standards, which spans the 300 to 100 ft HAT range, is appropriate in this 

analysis since it represents the transition from the instrument segment and into the visual segment. 

Under the JAR-AWO criteria, all configurations had 100% probability of successfully meeting the 

within 1/3 dot localizer tracking criteria. The probability of success in meeting the within 1-dot 

glideslope tracking criteria showed display configuration effects. On average, the probability of 

success was 90% for SVS HUD conditions, reducing to 66% for the Conventional HUD condition (no 

SV), and only being ~35% for the SVS HDD. These numbers reflect again, the influence of the 

display (HUD vs. HDD) and the improved flight technical performance afforded by the HUD versus 
83 

the HDD. 

The pilot’s ability to transition from the instrument segment to the visual segment is also manifested 

in the go-around rates seen in the experiment. For the baseline condition (i.e., conventional HUD), in 

1400 ft RVR visibility conditions, one pilot executed a go-around out of 12 total. For the SVD HDD 

condition, one out of 12 pilots executed a go-around with 1800 ft RVR visibility but four executed 

go-arounds with 1400 ft RVR. Similarly, the SVS HUD condition also had four out of 12 pilots 

execute go-arounds with 1400 ft RVR. This data suggests that SVS did not reduce the go-around 

rate, either using a HUD or HDD. In fact, the presence of SVS increased the go-around rate. The 

HDD condition is saddled with the head-down to head-up visual transition problem. In the SVS 

HUD condition, the data suggest that a careful design is required for the SVS HUD since the SV must 

be decluttered to see the natural vision references. Unlike EFVS where the FLIR imagery should 

enhance the OTW view, the SVS may actually obscure the OTW natural view. Although the data 

does not positively identify all conditions, pilots noted this possibility and actively tried to mitigate it. 

Only one SVS HUD design was used and it was not subjected to hundreds of hours of evaluation 

before starting formal data collection. It represents one data point. 

The go-around rate increased to seven out of 12 pilots executing go-arounds with only 1000 ft RVR 

in the SVS HUD condition. The impoverished visual references in this condition clearly triggered a 

high go-around rate since the flight crews did not have sufficient natural vision references to safely 

continue to landing. From a 150 ft DA/DH point, 1000 ft RVR will create a high go-around rate. 

For consideration of operational credit, the use of SVS should positively contribute toward the pilot’s 

ability to transition from the instrument to the visual segment for awareness of the landing runway 

location, positively influencing the identification/verification of the landing runway, and acquiring the 

natural vision landing references. The data shows that SVS on the HUD neither improves nor 

degrades awareness of the direction of the runway. Subjective data from the pilots suggest that the 

terrain and runway shown on the SV gives them better SA of the guidance and time to transition. SV 

does not alter the fundamental problem of head-down to head-up transition using a HDD for an 

instrument approach; however, the data suggests that there is a slight improvement in that transition 

with SV (compared to a HDD without SV). The SV used for the HUD can be a critical issue since 

the SV can obscure, not enhance, the pilot’s view of the natural vision references. De-clutter of the 

raster imagery is critical. 

5.3.3 SVS Visual Segment 

The landing and touchdown data shows no statistically significant differences in touchdown statistics 

(longitudinal and lateral position, sink rate) between the SVS concepts and Conventional HUD 

condition. The data showed that there were no wing/empennage strikes of the ground in either 

condition. 

All touchdowns, for both the SVS HUD and Conventional HUD conditions, resulted in a touchdown 

within the touchdown zone (200 ft to 2700 ft from the threshold), with the gear within the lateral 

confines of the runway, and with an acceptable sink rate. Two touchdowns, for the SVS HDD 

condition, resulted in a touchdown outside/short of the touchdown zone. One occurred just short of 

the 200 ft point and the other was short of the threshold. Since the pilot in these cases did not have a 

HUD, it is not likely that SVS contributed to these adverse landings so much as it was the pilot’s 

inability to land the aircraft visually. 

The eye gaze tracking data showed that pilots with the SVS HDD condition were still head-down 
84 

approximately 23% of the time in the visual segment. This time is obviously significantly different 

than the 93 to near 100% head-up time for the HUD conditions. In the flare segment, pilots still had 

on average one head-down/head-up transition when flying with the SVS HDD condition. This headup/head-down transition effect appears to be symptomatic of a HDD and is not unique to the presence 

or absence of SVS. Further investigation comparing SVS HDD and Conventional HDD concepts 

reveals SVS decreased the head-down time during the visual segment to on average 25% compared to 

35% head-down with the Conventional HDD concept, with no significant variation in number of 

transitions [38]. Analysis of landing and roll out segments showed no difference between concepts. 

These results suggest the transition behavior is symptomatic of the HDD and not unique to the 

presence of SVS. 

5.3.4 SVS Clutter and Obscuration 

In the case of the SV-HUD, the combination of what the pilot can see in the SVS image, and what can 

be seen through and around the HUD, must be as safe and effective as the view without the HUD. 

This performance primarily, but not exclusively, considers the influence of HUD clutter and 

obscuration on the pilot-flying. 

No quantitative data were taken to specifically test this effect. However, pilot commentary suggests 

that the SV presentation in use on the HUD must be carefully designed to avoid obscuring required 

natural vision landing references. In post-test debriefings, six of the 12 PFs commented that it was 

hard to see the real world (e.g., lights, terrain, etc.) through the SV imagery. Specifically, they said: 

 SV on HUD was hard to look through so you had to decide when you need to turn SV off to 

see the real terrain. However, PF would like SV on HUD over Conventional HUD. 

 SV on HUD had lots of clutter. It was difficult to distinguish between real world and display 

so PF toggled back and forth on displaying SV imagery. 

 SV on HUD is good on initial approach. It was hard to see OTW lights with SV imagery but 

would like SV when taxiing. PF suggested to blank out SV on HUD where you would expect 

to see approach lights 

 SV on HUD is not ready. PF had to declutter SV in close. 

 Decluttered SV on HUD to pick up strobes OTW and then put SV back on. HUD 

transmissivity was an issue regardless with or without stroke symbology. 

 PF could not see lights through SV on HUD so went around most times. 

The SV declutter function was useful in this process (i.e., allowing the pilots to declutter the SV 

presentation on the HUD to have a clear view thru the HUD to see the required visual references), but 

they preferred that the SV design would allow the references to appear without decluttering. For 

instance, SV depiction should be “cut-out” so that the runway, edge lines, threshold, and approach 

lights would not be obscured by the synthetic information. 

Two of the PFs praised the use of SV on the HUD during the post-test debriefings. Specifically, they 

said: 

 With SV on HUD all the way to touchdown, PF could easily have completed landings. SV 
85 

on HUD is nice for SA and maintaining level flight. 

 SV and FLIR on HUD made OTW visibility not relevant. 

Four of the 12 PFs made no post-test comments on using SV on the HUD. 

In the case of the SVS HDD, no effect of clutter or obscuration was observed. 

5.3.5 SVS Off-Nominal Operations 

Two off-nominal conditions using SVS were flown unexpected to the flight crew. One run included a 

lateral navigation system error scenario and the other included a vertical navigation system error 

scenario. The crew’s decision-making process when confronted with these non-normal events while 

flying in low-visibility conditions with SVS concepts was assessed through descriptive statistics and 

(where appropriate) ANOVA analyses on run type (normal and non-normal) differences for flight 

performance. 

Navigation System Errors 

Two navigation system errors, one lateral and one vertical, were flown by each flight crew while 

using SVS HDD in 1400 ft RVR and flying to a runway without TDZ/CL lights. The crew could 

potentially identify the navigation system error by looking OTW or by crosschecking the ILS 

deviation and SVS scene (minified, head-down). 

Lateral Navigation Error 

Table 35 shows the touchdown performance for the lateral navigation error non-normal runs 

compared to the normal runs flown in the same testing conditions (1400 ft RVR, without TDZ/CL 

lights). 

Table 35. Lateral Navigation System Error Touchdown Performance 

Touchdown Longitudinal

Position (ft)

Touchdown Lateral

Position (ft)

Touchdown Sink Rate

(ft/sec)

Touchdown Airspeed

(knots)

‐131

Lat

Error

+131

Lat

Error

Norm

Runs

‐131

Lat

Error

+131

Lat

Error

Norm

Runs

‐131

Lat

Error

+131

Lat

Error

Norm

Runs

‐131

Lat

Error

+131

Lat

Error

Norm

Runs

Number

of Go‐

Around  

1 0 5 1 0 5 1 0 5 1 0 5

Number

of Runs

5 6 43 5 6 43 5 6 43 5 6 43

Mean 1222.2 1106.4 859.8 6.9 4.9  ‐1.6  ‐9.5  ‐7.6  ‐6.8 127.6 126.6 127.2

Std Dev 774.4 434.8 428.3 14.5 20.7 20.8 3.5 2.5 3.1 2.4 1.6 2.1

Min 448.2 439.2  ‐50.0  ‐7.2  ‐23.8  ‐52.4  ‐13.1  ‐11.7  ‐12.5 124.5 124.7 121.3

Max 2396.5 1521.7 2146.5 22.8 28.0 50.7  ‐3.7  ‐4.9  ‐0.7 129.7 128.3 130.0
86 

ANOVA analyses with run type (normal, +131 ft lateral navigation error, -131 ft lateral navigation 

error) as the main factor indicated no significant (p>0.05) differences for any of the touchdown 

measures (longitudinal position, lateral position, sink rate, or airspeed). 

Video review of the +131 ft lateral navigation error runs revealed that: 

 All six runs were taken to a landing 

 Five of the six crews commented on the SVS misalignment 

Video review of the -131 ft lateral navigation error runs revealed that 

 Five out of six runs were taken to a landing 

 Two of the five crews commented on the SVS misalignment 

 For the one go-around, the PF called “go around” at 60 ft AGL. PM commented on the SVS 

misalignment after the run was completed. 

Vertical Navigation Error 

Table 36 shows the touchdown performance for the vertical navigation error non-normal runs 

compared to the normal runs flown in the same testing conditions (1400 ft RVR, without TDZ/CL 

lights). 

Table 36. Vertical Navigation System Error Touchdown Performance 

Touchdown Longitudinal

Position (ft)

Touchdown Lateral

Position (ft)

Touchdown Sink Rate

(ft/sec)

Touchdown Airspeed

(knots)

‐115

Vert

Error

+115

Vert

Error

Norm

Runs

‐115

Vert

Error

+115

Vert

Error

Norm

Runs

‐115

Vert

Error

+115

Vert

Error

Norm

Runs

‐115

Vert

Error

+115

Vert

Error

Norm

Runs

Number

of Go‐

Around  

2 1 5 2 1 5 2 1 5 2 1 5

Number

or Runs

4 5 43 4 5 43 4 5 43 4 5 43

Mean 999.8 831.9 859.8 15.4 ‐0.9 ‐1.6 ‐8.6 ‐6.8 ‐6.8 129.3 125.4 127.2

Std Dev 193.5 221.2 428.3 23.9 9.7 20.8 1.4 2.3 3.1 1.3 2.5 2.1

Min 710.7 522.5 ‐50.0 ‐12.3 ‐7.9 ‐52.4 ‐10.5 ‐9.8 ‐12.5 127.6 123.0 121.3

Max 1116.1 1113.1 2146.5 36.1 15.7 50.7 ‐7.2 ‐3.9 ‐0.7 130.8 129.6 130.0

ANOVA analyses with run type (normal, +115 ft vertical navigation error, -115 ft vertical navigation 

error) as the main factor indicated no significant (p>0.05) differences for any of the touchdown 

measures (longitudinal position, lateral position, sink rate, or airspeed). 
87 

Video review of the +115 ft vertical navigation error runs revealed that: 

 Five out of six runs were taken to a landing 

 For the one go-around, the PF did not see the runway environment by minimums and called 

“go around” (i.e., properly followed SVS Crew Procedures). 

Video review of the -115 ft vertical navigation error runs revealed that: 

 Four out of six runs were taken to a landing 

 One of the four crews commented on the SVS misalignment (“looks like we are landing short 

on synthetic vision”) 

 For the first go-around, the PM called “go around” at 240 ft AGL 

 For the second go-around, the PF called “go around” at 120 ft AGL because he did not see 

the runway environment (i.e., properly followed SVS Crew Procedures). 

These off-nominal performance conditions and scenarios did not create unacceptable or unsafe 

situations. Not all possible off-nominal situations were simulated in the test, but the few that were 

flown all ended safely. 

However, the absence of data in the form of verbal commentary by the pilots in flying these SV 

depictions with very large navigation errors suggests that the pilots are relying almost exclusively on 

the guidance for flight path tracking until visual flight references are obtained. 

5.4 SVS Summary 

The data suggests that operational consideration for the use of SVS to enable descent to a DA/DH as 

low as 150 ft HAT on an ILS approach by use of SVS may be warranted; however, several issues 

merit consideration in the design and approval of these systems. 

First, HUD implementations of SVS promote head-up attention for the PF and eased the instrument to 

visual transition. SVS HUD was very effective in augmenting HUD-based operations with 

significant SA of the runway and runway environment. However, the potential obscuration of the 

required natural vision references by the synthetic view must be considered in the design. Declutter 

by the PF using a declutter switch was effective but the pilots preferred that the SV did not obscure 

these cues by design. The SV should “cut-out” around these important visual cues. 

Second, HDD implementations of SVS were very effective in augmenting HDD-based operations 

with significant SA of the runway and runway environment. The benefit of SV to improve the 

instrument to visual transition was weakly supported but these implementations are still symptomatic 

of non-HUD flight. Several head-up/head-down visual transitions (with and without SV) were made 

on average by PFs, even as they approach the flare, to check head-down instrumentation. This 

divided attention must be considered in operational approval for low-visibility operations. In this test, 

a transition of control upon reaching visual conditions (i.e., pilot-monitored approaches) was not

tested. 

Lastly, the non-normal runs suggested significant reliance on the ILS-driven guidance and an absence 
88 

of concern for a mis-matched SV depiction. This result suggests that, while an SA benefit was 

provided by SVS, the guidance is critical. No safety issues were revealed in this test. However, nonILS-based guidance systems with less accuracy and integrity merit significant attention. Also, crosscomparison of the ILS-based guidance and non-ILS based SV depictions should be automatically 

made and alerting given in the event of significant differences since the data suggests that pilots 

cannot make this determination reliably. However, the flight crews were not specifically trained to 

perform this detection function. Future work should investigate this training effect. 

5.5 Traffic Awareness as Influenced by CDTI and SEVS 

The traffic awareness data (from SAGAT-like probes and unexpected runway incursion runs) shows 

generally mediocre awareness of traffic and runway incursion detection by pilots. This result 

underscores the importance of automatic flight deck-based conflict detection and resolution work, 

such as Enhanced Traffic Situational Awareness on the Airport Surface with Indications and Alerts 

(SURF IA). 

This work also highlights that three issues should be investigated for improved runway incursion 

detection, to complement SURF IA: 

1. The benefits of training in the use of EFVS for runway incursion should be examined. This test 

did not employ any training. A follow-on test should evaluate if specific training for the PF and 

PM would improve runway incursion detection with EFVS. 

2. The benefits of training in the use of CDTI for runway incursion should be assessed. This test did 

not employ any training in the use of CDTI, specifically for runway incursion detection. The 

function of the CDTI was explained to the crew, but the use of CDTI for clearing the runway was 

not described. A follow-on test should evaluate if specific training for the PF and PM would 

improve runway incursion detection with CDTI. 

3. The use of traffic locator boxes on ego-centric displays, driven by ADS-B information, should be 

investigated. The interaction with EFVS should also be considered in this work. Traffic locator 

boxes were not used in this examination of SEVS technologies; however, they may be useful for 

highlighting and cueing for runway incursion. This benefit must be evaluated against the 

potential negative effect of increased display clutter, confusion, or obscuration. 

5.6 Pilot-Monitoring and SEVS 

Selected runs were made by the PF without the PM being in the simulator. These data allow an 

assessment of the effect of OTW visual cue alerting provided by the pilot monitoring on the 

performance and visual attention for the pilot-flying during approach and landing low visibility 

operations. In addition, these runs collect data for correlation and comparison to a follow-on flight 

test configuration. These runs were not conducted to advocate nor imply the possible acceptance of 

single pilot operations for Part 25 aircraft. In addition, the general applicability of these single pilot 

results with respect to operations and equipage may not necessarily be representative of Part 23-type 

aircraft and GA operations. The limited applicability to GA is due to the high level of piloting 

experience for the subject pilots and higher flight path stability characteristics of the simulated 

aircraft compared to Part 23 GA aircraft. Table 37 shows the touchdown statistics for the EFVS and 

SVS HDD configurations by crew complement (single pilot, two pilots). 
89 

Table 37. Touchdown Statistics by Crew Complement 

Single Pilot  

EFVS HUD  

Without TDZ/CL Lights  

Crew  

EFVS HUD  

Without TDZ/CL Lights  

Single Pilot  

SVS HDD  

Without TDZ/CL Lights  

Crew  

SVS HDD  

Without TDZ/CL Lights  

1000 ft RVR/150 ft DH   1000 ft RVR/150 ft DH   1400 ft RVR/150 ft DH   1400 ft RVR/150 ft DH  

Number of  

Go‐Around /

Number of Runs  

0/36   0/12   7/36   1/12  

Touchdown Longitudinal

Position (ft)

Mean   892.7   798.6   755.8   685.0  

Std Dev   361.8   358.5   351.4   389.8  

Min   427.3   430.0   279.4   26.8  

Max   2179.0   1662.7   1839.5   1244.0  

Rating   Desired   Desired   Desired   Adequate  

Touchdown Lateral

Position (ft)

Mean   ‐0.7   ‐1.2   1.3   2.5  

Std Dev   9.9   14.7   15.9   21.8  

Min   ‐21.8   ‐30.8   ‐21.6   ‐33.5  

Max   30.2   25.9   28.1   50.7  

Rating   Desired   Desired   Desired   Desired  

Touchdown Sink Rate

(fps)

Mean   ‐7.1   ‐7.7   ‐7.4   ‐6.9  

Std Dev   2.2   3.7   3.0   2.8  

Min   ‐13.4   ‐15.8   ‐15.5   ‐11.7  

Max   ‐2.3   ‐3.4   ‐1.1   ‐2.4  

Rating   Adequate   Adequate   Adequate   Adequate  

The data reveals that: 

 All EFVS runs resulted in a landing, irrespective of single pilot or crew operations 

 19% (seven out of 36 runs) of the single pilot SVS HDD runs resulted in a go-around 

compared to 8% (one out of 12) of the crew SVS HDD runs flown in the same test condition 

(without TDZ/CL lights, 1400 ft RVR) 

 On average, both single pilot and crew runs using the EFVS HUD (without TDZ/CL lights, 

1000 ft RVR) and SVS HDD (without TDZ/CL lights, 1400 ft RVR) met desired lateral and 

longitudinal touchdown position criteria 

 Sink rates were higher than expected for both single pilot and crew runs using the EFVS 

HUD and SVS HDD configurations 
90 

ANOVA analyses for the EFVS HUD runs (without TDZ/CL lights, 1000 ft RVR) revealed no 

significant (p>0.05) crew complement (single, crew) differences for touchdown longitudinal position 

from threshold (mean=869 ft, =360 ft), lateral position from centerline (mean=-0.8 ft, =11 ft), or 

sink rate (mean=-7.2 ft/sec, =2.6 ft/sec). Similarly, ANOVA analyses for the SVS HDD runs 

(without TDZ/CL lights, 1400 ft RVR) revealed no significant (p>0.05) crew complement (single, 

crew) differences for touchdown longitudinal position from threshold (mean=736 ft, =359 ft), lateral 

position from centerline (mean=1.6 ft, =17 ft), or sink rate (mean=-7.3 ft/sec, =2.9 ft/sec). 

ANOVA analyses for the EFVS HUD runs (without TDZ/CL lights, 1000 ft RVR) revealed no 

significant differences (p>0.05) in crew complement (single, crew) or CDTI configuration (none, 

Moving Map, or Moving Map and Runway Inset) for PF post-run workload ratings (mean= 3.2). PFs 

rated their workload as being “moderate, easily managed, and having considerable spare time” while 

using the EFVS concepts on approach through landing in visibilities of 1000 ft RVR. 

Similarly, ANOVA analyses for the SVS HDD runs (without TDZ/CL lights, 1400 ft RVR) revealed 

no significant differences (p>0.05) in crew complement (single, crew) or CDTI configuration (none, 

Moving Map, or Moving Map and Runway Inset) for PF post-run workload ratings (mean = 3.7). PFs 

rated their workload as being “busy; challenging but manageable; and having adequate time 

available” while using the SVS HDD concepts on approach through landing in visibilities of 1400 ft 

RVR. 

Eye tracking analysis was also conducted for the head-down configurations to evaluate the influence 

of crew assistance on visual behavior and attention. (Eye tracking behavioral differences for the 

HUD configurations were not expected and were not analyzed.) Surprisingly, no significant 

differences in visual attention were found during the approach and flare segments. Significant 

findings were, however, observed during the landing/roll-out segment, shown in Figure 27, indicating 

single pilots made several more transitions between the HDD and OTW. This is explained by the 

difference in task loading between the two comparisons. Two-crew operations allow for the PF to 

maintain attention OTW with the other crew member providing speed, runway remaining, and turnoff information callouts especially using the advanced airport moving map display. These tasks are 

critical in the rollout phase, made particularly more difficult in low visibility operations, and are all 

tasked to the individual pilot in single crew operations. This information is only available headdown, requiring the single pilot to transition with increased frequency to retain critical attention OTW 

while at the same time collecting the necessary information from the HDDs. 
Figure 27. Two-Crew vs. Single Pilot SVS O

91 

perational Comparison OTW/HDD Transition Count
92 

6 Conclusions 

A fixed-base pilot-in-the-loop simulation test was conducted at NASA Langley Research Center 

which evaluated the use of synthetic vision systems/enhanced flight vision systems (SVS/EFVS) in 

Next Generation Air Transportation System low visibility approach/landing operations at Chicago 

O’Hare environment. Various scenarios tested the potential for EFVS for operations in visibility as 

low as 1000 ft runway visual range (RVR) and SVS to enable lower decision heights or visibilities 

than can currently be flown today. 

Objective results indicate that expanding the portion of the visual segment for which EFVS can be 

used -from decision height (DH) to the runway - in visibilities as low as 1000 ft RVR appears to be 

viable as longitudinal and lateral touchdown performance were excellent. Perhaps more important 

than the landing performance results is that the go-around rate was 0% when flying the EFVS 

concept, regardless of the out-the-window (OTW) visibility level (1000 or 1400 ft RVR) or if 

touchdown zone/center line (TDZ/CL) lights were present or not. The enhanced flight visibility was 

held at approximately 2400 ft. Subjective results also supported the expanded use of EFVS from DH 

to the runway. This concept was rated as having less workload and was ranked as the crew’s 

preferred display concept (over the Conventional and SVS concepts tested) to fly with in lowvisibility conditions. 

RVR appears to affect lateral touchdown performance in the presence of an EFVS failure (i.e., no 

Head-Up Display (HUD) or Forward Looking InfraRed (FLIR) imagery available), but not 

touchdown longitudinal position or sink rate performance. However, all lateral touchdown positions 

were within 19 feet of centerline in the presence of an EFVS failure. 

Results of PF visual behavior under the EFVS operational concepts showed significant increase in 

head up time and reduced number of head up and head down transitions between HUD and HeadDown Display (HDD) vision system locations respectively for all in-flight segments of the approach. 

Particular significance was observed in the visual segment, indicating that pilots flying the 

Conventional HDD condition remained head down 35% of the time even after visual acquisition of 

the approach lighting system, continuing to check guidance and instruments available on the HDD. 

All EFVS runs resulted in a landing, irrespective of single pilot or crew operations. Similarly, no 

significant crew complement (single, crew) differences were found for the touchdown position or sink 

rate performance measures. 

Objective and subjective results indicate that using SVS on a HUD to enable lower decision heights 

and/or lower OTW visibility than are currently flown in today’s National Airspace System appears 

viable. Regardless of OTW visibility level or the absence/presence of TDZ/CL lights, all SVS HUD 

approaches were within the landing box (i.e., between 200 ft to 2700 ft longitudinal distance from 

threshold and within 58 ft lateral distance of centerline) defined in existing performance-based autoland standards (FAA AC120-28D and JAR AWO) for touchdown longitudinal position and lateral 

position from centerline. However, OTW visiblity impacted the go-around rate with the SVS HUD 

concepts, with nearly twice as many go-arounds being performed in 1000 ft RVR than being 

performed in 1400 ft RVR. SVS HUD operations in visibilities as low as 1400 ft RVR with a 150 ft 

DH appear possible. The SVS HUD go-around rates observed in this experiment and post-test pilot 

commentary indicate that synthetic vision presentation in use on a HUD must be carefully designed to 

avoid obscuring required natural vision landing references. 
93 

Regardless of OTW visibility level or the absence/presence of TDZ/CL lights, all SVS HDD concepts 

evaluated easily met the desired lateral touchdown criteria defined for this test. However, OTW 

visibility impacted the go-around rate for the SVS HDD concepts, with four times as many goarounds being performed in 1400 ft RVR than being performed in 1800 ft RVR. SVS HDD 

operations in 1800 ft RVR with a 150 ft DH appear promising if TDZ/CL lights are present. 

The presence of large, unannunciated navigation system errors (lateral and vertical) did not affect 

pilot touchdown position or sink rate performance while flying with the SVS HDD concepts. The 

tendency to go-around was less profound with the large lateral navigation system error runs (one goaround out of 12 possible approaches) than it was with the large vertical navigation system error runs 

(three go-arounds out of 12 possible approaches). 

Results of Pilot Flying (PF) visual behavior under the SVS operational concepts showed significant 

increase in head up time and reduced number of head up and head down transitions between HUD 

and HDD vision system locations respectively for all segments of flight, including flare and landing 

rollout. During the visual segment of flight, the SVS HDD condition eye tracking results indicate 

pilot visual attention remains inside the flight deck 25% of the time. Pilot visual attention continued 

to transition between the OTW scene and flight instruments and guidance available on the HDD. 

Relative to the Conventional HDD condition, this is a 10% increase in head up time when using SVS 

during the visual segment. There were no significant effects in PF visual behavior observed when 

contrasting SVS and Conventional vision systems on the HUD. 

No significant crew complement (single, crew) differences were found for the touchdown position or 

sink rate performance measures for the SVS HDD concept tested. 

In general, having TDZ/CL lights appears to have aided the pilots in landing closer to the touchdown 

aim point (1000 ft past the runway threshold). 

Pilots reported significant gains in overall SA and traffic awareness when they had cockpit display of 

traffic information (CDTI). However, an unexpected runway incursion was not detected when a crew 

was flying with FLIR imagery on the HUD and CDTI head-down. 

Future research should include motion-based simulation testing for the SVS HUD and HDD concepts 

to assess its impact on approach and landing performance, especially in sink rate control on 

touchdown. 
94 

7 References 

[1] Joint Planning and Development Office. (2008). Next generation air transportation system 

integrated plan: a functional outline. Washington, DC: Author. 

[2] Federal Aviation Administration. (n.d.). Instrument Flight Procedures (IFP) Inventory Summary. 

Retrieved February 28, 2011, from 

http://www.faa.gov/air_traffic/flight_info/aeronav/ifpinventorysummary. 

[3] Federal Aviation Adminstration. (2010). Annual runway safety report. Washington, DC: Author. 

[4] Arthur, J. J., III, Prinzel, L. J., III, Kramer, L. J., Bailey, R. E., and Parrish, R. V. (2003). CFIT 

prevention using synthetic vision. Proceedings of SPIE, Enhanced and Synthetic Vision 2003, 

5018, 146-157. 

[5] Schiefele, J., Howland, D., Maris, J., Pschierer, C., Wipplinger, P., and Meuter, M. (2005) Human 

factors flight trial analysis for 3D SVS: Part II. Proceedings of SPIE, Enhanced and Synthetic 

Vision 2005, 5802, 195-206. 

[6] Schnell, T., Theunissen, E., and Rademaker, R. (2005). Human Factors Test & Evaluation of an 

Integrated Synthetic Vision and Sensor-Based Flight Display System for Commercial and Military 

Applications. Paper presented at the NATO Research and Technology Organization, Human 

Factors and Medicine panel workshop entitled “Toward Recommended Methods for Testing and 

Evaluation of EV and E/SV-Based Visionic Devices”, Williamsburg, VA, USA, 26-27 April 2005. 

[7] Kramer, L. J., Prinzel, L. J., III, Bailey, R. E., and Arthur, J. J., III (2003). Synthetic vision 

enhances situation awareness and RNP capabilities for terrain-challenged approaches. Proceedings 

of the American Institute of Aeronautics and Astronautics Third Aviation Technology, Integration, 

and Operations Technical Forum, AIAA 2003-6814, 1-11. 

[8] French, G. and Schnell, T. (2003). Terrain awareness & pathway guidance for head-up displays 

(TAPGUIDE): a simulator study of pilot performance. Proceedings of 22nd IEEE/AIAA Digital 

Avionics Systems Conference, 2, pp. 9.C.4 - 9.1-7. 

[9] Lemos, K. and Schnell, T. (2003). Synthetic vision systems: human performance assessment of the 

influence of terrain density and texture. Proceedings of 22nd IEEE/AIAA Digital Avionics Systems 

Conference, 2, pp. 9.E.3 - 91-10. 

[10] Schnell T., Kwon, Y., Merchant, S., Etherington, T., and Vogl, T. (2004). Improved flight 

technical performance in flight decks equipped with synthetic vision information system displays. 

International Journal of Aviation Psychology, 4, 79-102. 

[11] Alexander, A. L., Wickens, C. D., and Hardy, T. J. (2005). Synthetic vision systems: the effects of 

guidance symbology, display size, and field of view. Human Factors: The Journal of the Human 

Factors and Ergonomics Society, 47 (No. 4), 693-707. 

[12] Kramer, L. J., Arthur, J. J., III, Bailey, R. E., and Prinzel, L. J., III. (2005). Flight testing an 

integrated synthetic vision system. Proceedings of SPIE, Enhanced and Synthetic Vision 2005, 

5802, 1-12. 

[13] Kramer, L. J., Williams, S. P., and Bailey, R. E. (2008). Simulation evaluation of synthetic vision 

as an enabling technology for equivalent visual operations. Proceedings of SPIE, Enhanced and 

Synthetic Vision Conference 2008, 6957, 1–15. 

[14] McKenna, Ed. “Synthetic Vision Systems.” Avionics Magazine, May 2012, pp. 20-23. 

[15] Connor, Glenn. “On the Road to Zero-Zero.” Professional Pilot Magazine, April 2011. 

[16] Minimum Aviation System Performance Standards (MASPS) for Enhanced Vision Systems, 

Synthetic Vision Systems, Combined Vision Systems and Enhanced Flight Vision Systems. 

RTCA/DO-315, RTCA Inc. Washington, DC. Dec 2008. 

[17] Airworthiness Approval of Enhanced Vision System, Synthetic Vision System, Combined Vision 

System, and Enhanced Flight Vision System Equipment. FAA/AC 20-167, Washington, DC: 

Author. June 2010. 
95 

[18] Enhanced Flight Vision Systems. FAA/AC 90-106, Washington, DC: Author. June 2010. 

[19] Enhanced Flight Vision Systems. RIN 2120-AJ94. Washington DC: DOT/FAA. Retrieved May 1, 

2013, from http://www.reginfo.gov/public/do/eAgendaViewRule?pubId=201210&RIN=2120-

AJ94. 

[20] Minimum Aviation System Performance Standards (MASPS) for Enhanced Vision Systems, 

Synthetic Vision Systems, Combined Vision Systems and Enhanced Flight Vision Systems. 

RTCA/DO-315A, RTCA Inc. Washington, DC. Sept 2010. 

[21] Bailey, R. E., Kramer, L. J., and Williams, S. P. (2010). Enhanced vision for all-weather operations 

under NextGen. Proceedings of SPIE Enhanced and Synthetic Vision Conference 2010, 7689, pp. 

768903-1–768903-18. 

[22] Minimum Aviation System Performance Standards (MASPS) for Enhanced Vision Systems, 

Synthetic Vision Systems, Combined Vision Systems and Enhanced Flight Vision Systems. 

RTCA/DO-315B, RTCA Inc. Washington, DC. March 2011. 

[23] Corps, S. G. (1986). Airbus A320 side stick and fly by wire – an update. 5th SAE Aerospace 

Behavioral Engineering Technology Conference, Paper No. 861801. 

[24] Electronic Industries Association. Standard RS-343-A, Electrical performance standards for high 

resolution monochrome closed circuit television camera. Washington, DC: Electronic Industries 

Association, 1969. 

[25] Global Positioning System Wide Area Augmentation System (WAAS) Performance Standard. 

FAA. Washington, DC. October 2009. 

[26] Safety, Performance and Interoperability Requirements Document for ASTA-SURF Application. 

RTCA/DO-322, RTCA Inc. Washington, DC. December 2010. 

[27] Ames, Lawrence L. & George, Edward J. (1993). Revision and verification of a seven-point 

workload estimation scale. Air Force Flight Test Center: AFFTC-TIM-93-01. 

[28] Vidulich, M. A. and Hughes, E. R. (1991). Testing a subjective metric of situation awareness. 

Proceedings of the Human Factors & Ergonomics Society, 35th Annual Meeting, 1307-1311. 

[29] Endsley, M. R. (1988). Situational awareness global assessment technique (SAGAT). Proceedings 

of the National Aerospace and Electronics Conference, 789-795. 

[30] Federal Aviation Administration: Instrument Rating Practical Test Standards for Airplane, 

Helicopter, and Powered Lift, FAA-S-8081-4D, Dated April 2004. 

[31] Federal Aviation Administration Advisory Circular: Criteria for Approval of Category I and 

Category II Weather Minima for Approach, AC-120-29A, Dated August 12, 2002. 

[32] European Aviation Safety Agency, Joint Aviation Requirements, All-Weather Operations. 

Amendment 4, Dated February, 2007. 

[33] Willits, P. (Ed.). (2000). Jeppesen Sanderson Instrument/Commercial Manual. Englewood, CO: 

Jeppesen Sanderson. 

[34] Federal Aviation Administration Advisory Circular: Criteria for Approval of Category III Weather 

Minima for Takeoff, Landing, and Rollout, AC-120-28D, Dated July 13, 1999. 

[35] Duchowski, A. T. (2007). Eye Tracking Methodology. London: Springer-Verlag. 

[36] Field, Andy (2005). Discovering statistics using SPSS (2nd ed.). London: Sage Publications Ltd. 

[37] Grantham, William D. (1989). Comparison of flying qualities derived from in-flight and groundbased simulators for a jet-transport airplane for the approach and landing pilot tasks. NASA/TP1989-2962. 

[38] Ellis, K. K., Kramer, L. J., Shelton, K. J., Arthur III, J., & Prinzel, L. J. (2011). Transition of 

Attention in Terminal Area NextGen Operations Using Synthetic Vision Systems. Human Factors 

and Ergonomics Society 55th Annual Meeting. Las Vegas, NV: Human Factors and Ergonomics 

Society. 

 
8 App

8.1 Air

pendix A 

 Force Flight Test Center (AFFT

96 

C) Workload Estimate Scale 
97 

8.2 Paired Comparison Technique 
98 

 
99 

8.3 Example Traffic Probe for Runway 22R 
REPORT DOCUMENTATION PAGE Form Approved

OMB No. 0704-0188

2. REPORT TYPE 

Technical Publication

 4. TITLE AND SUBTITLE

Enhanced Flight Vision Systems and Synthetic Vision Systems for 

NextGen Approach and Landing Operations 

5a. CONTRACT NUMBER

 6. AUTHOR(S)

Kramer, Lynda J.; Bailey, Randall E.; Ellis, Kyle K. E.; Williams, Steven 

P.; Arthur, Jarvis J., III; Prinzel, Lawrence J., III; Shelton, Kevin J.

 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

NASA Langley Research Center

Hampton, VA 23681-2199

 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

National Aeronautics and Space Administration

Washington, DC 20546-0001

 8. PERFORMING ORGANIZATION

 REPORT NUMBER

L-20133

10. SPONSOR/MONITOR'S ACRONYM(S)

NASA

13. SUPPLEMENTARY NOTES

12. DISTRIBUTION/AVAILABILITY STATEMENT

Unclassified - Unlimited

Subject Category 03

Availability: NASA CASI (443) 757-5802

19a. NAME OF RESPONSIBLE PERSON

STI Help Desk (email: help@sti.nasa.gov)

14. ABSTRACT

Synthetic Vision Systems and Enhanced Flight Vision System (SVS/EFVS) technologies have the potential to provide additional margins of safety for aircrew 

performance and enable operational improvements for low visibility operations in the terminal area environment with equivalent efficiency as visual 

operations. A fixed-base pilot-in-the-loop simulation test was conducted at NASA Langley Research Center that evaluated the use of SVS/EFVS in NextGen 

low visibility approach and landing operations. Twelve crews flew approach and landing scenarios to test the potential for using EFVS to conduct approach, 

landing, and roll-out operations in visibility as low as 1000 feet runway visual range (RVR). Also, SVS was tested to evaluate the potential for lowering 

decision heights (DH) on certain instrument approach procedures below what can be flown today. Expanding the portion of the visual segment in which EFVS 

can be used in lieu of natural vision from 100 feet above the touchdown zone elevation to touchdown and rollout in visibilities as low as 1000 feet RVR 

appears to be viable as touchdown performance was acceptable without any apparent workload penalties. A lower DH of 150 feet and/or possibly reduced 

visibility minima using SVS appears to be viable when implemented on a Head-Up Display, but the landing data suggests further study for head-down 

implementations. 

15. SUBJECT TERMS

Enhanced Flight Vision System; Equivalent Visual Operations; Head-up Display; NextGen Air Transportation System; 

Synthetic Vision System

18. NUMBER

 OF 

 PAGES

103

19b. TELEPHONE NUMBER (Include area code)

(443) 757-5802

a. REPORT

U

c. THIS PAGE

U

b. ABSTRACT

U

17. LIMITATION OF 

 ABSTRACT

UU

Prescribed by ANSI Std. Z39.18

Standard Form 298 (Rev. 8-98)

3. DATES COVERED (From - To)

5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

284848.02.03.07.01

11. SPONSOR/MONITOR'S REPORT

 NUMBER(S)

NASA/TP-2013-218054

16. SECURITY CLASSIFICATION OF:

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, 

gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this 

collection of information, including suggestions for reducing this burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and 

Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person 

shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.

PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS.

1. REPORT DATE (DD-MM-YYYY)

01- 11 - 2013