GAO

United States General Accounting Office

Report to the Chairman, Subcommittee

on Transportation, Committee on

Appropriations, U.S. Senate

June 2000 NATIONAL AIRSPACE

SYSTEM

Persistent Problems in

FAA’s New Navigation

System Highlight Need

for Periodic

Reevaluation

GAO/RCED/AIMD-00-130

Page 1 GAO/RCED/AIMD-00-130 National Airspace System

Contents

Letter 3

Appendixes Appendix I: Objectives, Scope, and Methodology 28

Appendix II: Experts Contacted 31

Appendix III: FAA Identified Four Alternatives for a Benefit-Cost

Analysis 33

Appendix IV: Comments From the Department of Defense 37

Appendix V: GAO Contacts and Staff Acknowledgments 38

Tables Table 1: Development Costs and Schedules forWAAS and LAAS,

1994 Through September 1999 9

Table 2: Summary of FAA’s Economic Analysis of Four Alternatives 35

Figures Figure 1: The Operating System for WAAS 8

Abbreviations

ATA Air Transport Association of America

DOD Department of Defense

FAA Federal Aviation Administration

GPS Global Positioning System

LAAS Local Area Augmentation System

OMB Office of Management and Budget

WAAS Wide Area Augmentation System
Page 2 GAO/RCED/AIMD-00-130 National Airspace System
Page 3 GAO/RCED/AIMD-00-130 National Airspace System

United States General Accounting Office

Washington, D.C. 20548

Resources, Community, and

Economic Development Division

B-283632 Letr

June 12, 2000

The Honorable Richard C. Shelby

Chairman, Subcommittee on Transportation

Committee on Appropriations

United States Senate

Dear Mr. Chairman:

Currently, the Federal Aviation Administration (FAA) relies principally on a

ground-based navigation system that uses various types of equipment to

assist pilots in navigating their assigned routes and to provide them with

guidance for landing their aircraft safely in different types of weather.

However, this ground-based navigation system is aging and limited in its

geographic coverage. FAA is planning a transition from its ground-based

navigation system to a satellite-based system using radio signals generated

by the Global Positioning System (GPS) to provide greater geographic

coverage, among other things. The Department of Defense developed GPS

to support military missions and functions. However, the system is now a

dual-use system, and other users—pilots, truckers, and boaters—rely on

signals from the GPS satellites to calculate their time, speed, and position

anywhere on or above the earth’s surface. As part of its efforts to maintain

GPS and make it more useful for civilians, in May 2000, Defense ceased its

practice of intentionally degrading the accuracy of the GPS signal available

for civil use.1 Furthermore, Defense plans to begin gradually replacing the

existing satellites with new ones that will also improve system

performance.

Although GPS already provides some critical information to pilots, FAA

believes that even with the greater accuracy and other improvements

expected from Defense’s newer satellites, this system will not satisfy all

civil aviation requirements for ensuring safe aircraft operations. For

example, FAA requires that its navigation system be unavailable no more

than 5 minutes per year for some types of navigation and landing. In

contrast, GPS could be unavailable for periods of time equaling up to 4 days

per year. To satisfy its requirements, FAA decided in the 1980s to augment

1 In the past, the Department degraded the accuracy of the GPS signal using a process

known as “selective availability.”
B-283632

Page 4 GAO/RCED/AIMD-00-130 National Airspace System

GPS with other navigational aids—the Wide Area Augmentation System

(WAAS) and the Local Area Augmentation System. These systems consist

primarily of a network of ground stations that receive, process, and

validate data from GPS before transmitting these data to pilots.

In deciding to augment GPS, FAA has conducted several benefit-cost

analyses—most recently in 1999. In its latest analysis, FAA estimated that

its future investment in this augmented navigation system could exceed $8

billion from 2000 through 2020.2 WAAS is the largest component of this

augmented system—37 percent of the total. Over the years, the cost of

developing WAAS has increased by over $500 million primarily because of

unanticipated development and additional program support costs. In

addition, WAAS has been delayed for over 3 years and has experienced

performance problems.

In light of the expected cost of this new navigation system and continuing

concerns about WAAS’ ability to achieve cost, schedule, and performance

goals, you asked us to provide information on whether (1) the Department

of Defense’s current GPS or its planned improvements for GPS can meet

FAA’s navigation requirements, (2) the benefits of FAA’s chosen approach to

an augmented system currently outweigh the cost of this system, and (3)

other technologies are available to meet FAA’s requirements and users’

needs for a new navigation system. In conducting this review, we examined

studies and spoke with experts in aviation navigation and related

technologies to obtain their views on the capability of FAA’s new navigation

system and alternatives to that system. Appendix I discusses our detailed

scope and methodology. Appendix II lists the experts we spoke with, some

of whom represent various technologies.

Results in Brief According to the studies we reviewed and experts we contacted, the

current Global Positioning System does not meet all of FAA’s civil aviation

navigation requirements for accuracy, integrity, and availability. FAA

defines accuracy as the degree to which a navigation system calculates an

2 Throughout this report, we refer to the augmented satellite navigation system and its

components—including the Local Area Augmentation System and existing ground-based

navigation aids—as FAA’s new navigation system. This system also provides guidance to

help pilots land at airports. Also, throughout this report, unless otherwise specified, costs

are presented in “then-year-dollars,” which are current dollars, inflated using Office of

Management and Budget guidance.
B-283632

Page 5 GAO/RCED/AIMD-00-130 National Airspace System

aircraft’s true position. Integrity is the ability of a navigation system to

provide timely warnings when its signal is providing misleading

information that could potentially create hazards for pilots and thus should

not be used. Availability is the probability that a navigation system meets

the accuracy and integrity requirements. Even though the Department of

Defense has made the current GPS signal provided for civilian use more

accurate and plans to further improve GPSby, for example, providing an

additional, higher-powered signal to combat interference, GPS still will not

fully meet FAA’s requirements for navigation and landing. This is because

GPS does not provide the assurance that its signal will be available virtually

all the time.

FAA’s 1999 analysis concluded that the quantified benefits of its approach

would outweigh the cost. Since completing this analysis, FAA has

experienced delays and cost increases primarily because of difficulties in

meeting its integrity requirement. As a result, it is unclear whether

quantified benefits will still outweigh cost. For example, to meet FAA’s

integrity requirement—which requires the Wide Area Augmentation System

to virtually never fail to warn pilots of potentially hazardously misleading

information—we estimate that the agency may need 3 or more years, at an

additional cost of between $200 million to $240 million, to demonstrate that

this requirement can be met. The system, with the ability to provide

integrity, was to have been operational by September 2000.We are making

recommendations to better ensure that FAA delivers its new navigation

system, on time and within budget, and that it meets performance

requirements. Transportation and Defense officials as well as the officials

from the Satellite Navigation User Group—representing commercial,

general avaition, and Defense users—acknowledged the problems

encountered in developing the new navigation system. Transportation

officials concurred with GAO’s conclusions and recommendations.

At the present time, no other navigation technologies—including variations

of ground-based and less robust satellite-based systems—are available to

meet FAA’s requirements and users’ needs for precise landing guidance at

more airports. The Wide Area Augmentation System is designed to provide

such guidance to improve safety and offer access to more airports.

However, as noted, this system is experiencing difficulties that cast doubt

on whether it will perform as designed at a reasonable cost and be

delivered on a reasonable schedule. Moreover, experts in alternative

navigation technologies, some of which compete with the Wide Area

Augmentation System, told us that users may have overstated their need
B-283632

Page 6 GAO/RCED/AIMD-00-130 National Airspace System

for precise landing guidance and that other navigation technologies could

satisfy most of these needs.

Background Currently, civil aviation relies principally on a ground-based navigation

system that uses various types of equipment to provide navigation and

landing services to pilots in different types of weather. This equipment

meets FAA’s performance requirements for accuracy, availability, and

integrity; however, it is aging and has limitations in its geographic coverage.

For example, under today’s ground-based navigation system, pilots fly

structured routes (referred to as highways-in-the-sky) that may not be the

most direct and fuel-efficient. The wider coverage provided by WAAS,

coupled with other improvements, would not restrict pilots to these

structured routes and would therefore result in more direct and fuelefficient flights.

To a lesser degree, civil aviation relies on GPS for its navigation needs. The

Department of Defense (DOD) maintains a constellation of 24 orbiting GPS

satellites for both military and civilian use.3 These satellites are positioned

so that at any given time the signals from a minimum of four satellites will

be available to users.

FAA is developing a new navigation system to augment GPS in order to

provide broader geographic coverage, among other things. The largest

component of this system is WAAS. When fully developed, WAAS could

comprise a network of up to 76 ground stations and three to four

geostationary communications satellites. (see fig. 1.) The WAAS network is

being designed to provide the same level of service as today’s ground-based

equipment and is expected to support navigation through all phases of

flight as well as nonprecision and category I precision landing approaches

for a wider geographic area.4 In a nonprecision approach, the pilot relies on

instruments on board the aircraft to guide it safely from a height ranging

from between 700 and 400 feet above touchdown. In contrast, in a category

I precision approach, the pilot relies on instruments to provide an aircraft

with safe vertical guidance to a height of not less than 200 feet above

touchdown. Currently, pilots can make precision approaches at only about

3 Replacement satellites will be launched as needed.

4 Navigation guidance is provided to pilots through all phases of flight—at high altitudes and

in areas close to airports.
B-283632

Page 7 GAO/RCED/AIMD-00-130 National Airspace System

625 airports. WithWAAS, FAA estimates that it may be possible to expand

this capability to approximately 3,300 airports, thus providing benefits to

more users. To obtain the full benefit from WAAS, it would be necessary for

these airports to incur costs for items they would need for providing

greater access at 200 feet, such as airport lighting systems, which could

cost between $1 million to $2 million per airport system. Additionally, many

pilots are not now qualified to fly these precision approaches and will

therefore need training to obtain the full benefits ofWAAS.
B-283632

Page 8 GAO/RCED/AIMD-00-130 National Airspace System

Figure 1: The Operating System forWAAS

Source: FAA.

Wide Area Augmentation System

WAAS Signal

Ground

Earth

Station

Master

Station

Integrity

Accuracy

Availability

Reference

Station

Geostationary

Satellites

Satellites

Alaska

Hawaii

San Juan

WAAS Signal

Geostationary

Communication

Satellites
B-283632

Page 9 GAO/RCED/AIMD-00-130 National Airspace System

As part of its augmentation of GPS, FAA, in partnership with industry,5 is

also developing the Local Area Augmentation System (LAAS) to support,

among other things, even more stringent precision approach guidance than

expected from WAAS. For example, in these approaches, LAAS is expected

to provide pilots with safe vertical instrument guidance to heights ranging

from less than 200 feet to down to the runway surface. While LAAS is

independent of WAAS, it is also expected to complement WAAS and

provide precision approaches at airports where WAAS does not provide

sufficient geographic coverage. LAAS will require the development, testing,

evaluation, and fielding of a new generation of ground stations—up to 160.

In January 1998, FAA approved development costs and schedules for LAAS.

At that time, FAA estimated that LAAS would cost $530 million and would

be operational by 2003.

Table 1 shows the history of development costs and schedules for WAAS

and LAAS.

Table 1: Development Costs and Schedules for WAAS and LAAS, 1994 Through

September 1999

5 Beginning is fiscal year 1999, FAA established a partnership with interested commercial

entities for the purpose of developing LAAS. FAA expects this partnership to culminate in

the development of a certified category I precision approach using LAAS by the end of fiscal

year 2002.

Dollars in millions

As of

1994 Jan. 1998 Jan. 1999 Sept. 1999

Types of costs

Total WAAS

development

costs

$508 $1,007a $1,007 $2,484b

Total LAAS

development

costs

c $530 $530 $720d

Schedule

Information

WAAS’ initial

capability

June 1997 July 1999 Sept. 2000 Sept. 2000
B-283632

Page 10 GAO/RCED/AIMD-00-130 National Airspace System

Note: Since 1996,FAA has been including life-cycle costs, which include costs for developing,

operating, and maintaining projects. The current life-cycle cost estimate for WAAS is $3,187.6 million.

a The Jan. 1998 program development costs for WAAS include the prime contractor costs,

development of standards and procedures, technical engineering and program support, and the first

year of costs for satellites.

b The Sept. 1999 estimate for WAAS development costs includes $1.3 billion in satellite service

acquisitions through 2020. In earlier estimates, satellite service acquisition costs were included in the

cost of operating WAAS.

c Costs and schedules were not developed until 1998.

d The Sept. 1999 LAAS cost increase is due in part to FAA now planning to acquire up to 160 systems.

In earlier cost estimates,FAA only planned to acquire 143 systems.

Source: FAA.

FAA is retaining about 30 percent of its ground-based navigation

infrastructure to address concerns about the vulnerability of the GPS

signal, which WAAS relies on, and to support those users who choose not

to purchase the equipment that must be used withWAAS. This

infrastructure, along with WAAS and LAAS, make up the components of

FAA’s new navigation system. Both WAAS and LAAS would require airlines

and general aviation users to purchase on-board equipment for receiving

signals from this new technology. These purchases are expected to occur

over time, as the new navigation system is developed.

WAAS and LAAS are being developed under a single FAA integrated

product team, which includes representatives from FAA’s aircraft

certification and acquisition organizations. FAA established crossfunctional teams to help ensure that systems are developed and

implemented in an efficient and effective manner. These teams are to be

empowered to make decisions affecting systems and services, from their

inception to their eventual disposal or termination. The effective operation

of the integrated teams is key to FAA’s goal of producing timely, costeffective acquisitions.

As of

1994 Jan. 1998 Jan. 1999 Sept. 1999

Schedule

Information

WAAS’ full

capability

Dec. 2000 Dec. 2001 To be

determined

Dec. 2006

LAAS’ first site

implementation

c 2003 2003 2003

LAAS’ last site

implementation

c 2006 2006 2010

(Continued From Previous Page)
B-283632

Page 11 GAO/RCED/AIMD-00-130 National Airspace System

GPS Needs

Augmentation to Meet

FAA’s Civil Aviation

Navigation

Requirements

Studies we reviewed, experts we contacted, and DOD officials we

interviewed, agreed that to meet FAA’s civil aviation requirements, FAA

must augment even the improved GPS. For example, according to a 1999

Johns Hopkins University study,6 augmentation was needed because GPS

failed to meet the critical requirement of system availability, which is the

probability that at any given time a navigation system will meet the

accuracy and integrity requirements for a specific phase of flight.7 FAA

defines accuracy as the degree to which an aircraft’s position, as calculated

by its navigation system, conforms to its true position. Integrity is the

ability of a navigation system to provide timely warnings when its signal is

providing misleading information that could potentially create hazards for

pilots and thus should not be used for navigation. This study considered the

impact of DOD’s decision to stop intentionally degrading the accuracy of

the signal for civilian use and concluded that augmentation was still

necessary. In May 2000, DOD actually stopped degrading its signal for

civilian use. As a result, the predicted accuracy for civilian use will increase

from about 100 meters (about 300 feet) to 20 meters (about 60 feet).

However, FAA requires an accuracy of about 7.6 meters (about 23 feet) for

landing aircraft.

6 This study, entitled GPS Risk Assessment Study, Final Report (Jan. 1999), was conducted

by the Johns Hopkins University Applied Physics Laboratory and was co-sponsored by FAA,

the Air Transport Association, and the Aircraft Owners and Pilots Association. According to

the Air Transport Association, the primary purpose of this study was to determine the

capability of the augmented GPS signal usingWAAS and LAAS.

7 Continuity and service volume are also considered major requirements, and they are

derived from the accuracy, integrity, and availability requirements. Continuity is the

probability that a navigation signal will meet accuracy and integrity requirements

continuously for a specified period. Service volume is the area of coverage for which a

navigation signal will meet availability requirements.
B-283632

Page 12 GAO/RCED/AIMD-00-130 National Airspace System

Moreover, Defense plans to begin gradually replacing the existing satellites

with new ones that will have an additional higher-powered signal to help

combat interference and improve overall system performance. However,

even with these improvements, GPS still falls short in meeting the

requirement that it be available virtually all of the time. For example, the

Johns Hopkins study found that GPS could be unavailable for periods of

time equaling up to 4 days per year for en route navigation.8, 9 In contrast,

FAA requires that its navigation system be unavailable no more than 5

minutes per year for en route navigation and nonprecision approaches.

Furthermore, GPS alone does not provide guidance for precision

approaches.

In addition, an expert we spoke with noted that FAA’s efforts to augment

GPS are in line with those of other nations to augment a navigation system

based on global satellites.10 Nations with navigation systems based on

global satellites will likely augment the signal provided by the global

system in order to maintain surveillance over civil navigation services. This

would enable them to independently monitor—from the ground—the

usability of the GPS signals and provide GPS status for their air traffic

controllers and aircraft.

8 En route navigation occurs when planes are in transit over the continental United States.

9 According to DOD, GPS’ availability is contingent upon the alignment of satellites with the

earth’s rotation.

10 Europe and Japan are currently developing augmentation systems.
B-283632

Page 13 GAO/RCED/AIMD-00-130 National Airspace System

Integrity Issues and

Other Problems in

Developing WAAS

Raise Questions About

Whether the Quantified

Benefits of FAA’s New

Navigation System

Outweigh Its Cost

According to FAA’s 1999 analysis, its new navigation system yielded

quantified benefits that exceeded costs over the period 2000 through 2020,

but development problems—principally related to proving theWAAS

integrity requirements—will raise costs, making it unclear whether

quantified benefits still exceed the cost of the new system. Also, in the

short term, because of the delays resulting from these problems, users will

not receive the benefits they expected from the system later this year, such

as greater capability to land at more airports in bad weather. Additional

costs are occurring because FAA has not appropriately overseen its

contractor11 and has underestimated software costs. Finally, it has been

suggested that the system’s cost may be higher if performance problems

indicate that FAA needs to maintain separate navigation equipment at the

airport.

11 The Raytheon Company has been the prime contractor since May 1996.
B-283632

Page 14 GAO/RCED/AIMD-00-130 National Airspace System

FAA’s Chosen Alternative

Yielded Net Benefits, but

Problems Will Most Likely

Reduce These Benefits

In 1999, in response to a congressional request, FAA reevaluated its plans

for a new navigation system.12 As part of this reevaluation, FAA conducted

a benefit-cost analysis of four alternatives. According to FAA’s analysis,

continuing with the WAAS/LAAS system, coupled with using about 30

percent of the agency’s current ground-based infrastructure as a backup to

WAAS, would yield the greatest net quantified benefits (dollar value of

benefits minus costs),13 regardless of whether passengers’ time savings are

included.14, 15 According to FAA, for 2000 through 2020, there is an 80-

percent chance that the net benefits of its preferred alternative are about

$2.5 billion if passengers’ time savings are counted and $72 million if these

savings are not counted. Expressed another way, the benefit-cost ratios for

these two scenarios are 2.4 and 1.1, respectively.16 The benefit-cost analysis

also identified at least 11 benefits that were not quantified. Included among

these were safety benefits, such as improved surface surveillance, as well

as operational benefits, such as enabling more landings at airports whose

operations are today limited by the lack of ground-based navigational

aids.17 According to FAA, its decision to pursue this approach had the

support of major users.18 (See app. III for a discussion of FAA’s analysis.)

12 See Satellite Navigation Investment Analysis Report, Federal Aviation Administration

(Sept. 25, 1999).

13 As an alternative to the benefit-cost ratio, for which the present value of benefits is

divided by the present value of costs, analysts sometimes calculate the present value of net

benefits. This value is equal to the present value of benefits minus the present value of

costs.

14 DOT values passenger time from $22 to $33 per hour, depending on the nature of air travel.

In 1998, our review of the economic literature found that no consensus exists on the validity

of valuing small increments of time, in part because passengers might not perceive and

value time savings of as little as 30 seconds.

15 It should be noted that the other three options had benefit-cost ratios of less than 1.

16A benefit-cost ratio is a measure of the relationship between the present value of a

project’s benefits and costs. Since benefits are divided by costs, any ratio above 1.0

indicates that the project is cost-beneficial; any ratio below 1.0 indicates that the project is

not cost-beneficial.

17 Data were not available to quantify 5 of the 11 benefits. For the other six, FAA determined

that the benefits would derive from WAAS, combined with other technologies, and therefore

did not attempt to separate and quantify the WAAS benefits.

18 The Satellite Navigation User Group—which includes representatives from commercial,

general aviation and Department of Defense users—was created to achieve a consensus

throughout the user community and within FAA for making the transition to the new

navigation system. In Mar. 2000, this group reaffirmed its support for WAAS.
B-283632

Page 15 GAO/RCED/AIMD-00-130 National Airspace System

While we did not conduct a detailed benefit-cost analysis, we believe that

under certain conditions the quantified net benefits of FAA’s chosen

alternative, without passenger value of time, could be negative, and the

benefit-cost ratio could be less than 1. This could occur because of added

costs resulting from delays or other problems. Furthermore, these delays

could have implications for FAA and users.

The Requirement for System

Integrity Will Delay WAAS

Deployment and Was Not

Addressed in a Timely

Manner by FAA

Management and the

Integrated Product Team

In December 1999, FAA found that the WAAS design could not be relied

upon to satisfy the agency’s requirement for system integrity for precision

approaches, which stipulates that WAAS cannot fail to warn pilots of

misleading information that could potentially create hazardous situations

more than once in 10 million approaches. Consequently, FAA has

determined that it will not make its scheduled date of September 2000 to

begin providing an initial capability for precision guidance (category I

approaches) through WAAS.19 The delay could have implications for FAA,

system users, and equipment manufacturers. For example, FAA may need

to buy new ground-based navigation equipment or maintain existing

equipment longer than expected—maintaining existing equipment costs

about $170 million annually, according to FAA. Likewise, system users and

equipment manufacturers could question the wisdom of making further

investments in WAAS technology. Because of these implications, FAA, with

users’ support, has decided to provide only a limited precision guidance

capability with WAAS by 2002.20 FAA has yet to determine when WAAS will

achieve its initial capability. However, FAA and major user groups contend

that they still will receive benefits from a limited WAAS. Using information

provided by FAA and its experts, we estimate that resolving the integrity

problem could potentially delay making WAAS’ initial capability available

by 3 years or more and add approximately $200 million to $240 million to

the cost of developingWAAS.

19 WAAS’ initial capability was defined as vertical guidance of 200 feet above touchdown

with a one-half to three-fourths mile visibility and a 19.2 meter vertical protection limit in

which an aircraft can maneuver and still land safely. This capability was to be available 95

percent of the time to about 50 percent of the continental United States.

20 By 2002, FAA plans to provide vertical guidance of 350 feet above touchdown with one

mile visibility and a 50 meter vertical protection limit in which an aircraft can maneuver and

still land safely. This capability is to be available 95 percent of the time to about 75 percent

of the continental United States.
B-283632

Page 16 GAO/RCED/AIMD-00-130 National Airspace System

To satisfy the integrity requirement, FAA and its contractor plan to make

changes to improve the calculations for better identifying misleading

information that could potentially create hazards for pilots; these changes

will cause software changes and may result in hardware upgrades. A team

consisting of FAA officials, its contractor, and consultants proposed these

changes and plans to be actively involved in ensuring that they will result in

proving the system’s integrity performance. By the end of 2000, the team

expects to determine what, if any, changes may be needed to theWAAS

design to achieve precision approaches down to 200 feet above touchdown.

The difficulties in proving the integrity requirement have occurred largely

because FAA management and the integrated product team underestimated

the complexity of resolving the integrity issue and, as a result, failed to

recognize the seriousness of the problem. Moreover, FAA did not closely

monitor the contractor’s effort to demonstrate integrity, and members of

the team did not have a clear understanding of their roles. Consequently,

team members did not effectively communicate with each other and the

contractor. Lack of monitoring and poor communications have been

recurring problems in FAA’s air traffic control modernization program. For

example, in 1996, we reported that inadequate oversight of contractors’

performance was a major contributor to FAA’s recurring cost, schedule,

and performance problems with other projects in the modernization

program.21

According to a WAAS study group FAA convened in October 1997, it would

be difficult to proveWAAS’ stringent requirements. Therefore, FAA would

need both a sound mathematical approach and evidence acquired through

operational experience.22 This group’s conclusions were validated in

December 1997, when according to FAA, it discovered at a key project

milestone that the contractor did not have an adequate plan for proving the

WAAS integrity requirement. Over the next 21 months, FAA, its experts, and

the prime contractor attempted to resolve the integrity issue; however, not

until September 1999, when the aircraft certification members of the

integrated product team became actively involved, did FAA fully

understand the difficulty in trying to prove this requirement. We found that

FAA’s progress in resolving this issue was hampered because (1) the

21 See Aviation Acquisition: A Comprehensive Strategy is Needed for Cultural Change at

FAA (GAO/RCED-96-159, Aug. 22, 1996).

22 Final Report of theWAAS Study Group (Oct. 16, 1997).
B-283632

Page 17 GAO/RCED/AIMD-00-130 National Airspace System

contractor took about a year to submit the limited results of its assessment

of the integrity issue and (2) the integrated product team was slow to

respond to advice from other FAA experts that the contractor’s integrity

assessment was inadequate.

FAA officials agreed that they should have monitored this situation more

closely but provided three reasons for not addressing the integrity

problems more promptly. These reasons largely concerned the actions of

FAA’s senior management and the integrated product team forWAAS:

• Competing priorities between FAA’s acquisition and aircraft certification

organizations, which are part of the integrated product team, negated

the effectiveness of this team’s approach for meeting the agency’s WAAS

goals. This situation may have developed because FAA’s aircraft

certification organization is more accustomed to being involved after a

project is developed, rather than actively participating throughout its

development. As we reported in 1996, FAA’s product teams have not

always forged true partnerships across organizational “stovepipes.”23

• A shortage of in-house technical expertise and the team’s attention to

other important issues, such as systems engineering, prevented the team

from monitoring the situation more closely.

• FAA did not have a sufficiently defined process for identifying and

conveying to the contractor the results that would be acceptable for

proving WAAS’ integrity.

Moreover, some team members did not feel empowered to take actions to

address performance issues because they did not believe that they had

senior management’s support in dealing with contractor-related problems.

In addition to these internal problems, an FAA official told us that the

contractor lacked sufficient expertise to prove the integrity requirement. In

any case, after being alerted to the difficulty in proving this requirement 2

years ago, FAA is only now beginning to pay it serious attention.

23 See Aviation Acquisition: A Comprehensive Strategy is Needed for Cultural Change at

FAA (GAO/RCED-96-159, Aug. 22, 1996).
B-283632

Page 18 GAO/RCED/AIMD-00-130 National Airspace System

Underlying these problems is a basic issue of how FAA decided to proceed

with WAAS’ development. Essentially, FAA took an acknowledged high-risk

approach: It agreed on a design for the system and established milestones

for system deployment before completing the research and development

needed to demonstrate the system’s capability. If FAA continues along this

path, it could incur significant costs for other system components, such as

satellites, to make the design fully operational—without knowing whether

the system will meet its performance requirements. A FAA senior manager

acknowledged that, in hindsight, the agency should have placed more

emphasis on how it would prove WAAS’ integrity performance prior to

agreeing to a design for the system. As we have reported for other FAA

modernization projects, when the agency attempts to combine different

phases of system development in an effort to more quickly implement

systems, it repeatedly experiences major performance shortfalls, which

lead to delays and additional costs.24

Recognizing these problems, FAA is in the process of implementing a new

approach to developing WAAS. Under this approach, before making

additional investments, FAA plans to allow time for collecting and

evaluating data on (1) system performance, (2) the extent to which users

have purchased equipment, and (3) the availability of emerging new

technologies for the new navigation system. In essence, FAA plans to

reevaluate WAAS at critical points—“checkpoints”—in its development.

We believe that reevaluating WAAS at checkpoints should provide critical

information for deciding about the need for future investments. Moreover,

such checkpoints would allow FAA to better assess whether it should shift

resources to other parts of the navigation system, such as LAAS, to meet

users’ needs. While FAA recognizes the need for checkpoints, it has not

developed a detailed plan explaining when these checkpoints would occur,

what they would accomplish, and who would be responsible for overseeing

them. We also believe that it would be prudent for FAA to have progress on

WAAS independently validated at the established checkpoints before the

Congress approves additional funding.

24 See Air Traffic Control: Observations on FAA’s Air Traffic Control Modernization Program

(GAO/T-RCED/AIMD-99-137, Mar. 25, 1999).
B-283632

Page 19 GAO/RCED/AIMD-00-130 National Airspace System

Software Development

Problems Are Likely to

Cause Delays

Given past problems and the amount of and potential complexity

associated with the remaining systems engineering development effort—

including developing algorithms to resolve the integrity issue—we believe

that the WAAS software effort will continue to experience delays. For

example, the schedule for the initial WAAS capability has already slipped

14 months—from July 1999 to September 2000—largely because of

problems in developing the WAAS design, which led to more software

development. At the end of its initial phase, WAAS will have approximately

350,000 lines of code and 370,000 additional lines or more of code will be

required before the system becomes fully operational.25 According to FAA,

this additional code will be needed to, among other things, provide for the

security of the system and to expand its operating capability.

We have reported on numerous occasions that in acquiring software, FAA

has not adequately detailed its design requirements and overseen

contractors’ development of the software.26 Software development—the

most critical component of key FAA modernization programs—has been

the Achilles’ heel of FAA’s efforts to deliver programs on time and within

budget.

FAA Underestimated

Annual Costs for

Maintaining Software

Even if all navigation system and development issues were resolved, the

quantified cost of the new navigation system is likely to outweigh the

quantified benefits because FAA underestimated the cost of maintaining

software by between 120 percent to over 200 percent, according to our

analysis.27 That is, we found that FAA’s estimate of $85 million for WAAS

software maintenance, which includes the staffing cost for modifying or

adding software throughout the life of the project, was understated by

between $101 million to $181 million. FAA’s estimate differed from our

estimate in part because the agency used more optimistic assumptions in

two instances. First, FAA assumed that 7 percent of the software would be

changed annually throughout the life of the project. In our analysis,

however, we assumed that 10 percent of the software would be changed

25 Software size is usually measured in lines of code. A line of code is a set of instructions

for the computer to perform a certain task and is one basis for estimating costs for items

such as the coding, analysis, design, and test efforts required for producing the line of code.

26 See Air Traffic Control: Immature Software Acquisition Processes IncreaseFAA System

Acquisition Risks (GAO/AIMD-97-47, Mar. 21, 1997).

27 This assumes no value to any passenger time savings.
B-283632

Page 20 GAO/RCED/AIMD-00-130 National Airspace System

annually. We made this assumption because (1) FAA’s own data suggested

that an annual factor of between 5 percent to 10 percent could be used and

(2) the model we used suggested that for software projects such as WAAS,

annual software change could approach 10 to 20 percent. Second, FAA

assumed that the government would maintain the software and used an

hourly labor rate of $75 per hour—the rate the agency uses when it

maintains software in-house. FAA’s own data suggested that having the

government maintain the WAAS software would be optimistic, given its

considerable complexity and safety-of-life purpose and the government’s

unfamiliarity with the software’s development. For our analysis, we used

the prime contractor’s software labor rate of $127 per hour because the

contractor is very familiar with the software’s development and well

qualified to maintain it.

Two Experts in Navigation

Technology Believe

Separate Equipment Must

Be Used in Conjunction

With WAAS

Costs for WAAS may also be higher than FAA anticipates if, as some

navigation experts believe, separate equipment is needed at airports. This

equipment would be used to monitor the informationWAAS provides to

aircraft as they approach an airport for a landing. Currently, FAA has no

plans to have such equipment in place. But these two experts note that FAA

has always ensured safety by requiring such equipment and that theWAAS

design should conform to this requirement.

To illustrate the need for such equipment, these experts noted that local

conditions around an airport, such as interference from man-made and

natural occurrences, may affect the GPS signal. The WAAS reference

station, which would send corrections to an aircraft, might be located

hundreds of miles from the airport and might not pick up these local

conditions. Local navigation equipment would factor in local conditions

and their effect on an aircraft’s true position. Without such information,

these experts maintain that a pilot will not have the critical information

needed to land safely. Even though this occurrence has not been observed

in tests to date, experts contend that the lack of observance is not proof

that it will not occur.

In response, FAA stated the agency’s experts have reasoned that the WAAS

design and the location of the reference stations are sufficient to account

for and correct any inaccuracies in the GPS signal. FAA disagrees that

separate, independent monitoring equipment is necessary because, given

the WAAS design, an airport-based monitor will not provide the required

integrity. This might occur because an airport monitor could receive its

information from a set of GPS satellites that are different from those that
B-283632

Page 21 GAO/RCED/AIMD-00-130 National Airspace System

are providing information to the aircraft. Nevertheless, FAA asked its

experts to model extreme conditions, and these experts have yet to detect

local inaccuracies. FAA also notes that it will not provide guidance for the

category I precision approaches until it has had an opportunity to evaluate

operational data and feedback from pilots. As it gains experience, FAA

acknowledges it may need to make changes to WAAS, which could increase

the cost of the WAAS investment.

Other Technologies Do

Not Meet FAA’s

Requirements and

Users’ Needs, but

Experts in Competing

Technologies Contend

That Some Needs Are

Overstated

At the present time, alternative technologies, including other ground-based

and less robust satellite-based systems, fall short in meeting FAA’s

requirements and users’ needs.28 WAAS is designed to provide precise

position and landing guidance (category I precision approaches) to meet

FAA’s requirements and users’ needs for improved safety and greater

access to more airports. The alternative technologies cannot meet FAA’s

requirement for category I precision approaches. However, according to

several experts supporting alternative technologies, they believed users

overstated the level of precision needed to safely land at more airports and

therefore a less robustWAAS or other technologies could satisfy users’

needs. However, FAA contends that WAAS serves other purposes in

addition to serving users’ needs. FAA told us that other modernization

efforts depend on the precise position information expected fromWAAS

and that if the agency were to rely on technologies other thanWAAS, it

would have to consider redesigning these other projects.

Other Technologies Cannot

Provide Category I

Precision Approaches

Experts on technologies that compete with WAAS acknowledge that,

unlike WAAS, these technologies cannot provide vertical guidance to a

height of not less than 200 feet above touchdown (category I precision

approaches) at more airports. Users—primarily general aviation pilots—

told FAA that they need this level of precision. It is important to note that

FAA has yet to demonstrate if and when WAAS will provide users with

guidance for category I precision approaches.

28 In considering FAA requirements and users’ needs, it is important to factor in the cost of

providing precision landing guidance. FAA notes that it could provide this guidance through

technologies such as instrument landing systems at all runway ends in the United States, but

this would be cost prohibitive.
B-283632

Page 22 GAO/RCED/AIMD-00-130 National Airspace System

While recognizing that other technologies cannot provide category I

precision approach guidance, these experts question whether such

guidance is needed. They note that providing users with access to more

airports depends upon factors other than the precise position information

WAAS offers. For example, to experience the full benefits of WAAS,

airports would need to invest in infrastructure, such as lighting systems,

and pilots would need training to fly precision approaches into these

airports. Given this potential investment and other considerations, these

experts, who support technologies that compete withWAAS, pointed out

three different available combinations of ground-based and satellite-based

technologies that could meet the vast majority of users’ needs for precise

landing information at a lower cost than WAAS: (1) Long Range Navigation

(LORAN C), a ground-based navigation system that provides position and

timing information to both aviation and nonaviation users;29 (2) a scaledback WAAS, which would not provide category I precision approaches; and

(3) LORAN C with an enhancement known as Eurofix. For example,

according to a 1998 study conducted for the Department of Transportation,

the cost to develop and operate LORAN C is estimated at over $600 million

for 2000 through 2015.30 Similarly, the cost to develop and operate WAAS

during this period is estimated at $2.3 billion. Most of the difference in cost

is attributable to the fact that WAAS would rely on newly launched

geostationary satellites, which LORAN C does not need.

These experts maintain that the competing technologies, combined with

on-board equipment that provides an increased vertical guidance

capability, can guide users down to a height of about 250 feet and therefore

achieve “near” precision approaches.31 They therefore contend that a less

costly technology than the WAAS now contemplated could still meet the

vast majority of the aviation community’s needs for precise landing

information—especially for general aviation users, who are expected to

benefit the most from WAAS. At runways where a 200-foot minimum

29 LORAN C was originally designed as a primary navigation system for the maritime

community. This technology is ground-based and can provide an independent navigation

service that does not rely on GPS.

30 An Assessment of the Proposed Phase Out of the LORAN C Navigation System, Booz-Allen

& Hamilton, Inc., July 17, 1998.

31 Users can receive vertical guidance from different types of equipment. For example, some

commercial aircraft are currently equipped with flight management systems that are

coupled with altimeters to provide vertical guidance. Using this equipment, pilots can make

“near” precision approaches.
B-283632

Page 23 GAO/RCED/AIMD-00-130 National Airspace System

category I approach is critical, these experts stated that these requirements

could be satisfied by LAAS.

According to FAA, it analyzed the technical capability of these three

alternatives as part of its 1999 reevaluation of its plan for a new navigation

system. It rejected the LORAN C and scaled-back WAAS alternatives

because they did not provide the precision navigation guidance expected

from WAAS. The agency rejected the Eurofix alternative because it did not

meet the integrity requirements for precision approaches. FAA also

believes that the alternative technologies and on-board equipment would

bring pilots down to about 350 feet, not the 250 feet the experts contend

could be achieved. Moreover, implementing these technologies would shift

a large cost from the government to general aviation system users—to

purchase additional equipment estimated at about $1,000 to $4,000 per

aircraft—and yet these users would still be limited in their ability to land at

more airports in foul weather. A group representing major users reaffirmed

its support for WAAS, with the expectation that it will provide category I

precision guidance. In addition, according to an interest group representing

general aviation aircraft owners and pilots, these individuals support FAA’s

approach because while proponents of the three alternative technologies

make claims about their capabilities, these capabilities have yet to be

validated. Furthermore, owners and pilots are not in favor of changing the

design of the new navigation system after FAA and aviation manufacturers

have agreed to specifications for equipment design and invested hundreds

of millions of dollars in this new design.

Using Other Technologies

Could Require Design

Changes in Modernization

Projects That Depend on

Precise Position

Information

As currently designed, several of FAA’s airspace modernization projects

depend onWAAS for precise position information and, according to FAA,

could not readily be developed withoutWAAS. While GPS provides

position information, WAAS is designed to ensure the integrity of that

information. If an alternative technology is used, these programs may need

to be redesigned. Chief among these is the Automatic Dependent

Surveillance Broadcast, one of a group of technologies designed to allow

FAA to implement a new system of traffic management. Under this system,

known as free flight, pilots and air traffic controllers would receive more

precise information about the location of aircraft in order to improve

system safety while allowing airspace and airport resources to be used

more efficiently. Specifically, the Automatic Dependent Surveillance

Broadcast communicates information about an aircraft’s position from onboard equipment that receives signals from global satellites and sends this
B-283632

Page 24 GAO/RCED/AIMD-00-130 National Airspace System

information directly to ground receivers and to nearby aircraft. With such

information, more aircraft can fly with increased efficiency and safety.

Other projects that depend on precise position information include

enhanced systems to warn pilots and air traffic controllers of proximity to

the ground and related obstacles. Depending on the application, the

effectiveness of these projects might be somewhat reduced in the absence

of WAAS. LAAS could provide monitoring ifWAAS was not deployed;

however, it would only provide for such monitoring at a specific

location/airport and would not provide the coverage expected from WAAS.

Conclusions FAA’s recently announced delays in the initial deployment of WAAS,

because of difficulties in proving integrity, are likely to delay the benefits

and raise the cost of the new navigation system. These delays have not

changed FAA’s original plans for WAAS, which were based on users’ needs.

However, if it is determined that WAAS cannot perform as intended, FAA

will need to revisit its investment in the new navigation system.

With regard to WAAS integrity, FAA’s management and its integrated

product team should have addressed these concerns in a more timely

manner. FAA recognizes that its management of this new system,

particularly WAAS, has fallen short, but its plans for resolving current

problems, including the integrity requirement, and preventing future delays

are ambiguous. FAA talks about establishing checkpoints to ensure

progress and the appropriate use of funds, but it has not developed a plan

to assure the Congress that these checkpoints are in place and are likely to

function appropriately.

Moreover, program success will only come about if senior FAA

management embraces and fully supports the integrated product team

concept and establishes an on-going process for the team to reach

consensus on how the contractor must demonstrate that a project meets

the agency’s performance requirements and to convey this information to

the contractor. Otherwise, more projects may experience the same

problems WAAS encountered.

Finally, our past reviews of FAA’s efforts to develop systems show that the

agency does not always inform the Congress in a timely fashion of

problems it is encountering before requesting additional funds. Potential

users also need information on system performance so that they can make

informed decisions on the merits of purchasing equipment to use with the
B-283632

Page 25 GAO/RCED/AIMD-00-130 National Airspace System

new navigation system. If further delays occur, these users may not

continue to support FAA in the development of this new system, as they

now do.

Recommendations To enhance FAA’s ability to develop its new navigation system within

budget and on time while meeting performance requirements, we

recommend that the Secretary of Transportation direct the Administrator

of FAA to take the following actions:

• Develop a comprehensive plan that would provide the framework for

the agency’s future investments in its new navigation system. This plan

should establish future checkpoints at which FAA would determine

whether (1) the contractor’s approach for meeting performance

requirements conforms with the agency’s guidelines, (2) users’ needs

have changed, and (3) other technologies have matured and could meet

users’ needs and the agency’s requirements.

• Have an external organization evaluate the agency’s progress at these

checkpoints and include the results of this evaluation in the agency’s

request for future funding of the navigation system.

Agency Comments GAO provided the departments of Transportation and Defense and the

Satellite Navigation User Group with a draft of this report for their review

and comment.

GAO met with officials from the Department of Transportation, including

the Program Manager, Radionavigation and Positioning, Office of the

Assistant Secretary for Transportation Policy and FAA’s Product Team Lead

for the Global Positioning System. These officials acknowledged the

problems addressed in the draft report concerning the Wide Area

Augmentation System and agreed with the conclusions and

recommendations. However, they expressed two major concerns. First,

these officials stated that our characterization of the increases in costs for

the Wide Area Augmentation System as “cost growth” was misleading

because the increases are attributable to cost growth as well as other

factors, including a change in how the department calculated project costs.

The officials provided revised estimates to reflect total lifecycle costs

covering a 20-year period. To address this concern, we revised the draft

report to clarify that the cost increases were due to growth as well as other

factors, such as additional program support costs. The focus of our review
B-283632

Page 26 GAO/RCED/AIMD-00-130 National Airspace System

was system development; therefore, we also clarified our draft report to

indicate that the cost estimates represented development costs only.

Second, these officials expressed the view that our discussion of the

benefits from the Wide Area Augmentation System implied that users

would only benefit if they were able to do the more stringent precision

approaches and that airports would need to spend a minimum of $1 million

in infrastructure upgrades to get precision approach capabilities from the

Wide Area Augmentation System. These officials maintained that users

would receive major benefits from any level of precision guidance into

airports and that airports would not necessarily need to upgrade their

facilities to provide for these approaches.We agree with the officials that

users would receive benefits even from the limited precision approach

capability provide by this system and that airports would not necessarily

need to upgrade their investment to provide for these approaches.We

revised the draft report accordingly. These officials also provided us with

technical clarifications, which we incorporated into the draft report as

appropriate.

The Department of Defense generally concurred with the draft report. In

addition, it offered two comments. First, it clarified the department’s

strategy for maintaining the Global Positioning System, and we revised the

draft report accordingly. Second, it commented that the continued delays in

the Wide Area Augmentation System and questions about worldwide

interoperability of this augmentation system could create additional cost

and schedule delays and adversely impact the department’s operations if

not addressed. (See app. IV for the Department of Defense’s comments.)

GAO met with representatives from the Satellite Navigation User Group,

who generally agreed with the draft report. These representatives

reiterated their support for FAA’s efforts to field a new navigation system.

In light of this, they were concerned that the draft report did not recognize

that users would receive benefits even from the limited precision approach

capability provided by the Wide Area Augmentation System and that

airports would not necessarily need to upgrade their investment to provide

for these approaches. We agreed and revised the draft report accordingly.

These officials also provided us with technical clarifications, which we

incorporated into the draft report as appropriate.

We are sending copies of this report to interested Members of Congress;

the Honorable Rodney E. Slater, Secretary of Transportation; the
B-283632

Page 27 GAO/RCED/AIMD-00-130 National Airspace System

Honorable William S. Cohen, Secretary of Defense; and the Honorable Jane

F. Garvey, Administrator, Federal Aviation Administration. We will also

make copies available upon request.

If you have any questions about this report, please contact me at (202) 512-

2834. Key contributors are listed in appendix V.

Sincerely yours,

Gerald L. Dillingham, Ph.D.

Associate Director,

Transportation Issues
Page 28 GAO/RCED/AIMD-00-130 National Airspace System

Appendix I

Objectives, Scope, and Methodology AAppenednxidxeIis

In light of the expected cost of the Federal Aviation Administration’s (FAA)

new navigation system and continuing concerns about the ability of the

largest component of that system—the Wide Area Augmentation System

(WAAS)—to achieve cost, schedule, and performance goals, the Chairman,

Subcommittee on Transportation, Senate Committee on Appropriations,

asked us to provide information on whether (1) the Department of

Defense’s (DOD) current Global Positioning System (GPS) or its planned

improvements for GPS can meet FAA’s navigation requirements, (2) the

benefits of FAA’s chosen approach to an augmented system currently

outweigh the cost of this system, and (3) other technologies are available to

meet FAA’s requirements and users’ needs for a new navigation system.

To address the first objective, we consulted with DOD officials and GPS

experts. In addition, we reviewed a 1999 study prepared by Johns Hopkins

University’s Applied Physics Laboratory, sponsored in part by FAA,1 that

assessed, among other things, whether GPS can meet navigation

requirements for civil aviation. In addition, we reviewed a 1998 study by the

Mitre Corporation, which supports FAA in developing the national airspace

system, that analyzed GPS’ current and future ability to meet navigation

requirements.

To respond to the second objective, we discussed with FAA and the Mitre

Corporation, the process they followed to develop a range of alternatives.

We reviewed guidance from the Office of Management and Budget (OMB)

to identify analytical issues agencies should address when doing benefitcost analyses. We also reviewed FAA’s analysis of the costs and benefits of

various technologies to determine whether the agency followed OMB

guidance and used empirical data to support its findings. In our review, we

also confirmed the reasonableness of the largest costs estimates—for

equipment on board the aircraft and additional satellites—with equipment

manufacturers, satellite providers, and aviation industry trade associations

and special groups, including the General Aviation Manufacturers

Association, Aircraft Owners and Pilots Association, the Air Transport

Association of America, and the Regional Airline Association. The latter

four associations and other users are also part of the Satellite Navigation

User Group, which was created to achieve a consensus in the user

1 The Johns Hopkins University study was performed under contract for the Air Transport

Association of America (ATA) and jointly sponsored by ATA, FAA, and the Aircraft Owners

and Pilots Association.
Appendix I

Objectives, Scope, and Methodology

Page 29 GAO/RCED/AIMD-00-130 National Airspace System

community and in FAA for making the transition to the new navigation

system.

We also interviewed representatives from the Air Line Pilots Association

International and the National Air Traffic Controllers Association to obtain

their views on the reasonableness of cost and benefit assumptions made in

FAA’s analysis. Furthermore, we relied on our past work related to WAAS to

compare the process FAA used for, and the results of, its most recent

benefit-cost analysis with a similar analysis performed in early 1998. While

we did not extensively review the model FAA used to calculate net benefits

and benefit-cost ratios, the methodology of this model, including

spreadsheet analysis and Monte Carlo Simulation, is commonly used in

making such calculations. Finally, we interviewed representatives from

Raytheon, the prime contractor for WAAS, to obtain data on the nature and

extent of problems in demonstrating thatWAAS can meet its integrity

requirement and in developing software for navigation.To determine

whether FAA’s estimates of the costs of software development and

maintenance were reasonable, we performed an independent assessment

of the contractor’s cost estimates using a model known as “CostXpert,”

which is used in the federal systems acquisition community to, among

other things, determine the costs of developing projects using historical

costs of completed projects with software development factors similar to

WAAS.

Finally, for the third objective, we elicited the views of experts in aviation

navigation and related technologies to discuss the capability of other

technologies to meet FAA’s requirements and users’ needs.To accomplish

this effort, we provided these experts with detailed data on the alternative

technologies developed by the Mitre Corporation that served as the basis

for FAA’s decision to continue with the new navigation system. We asked

these experts a series of questions that focused on whether FAA adequately

considered a full range of alternatives and whether they agreed with FAA

when the agency rejected certain technologies from its new navigation

system. Some of these experts supportedWAAS, while others supported

technologies that compete with WAAS. Therefore, where we identified

conflicting views between the FAA and these experts, we discussed the

points of contention openly with both sides to fully understand their

positions, and we present the views for and against using alternative

technologies. (See app. II for background on the experts we contacted

during this review.) Finally, we interviewed FAA officials to discuss

whether interdependencies exist between WAAS, LAAS, and other
Appendix I

Objectives, Scope, and Methodology

Page 30 GAO/RCED/AIMD-00-130 National Airspace System

modernization projects, and the impact of these other efforts ifWAAS is

not deployed as scheduled or is terminated.

We conducted our work from July 1999 through May 2000 in accordance

with generally accepted government auditing standards.
Page 31 GAO/RCED/AIMD-00-130 National Airspace System

Appendix II

Experts Contacted ApendxiI

Mr. John M. Beukers

Founder, Beukers Laboratories, Inc. (today known as Beukers

Technologies, Inc.), a company specializing in the implementation and use

of radionavigation systems since 1963. Mr. Beukers is a director of the

International Loran Association and the International Navigation

Association and also serves as a consultant to the U.S. government, the

European Union, and others on radionavigation policy.

Dr. John Diesel

Chief Scientist, Litton Aero Products. Litton Aero is a manufacturer and

supplier of inertial navigation and global positioning systems to the

aviation industry. Dr. Diesel has analyzed several navigation systems—

including GPS, Inertial Reference System, Loran, VHF Omni-directional

Range/Distance Measuring Equipment, and Instrument Landing Systems.

Mr. Paul R. Drouilhet

Consultant to the Director of the Massachusetts Institute of Technology’s

Lincoln Laboratories, where he oversaw programs in surveillance and

control technology and air traffic control. Mr. Drouilhet is also the

Chairman of FAA’s Subcommittee on Air Traffic Services and is an airplane

pilot.

Dr. Per Enge

Professor, Stanford University, Department of Aeronautics and

Astronautics; Co-Director of the GPS Research Laboratory. Dr. Enge is also

a developer of the prototype system and signal for WAAS and a researcher

for the prototype of LAAS.

Dr. G. Benjamin Hocker

Principal Research Fellow, Honeywell Technology Center. Currently, Mr.

Hocker focuses on the evaluation of technologies that include, among

other things, inertial navigation systems.

Mr. George H. Quinn

Until 1994, Mr. Quinn was FAA’s National Project Manager for the

establishment of the LORAN C navigation system for aviation. Currently,
Appendix II

Experts Contacted

Page 32 GAO/RCED/AIMD-00-130 National Airspace System

Mr. Quinn is a self-employed engineering consultant, developing ways to

use GPS in vessel tracking systems.

Mr. William Roland

Private consultant. Former Loran C Branch Chief and Commanding Officer

of the U.S. Coast Guard’s New Jersey Electronics Engineering Center. Mr.

Roland recently retired from his position as President of Megapulse, the

company responsible for the development and construction of a large part

of the world’s Loran transmitters.

Dr. G. Linn Roth

President, International Loran Association. Dr. Roth is also the president of

Locus, Inc., a developer and manufacturer of spread-spectrum radio

modules for integration into industrial, utility, GPS, and high-performance

digital Loran receivers for navigation and timing applications.

Mr. Robert Siegfried

Former commercial airline pilot with approximately 24,000 hours of

experience. Mr. Siegfried’s general aviation experience totals

approximately 12,000 hours—including about 1,200 hours operating

helicopters.

Mr. Victor Strachan

Director, Strategic Development, Litton Aero Products. Mr. Strachan served

as a navigator in the Royal Air Force for 18 years, including 5 years in flight

testing. Mr. Strachan is a director of the International Navigation

Association and a member of the Civil Aviation Council of the Aerospace

Industries Association. Litton Aero is a manufacturer and supplier of

inertial navigation and global positioning systems to the aviation industry.
Page 33 GAO/RCED/AIMD-00-130 National Airspace System

Appendix III

FAA Identified Four Alternatives for a BenefitCost Analysis ApendxIi

In 1999, in response to congressional concerns in late 1998 about its

proposed investment in WAAS and LAAS and the vulnerabilities associated

with satellite navigation, FAA reevaluated whether the investment was

sound. As a first step in this reevaluation, FAA identified four alternatives

that combined a range of technologies, based on different levels of

investment.1 Three of the four alternatives contained one or more

variations of WAAS and LAAS as well as variations of ground-based

navigation aids that could serve as the principal backup system. The four

alternatives were the following:

• Alternative I: No WAAS or LAAS. FAA retains its existing groundbased navigation infrastructure and enhances it to accommodate

demand.

• Alternative II: Simplified WAAS With LAAS. FAA develops a

simplified WAAS, without incorporating precision approach guidance,

develops LAAS, and retains about 50 percent of the existing groundbased infrastructure used for navigation and about 30 percent of the

ground-based infrastructure used for precision approaches.

• Alternative III: Less Robust WAAS With LAAS. FAA develops

WAAS with precision approach capability, including LAAS, for more

stringent approach requirements, and retains about 50 percent of its

ground-based infrastructure.

• Alternative IV: Full WAAS With LAAS. FAA develops WAAS with

precision approach capability, including LAAS, for more stringent

approach requirements, and retains about 30 percent of its existing

ground-based infrastructure.

In developing the alternatives, FAA made three major assumptions. First, it

assumed that the new navigation system would not be the only system that

pilots would use to navigate and land—a “sole-means” system—as

originally intended. By moving away from this assumption, FAA realized

that it would need to retain a backup system. Second, for all the

alternatives, FAA assumed that all new commercial jets would be produced

with avionics on board the aircraft to help pilots navigate (inertial

navigation systems), which would be used in conjunction with other

ground-based navigation aids, and that affordable inertial systems would

not be available until about 2008. Finally, FAA assumed that the Long Range

Navigation (LORAN C) technology would continue to be provided and used

by some general aviation pilots until 2008.

1 In total, FAA considered 12 alternatives for its new navigation system.
Appendix III

FAA Identified Four Alternatives for a

Benefit-Cost Analysis

Page 34 GAO/RCED/AIMD-00-130 National Airspace System

After identifying the four alternatives, FAA analyzed the costs and benefits

of each. According to this analysis, continuing with the WAAS/LAAS

system, coupled with using about 30 percent of its current ground-based

infrastructure as a backup to GPS, would yield the greatest net quantified

benefits (dollar value of benefits minus costs),2 regardless of whether

passengers’ time savings are included.3, 4 According to FAA, over the period

2000 through 2020, there is an 80-percent chance that the net benefits of its

preferred alternative is about $2.5 billion if passengers’ time savings are

counted and $72 million if these savings are not counted. The benefit-cost

analysis also identified at least 11 benefits that were not quantified.

Included among these were safety benefits, such as improved surface

surveillance, as well as operational benefits, such as enabling more

landings at airports where today’s operations are limited by lack of groundbased navigational aids. According to FAA, its decision to pursue this

approach had the support of all major user groups.5

Table 2 summarizes FAA’s analysis for the four basic alternatives the

agency considered in terms of each alternative’s benefit-cost ratio and net

benefits, expressed as present values.6 FAA considered these alternatives

with and without passengers’ value of time. As the table shows, the fourth

alternative—which is the one FAA decided to pursue—provided a benefitcost ratio of 2.4, when passengers’ value of time are included, and a ratio of

1.1, when these values are not included.

2

As an alternative to the benefit-cost ratio, for which the present value of benefits is divided

by the present value of costs, analysts sometimes calculate the present value of net benefits.

This value is equal to the present value of benefits minus the present value of costs.

3 The Department of Transportation values passenger time from $22 to $33 per hour,

depending on the nature of air travel. In 1998, we reviewed the economic literature and

found that no consensus exists as to the validity of using small increments of time, in part

because passengers might not perceive and value time savings of as little as 30 seconds.

4 It should be noted that the other three options had benefit-cost ratios of less than 1.

5 The Satellite Navigation User Group was created to achieve a consensus throughout the

user community and with FAA for making the transition to the new navigation system.

6 Since benefits are divided by costs, any benefit-cost ratio above 1.0 indicates that the

project has benefits exceeding costs, and any ratio below 1.0 indicates that the project has

benefits less than costs.
Appendix III

FAA Identified Four Alternatives for a

Benefit-Cost Analysis

Page 35 GAO/RCED/AIMD-00-130 National Airspace System

Table 2: Summary of FAA’s Economic Analysis of Four Alternatives

Note: Benefit-cost ratios and net benefits are calculated in present value terms, which account for

future benefits and costs, here expressed in 1999 dollars.

Source: FAA.

We found that FAA combined empirical data, where available, with

professional judgment to develop its estimates of costs and benefits. For

example, FAA’s estimates for key cost elements, such as geostationary

satellites, on-board aircraft equipment, and ground-based navigation aids,

were based on vendor quotes and manufacturer pricing surveys.We

independently validated FAA’s cost for developing WAAS software and

found that it was within an allowable range. Moreover, we found that FAA

did not assume that some benefits, such as direct routing—which yields

savings from shorter flight times—could only be achieved withWAAS and

did not fully attribute these benefits to WAAS as it had in earlier studies.

Furthermore, in accordance with OMB guidance and acceptable practice,

FAA identified a number of benefits that could not be quantified because of

the lack of verifiable data but that are nonetheless important for

decisionmakers to consider when choosing an investment option. For

example, OMB guidance notes that a comprehensive evaluation of the

different types of benefits and costs, quantified or not, can be helpful in

identifying the full range of program effects. We have also reported that

estimates of benefits are typically uncertain because of imprecision in both

underlying data and modeling assumptions, and it is appropriate to

consider nonquantified benefits when choosing investment options.

Industry representatives share FAA’s views on the nonquantified benefits

Dollars in millions

With passenger value of

time

Without passenger value of

time

Alternatives

Benefit-cost

ratio Net benefits

Benefit-cost

ratio Net benefits

I. No WAAS; No LAAS 1.5 $280 0.5 ($287)

II. Simplified WAAS, With

LAAS

1.0 $94 0.5 ($1,007)

III. Less Robust WAAS,

with LAAS

2.1 $1,857 0.9 ($25)

IV. Robust WAAS with

LAAS While Retaining 30

Percent of Ground

Infrastructure

2.4 $2,469 1.1 $72
Appendix III

FAA Identified Four Alternatives for a

Benefit-Cost Analysis

Page 36 GAO/RCED/AIMD-00-130 National Airspace System

the agency has identified, and some believe that these and other

nonaviation benefits could be substantial if they could be quantified.

Despite these strengths, we found that one key benefit—FAA’s estimate of

the $4.4 billion in savings achieved by allowing planes to decrease the time

needed for an approach to landing—was based largely on professional

judgment, without empirical data. This benefit accounted for nearly 42

percent of all quantified benefits. Specifically, FAA assumed that 40 percent

of all flights during nonpeak periods could decrease approach time into

airports, even though it recognized that the percentage could range from 20

percent to 50 percent during nonpeak periods. Because of the lack of

empirical data, we asked FAA to perform a more conservative analysis,

assuming that only 20 percent of the flights would benefit.We found that

this resulted in a benefit-cost ratio of 1.9 if passenger value of time was

counted and a ratio of .8 if not counted.
Page 37 GAO/RCED/AIMD-00-130 National Airspace System

Appendix IV

Comments From the Department of Defense ApendxIVi
Page 38 GAO/RCED/AIMD-00-130 National Airspace System

Appendix V

GAO Contacts and Staff Acknowledgments ApendxVi

GAO Contacts Gerald L. Dillingham, (202) 512-2834

Belva M. Martin, (202) 512-2834

Staff

Acknowledgments

In addition to those named above, Charles W. Bausell, Jennifer W.

Clayborne, Richard B. Hung, Peter G. Maristch, John T. Noto, Madhav S.

Panwar, Karen A. Richey, and Carol Herrnstadt Shulman made key

contributions to this report.

(348184) Letr
Ordering Information The first copy of each GAO report is free. Additional copies of

reports are $2 each. A check or money order should be made out to

the Superintendent of Documents. VISA and MasterCard credit

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any list

from the past 30 days, please call (202) 512-6000 using a touchtone

phone. A recorded menu will provide information on how to obtain

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet,

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at:

http://www.gao.gov

To Report Fraud,

Waste, or Abuse in

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00