National Airspace System: Status of Wide Area Augmentation System Project
Published: 1998-04-30T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a legislative requirement, GAO reviewed the status of the Federal Aviation Administration's (FAA) Wide Area Augmentation System (WAAS) project, focusing on: (1) whether the Secretary of Transportation's report provides a complete assessment of FAA's risks in developing the WAAS project; and (2) how alternative assumptions would affect WAAS' benefit-cost analysis of January 1998.
section_title: What GAO Found
paragraphs: GAO noted that: (1) the Secretary's report provided a complete assessment of FAA's risks in achieving the WAAS project's performance and cost goals but not its scheduled goals; (2) in terms of system performance, the Secretary's report recognized that WAAS' vulnerability to intentional or unintentional interference from electronic equipment must be addressed; (3) in January 1998, FAA estimated that it would save about $500 million (in net present value) over the WAAS project's life cycle by fully phasing out its network of ground-based navigation aids; (4) if FAA retains some portion of this network, these benefits would decrease; (5) FAA also estimated that aircraft operators could save $350 million by removing ground-based navigation equipment from their aircraft; (6) these benefits would be reduced to the extent that operators must continue to keep such equipment on board; (7) by identifying a range of cost estimates and associated probabilities, the Secretary's report addressed GAO's past concern that FAA's firm, discrete-point cost estimates implied a level of precision that could not be supported; (8) GAO agreed with the Secretary's report that the greatest degree of uncertainty about the WAAS cost estimates relates to the costs of the geostationary communications satellites; (9) the uncertainty exists because FAA does not know exactly how many satellites will be needed and how much the per-unit costs will be; (10) the Secretary's report fell short of providing a complete assessment of the uncertainties FAA faces in achieving WAAS' schedule goals; (11) the report also did not discuss the risks to the overall schedule if FAA does not award the contract to lease the satellites by July 1998 as planned; (12) in January 1998, FAA's analysis found that the benefits to aviation from WAAS would be three times as great as its costs; (13) GAO requested that FAA recalculate its benefit-cost analysis to determine the impact of three alternative assumptions; (14) using these cost and decommissioning assumptions did not cause much of a decrease in the benefit-cost ratio or the net benefits; (15) however, the exclusion of small increments of passenger time savings had a much more significant impact; and (16) when these alternative assumptions were taken together, GAO found that the net present value of the project's net benefits decreased by more than $1 billion but were still about twice as great as the costs.

section_title: Letter
paragraphs: As a key element of its overall program for modernizing the National Airspace System, the Federal Aviation Administration (FAA) is planning a transition from ground- to satellite-based navigation by using satellite signals generated by the Department of Defense’s (DOD) Global Positioning System (GPS). However, GPS by itself does not satisfy all civil air navigation requirements, such as the one requiring that aircraft operators be provided timely warnings of system malfunctions and another requiring that the system be available virtually all of the time. FAA is developing a network of ground stations and geostationary communications satellites to enhance GPS so that satellite-based navigation can meet those requirements. This network is known as the Wide Area Augmentation System (WAAS). The system is expected to improve the safety of flight operations, allow the fuel-efficient routing of aircraft, and enable FAA to eventually phase out its network of ground-based navigation aids.
In light of concerns about the WAAS project, the conferees for the Department of Transportation Appropriations Act for fiscal year 1998 and the Senate Appropriations Committee placed reporting requirements on the Secretary of Transportation and on us. The Secretary was directed to report on the status and management of the project, including an identification of baseline performance, cost, and schedule goals and to provide a risk assessment. The Secretary’s report was issued on February 11, 1998. We were directed to review the status of the WAAS project and report by March 1, 1998. As agreed with your offices, we delayed the issuance of our report until we had an opportunity to review the Secretary’s report. We examined whether the Secretary’s report provides a complete assessment of FAA’s risks in developing the WAAS project. We also examined how alternative assumptions would affect WAAS’ benefit-cost analysis of January 1998.
subsections: section_title: Background
paragraphs: In the 1980s, FAA began considering how a satellite-based navigation system might eventually replace the ground-based system that has long provided navigation guidance to aircraft. In August 1995, after several years of research, FAA contracted with Wilcox Electric to develop WAAS to enhance GPS. However, because of concerns about the contractor’s work, FAA terminated the contract in April 1996. In May 1996, the agency entered into an interim contract with Hughes Aircraft Company (now Raytheon Systems), with the contract becoming final in October 1996.
Accuracy, integrity, availability, continuity, and service volume are the major performance goals for the system to meet. Accuracy is defined as the degree to which an aircraft’s position as calculated using the system conforms to its true position. For precision approaches to runways, WAAS is expected to provide aircraft operators with position accuracy within 7.6 meters 95 percent of the time. Integrity is the system’s ability to provide timely warnings when its signals are providing erroneous information and, thus, should not be used for navigation. WAAS is expected to provide a warning to aircraft operators within 5.2 seconds. Availability is the probability that, at any given time, the system will meet FAA’s accuracy and integrity requirements for a specific phase of flight. For precision approaches, WAAS is expected to be available all but 9 hours per year. Continuity is the probability that the system’s signal will meet accuracy and integrity requirements continuously for a specified period. Service volume is the area of coverage for which the system’s signal will meet availability requirements.
As shown in figure 1, WAAS is a network of ground stations and geostationary (GEO) communications satellites:
Reference stations on the ground (up to 53 units) will serve as the primary data collection sites for WAAS. These stations will receive data from the GPS and GEO satellites.
Master stations on the ground (up to seven units) will process the data collected by the reference stations and generate accuracy corrections and integrity messages for each of the GPS and GEO satellites. These stations will also validate the transmitted corrections.
Ground earth stations (up to 14 units) will, among other things, transmit accuracy corrections and integrity messages generated by the master stations to FAA’s GEO satellites.
GEO satellites (up to six satellites) will transmit wide-area accuracy corrections and integrity messages to aircraft and also broadcast signals that will be similar to the signals broadcast by the GPS satellites.
A ground communications system will transmit information among the reference stations, master stations, and ground earth stations.
For pilots to use WAAS for navigation, their aircraft will have to be equipped with receivers that process the information carried by the GPS and GEO signals. The receivers will enable the pilots to determine the precise time and the speed and three-dimensional position (latitude, longitude, and altitude) of their aircraft.
By July 30, 1999, FAA expects that WAAS’ initial operational capability will be available for pilots’ use. At that time, WAAS is expected to support aircraft navigation for all phases of flight. However, the initial system will not contain all the required hardware and software components needed for (1) redundancy in the event of equipment failures and (2) availability for the nation’s entire airspace. By December 2001, FAA plans to develop a fully operational WAAS by adding reference stations and upgrading software under the Raytheon contract and adding GEO satellites under a separate contract. The full system is expected to be capable of eventually serving as a “sole means” navigation system. That is, the system must, for a given operation or phase of flight, allow the aircraft to meet all navigation system performance requirements.
section_title: Performance and Cost Risks Disclosed, but Schedule Uncertainties Not Fully Recognized
paragraphs: The Secretary’s report provided a complete assessment of the major risks FAA faces in achieving the technical performance goals of the WAAS project. It also disclosed the cost uncertainties and the range and probabilities of potential costs. However, the report could have done more to disclose the uncertainties associated with FAA’s schedule for making WAAS fully operational.
subsections: section_title: Performance Risks Disclosed
paragraphs: In discussing the risks FAA faces in developing WAAS, the Secretary’s report highlighted the vulnerability of the system’s signals to intentional or unintentional interference from electronic equipment. It also discussed mitigation strategies, including the possibilities of an independent backup system and full access to a second GPS frequency.
Concerns about the system’s vulnerability to electronic interference have been highlighted in recent months. In an October 1997 report, the President’s Commission on Critical Infrastructure Protection warned against relying on satellite navigation as the sole source of aircraft landing guidance in light of potential interference. That same month, a group of independent experts from outside FAA, called together by the agency’s management to study the technical issues facing WAAS, raised concern about the possible intentional jamming of the signals. In February 1998, the FAA Administrator’s task force on the National Airspace System’s modernization recommended that FAA address the risks posed by electronic interference and gain consensus among users of the system about the agency’s plan to switch from ground- to satellite-based navigation.
The Secretary’s report recognized that WAAS’ vulnerability to interference must be assessed and appropriate countermeasures must be in place before FAA can complete the transition to a satellite-based navigation system. The report cited several elements of a risk mitigation plan. For example, FAA has developed procedures for reporting and responding to interference that include outfitting flight inspection aircraft with the capability to locate sources of interference.
FAA may employ other risk mitigation strategies as well. One is the retention of an independent backup system. The Secretary’s report noted that FAA is studying the need for such a system, and if the need for a backup is established, the agency would evaluate various alternatives. While the backup system would not have to provide aircraft operators with the same operational capability as WAAS or the current ground-based system, it would have to provide, at a minimum, safe navigation in the event of a loss of service from WAAS. Rather than designating WAAS as a sole means navigation system, FAA may designate it initially as a “primary means” system until concerns about electronic interference are resolved. This means that WAAS would not be expected to fully meet all availability and continuity requirements for navigation. As a result, aircraft operators would either have to be equipped with a backup navigation system or have restrictions on when and where they could fly.
FAA and Mitre Corporation officials told us that if an independent backup system is retained, FAA may decide to deploy fewer WAAS reference stations and satellites. In making this decision, the agency would consider whether civil air navigation requirements could be met more cost-effectively with a combination of an independent backup system and WAAS with fewer reference stations and satellites.
Even if WAAS remains unmodified, the system’s benefits to FAA and aircraft operators could be expected to decrease if some portion of the current ground-based network is retained as an independent backup system. FAA has intended to decommission its entire network of ground-based navigation aids between 2005 and 2010—with the phaseout concentrated toward the end of that period. In January 1998, FAA found that full decommissioning would result in the agency’s saving about $500 million (in net present value) over WAAS’ life cycle. The agency also expected aircraft operators to be able to reduce the proliferation of on-board navigation equipment. The benefit-cost analysis estimated that the operators would save about $350 million by removing such equipment.
Another risk mitigation strategy to counteract WAAS’ vulnerability to electronic interference (particularly unintentional interference) is the use of a second frequency. If one GPS frequency was lost because of interference, a second frequency could be used to provide service.However, the current WAAS design assumes the use of single-frequency receivers on board aircraft. The Department of Transportation (DOT) and DOD, as joint chairs of the Interagency GPS Executive Board, are working toward providing aviation and other civil users with full access to a second GPS frequency on the next generation of GPS satellites.
Although the second civil frequency would not be fully operational on GPS satellites until about 2010, FAA would prefer to build WAAS ground- and space-based equipment so that users could operate with “forward compatible” receivers—that is, receivers that can be built to operate with a single frequency now and also operate with dual frequencies in the future. Once a final decision on the second frequency is made, FAA and industry will need up to 2 years to develop the minimum operational performance standards so that manufacturers can begin producing receivers capable of single- and dual-frequency operations.
section_title: Cost Risks Identified
paragraphs: In 1997, we expressed our concern that FAA’s cost estimates for WAAS were firm, discrete-point estimates, implying a level of precision that could not be supported, particularly early in the project’s development. The Secretary’s report addressed this concern by identifying a range of possible costs and associated probabilities.
The Secretary’s report stated at a high confidence level (an 80-percent probability) that WAAS’ 15-year life-cycle cost will not exceed about $3 billion. Overall, this estimate is $600 million higher than the agency’s September 1997 estimate. FAA attributes this increase to the costs of leasing additional GEO satellites being higher than expected. (See table 1.)
We agree with the Secretary’s report that the greatest degree of uncertainty about the WAAS cost estimates surrounds the costs of the satellites. The uncertainty exists because FAA does not yet know exactly how many additional satellites will be needed and how much the per unit costs will be. The Secretary’s report also states at a high confidence level that the operations and maintenance cost of satellites will be no more than about $1.2 billion, or about 40 percent, of the project’s total cost of $3 billion. This estimate includes about $200 million for the cost of maintaining the leases on the two existing satellites for which FAA currently contracts with Comsat and $1 billion for leasing additional satellites. FAA’s cost estimate assumes that the two satellites leased from Comsat will be retained and two to four additional satellites (with three being the most probable number) will be obtained. The annual unit costs for the added satellites range from about $12 million to $25 million (with $17 million being the most probable cost).
The uncertainty surrounding GEO satellite costs is likely to be reduced as more data become available. FAA intends to make a decision on the number of satellites needed for the full WAAS after determining the placement of satellites in space and how well the GPS satellites are performing. The agency will know what the per unit costs will be after it comes to an agreement with a vendor for satellite services. On January 8, 1998, FAA issued a request for information seeking input from vendors that would be willing to finance the costs of designing, building, and launching the GEO satellites. FAA would commit to a multiyear lease for satellite services and reimburse the vendor for its costs. According to the Secretary’s report, the agency has targeted April 1998 for issuing a request for proposals to solicit vendors’ bids and July 1998 for awarding a contract.
Two types of leases are potentially applicable to FAA’s satellite leasing strategy: the operating lease and the capital lease. An issue to be resolved is how budget authority for the satellite leasing costs will be scored. According to the scorekeeping guidelines contained in the Conference Report for the Budget Enforcement Act of 1997, operating leases for physical assets are primarily intended to meet short-term capital needs and are to be used to obtain general purpose equipment (that is, equipment not built to meet a unique government specification or need) and equipment that has a private sector market. Capital leases for physical assets, on the other hand, are intended to be generally longer term and used to obtain equipment built to meet unique government-specified needs or uses and leased to the government for most of its useful economic life.
FAA’s satellite lease would likely be scored as an operating lease if FAA signs a long-term lease through which the agency leases space on “hosted” GEO satellites. In other words, FAA’s WAAS payload would share space on satellites with other users. Scorekeeping guidelines require that an agency have sufficient budget authority to cover at least the cost of the first year of the contract plus any cancellation fees. According to WAAS’ funding profile, FAA expects no leasing costs in fiscal years 1999, 2000, and 2001 if the vendor agrees to cover the costs of building and launching the satellites and to wait until 2002 for FAA’s first payment on the contract. FAA’s Assistant Chief Counsel, Procurement Law Division, told us that the agency may enter into contracts without budget authority for its cancellation fees because FAA has multiyear contracting authority that exempts it, under certain conditions, from the Anti-Deficiency Act.
FAA’s satellite lease would likely be scored as a capital lease if FAA signs a long-term lease for “dedicated” satellites that would be built to meet WAAS’ specifications and used primarily, if not exclusively, for WAAS’ operations. In that case, scorekeeping guidelines require enough up-front budget authority to reflect the estimated net present value of the entire lease, about $290 million, in fiscal year 1999, the first year of the contract.Congressional approval of this amount would result in less budget authority being available for other programs funded through the appropriations process in that fiscal year.
section_title: Schedule Uncertainties Not Fully Recognized
paragraphs: Although the Secretary’s report discussed risk factors that could affect the achievement of FAA’s schedule goals for developing WAAS, it fell short of providing a complete assessment. For example, while it assigned a 99-percent degree of confidence in meeting various milestones during fiscal year 1998, the report did not assign probabilities for milestones for fiscal year 1999 and beyond. The agency has set schedule goals for the development of the initial and full system but has provided no range or confidence levels for achieving those goals.
The conferees for the DOT Appropriations Act for fiscal year 1998 required the Secretary of Transportation to provide by February 15, 1998, a detailed report on FAA’s plans to provide satellite communications for WAAS.According to the Secretary’s transmittal letter to the Congress, his Department’s report of February 11 included these plans. In our view, however, the report could have done more to discuss the uncertainties FAA faces in obtaining the required GEO satellites.
As already noted, FAA released a request for information from satellite vendors on January 8 and has evaluated this information. According to the Secretary’s report, the agency expects to issue a request for proposal by April 1998, award a contract to a satellite provider by July 1998, and complete the launching and testing of the satellites by October 2001. By December 2001, only 2 months later, WAAS is scheduled to become fully operational.
One major uncertainty is whether FAA will find a vendor willing and able to complete the launching and testing of the satellites by October 2001. In responding to the January 8 information request, a number of potential vendors pointed to 2002 or 2003 as a more realistic schedule for putting the satellites in orbit.
If the GEO satellites are launched after 2001, the resulting delay would be likely to have implications for the project’s benefits and costs. Benefits would decrease, for example, because users would not have a fully operational system available for navigation as early as expected. Aircraft operators would not realize some portion of the $350 million (in net present value) that FAA estimates operators would save by removing ground-based navigation equipment from their aircraft. At the same time, the project’s costs would be likely to increase. In April 1998, FAA’s WAAS program office estimated that a 12-month delay would cost an additional $6 million. This amount would be needed to pay Raytheon to retain a core staff of system engineers to complete the integration and testing of the GEO satellites.
Another major uncertainty centers on the time needed to award a contract for the satellites. If the satellite contract is not awarded by July 1998 as planned, the remainder of the schedule is likely to slip. Contract award by that date, however, is doubtful for two reasons. First, negotiations over the terms of the contract might become protracted as FAA and the vendor seek to minimize their financial risks. For example, FAA expects the vendor to invest hundreds of millions of dollars to cover the costs of building and launching the satellites. However, while FAA expects to pay a premium for the vendor to finance the satellite costs, the vendor may not wish to carry the costs until FAA begins paying, as planned, in fiscal year 2002—more than 3 years after the contract is awarded. FAA and the vendor will be negotiating on the extent of the government’s financial guarantees. These guarantees are likely to take the form of cancellation fees that FAA would pay in the event the contract is terminated.
Second, FAA may defer contract award until it receives congressional approval to enter into a 10-year lease for the GEO satellites. Under 49 U.S.C. 40111 and 40112, the agency is currently limited to contracts with a 5-year base period with 3 option years. To reduce its costs for the satellite lease, FAA would like to be able to extend the satellite leasing period from 5 years to 10 years and intends to seek the authority to enter into multiyear contracts for an unlimited number of years.
section_title: Under Alternative Benefit and Cost Assumptions, WAAS Remained Cost-Beneficial, but Net Benefits Were Reduced
paragraphs: In making investment decisions, FAA conducts benefit-cost analyses to determine if the benefits to be derived from acquiring new equipment outweigh the costs. FAA’s analyses dating back to 1994 have always found WAAS to be a cost-beneficial investment—that is, the benefits clearly exceeded the costs. (See app. I for details on FAA’s benefit-cost analyses for the WAAS project in 1994, 1996, 1997, and 1998.)
In FAA’s benefit-cost analyses, the costs for WAAS included the future life-cycle costs for facilities and equipment as well as operations and maintenance costs and the costs for decommissioning the current ground-based navigation aids, such as very high frequency omnidirectional ranging (VOR) units. The system’s benefits to FAA included the savings from reduced maintenance of the navigation aids that are to be decommissioned and the avoidance of capital expenditures for replacing those aids with new ground-based equipment. Aircraft operators—the users of WAAS—also benefit. The users’ benefits included the reduction of accident-related costs (from death, injury, and property damage) because the system’s landing signals would be available at airports or runways that currently lack precision landing capability. Also, aircraft operators could benefit by reducing the proliferation of on-board navigation equipment and receiving savings that result from the shorter flight times on restructured, more direct routes that aircraft could fly using WAAS. Shorter flight times from these more direct routes also benefit passengers. Nonaviation benefits were excluded from FAA’s analyses.
FAA’s investment analysis group prepared the agency’s most recent benefit-cost analysis, in January 1998, to assist FAA in evaluating whether WAAS was a sound investment. Unlike previous analyses, FAA’s January 1998 analysis used a risk assessment methodology that recognized uncertainties and placed confidence levels on each outcome. The base case analysis assumed that the two existing satellites will continue to be leased throughout the WAAS life cycle and that additional dedicated satellites will be necessary according to the following probabilities: two more satellites, 20-percent probability; three more satellites, 65-percent probability; and four more satellites, 15-percent probability. The base case analysis also assumed that there is a 100-percent probability that all ground-based navigation systems will be decommissioned by 2010. In its analysis, FAA also included the value of the time passengers would save, assuming a range of savings that generally varied from about 20 to 60 seconds, with the most probable amount being 30 seconds, in calculating the benefit-cost ratios.
This analysis found (1) a 20-percent chance (the low confidence level) that the WAAS benefit-cost ratio could be 4.0 or greater and (2) an 80-percent chance (the high confidence level) that the ratio could be 3.0 or greater.Expressed another way, the net benefits (dollar value of benefits minus costs) of WAAS were $3.4 billion or greater at the low confidence level and $2.4 billion or greater at the high confidence level.
As discussed previously, it is possible that satellite costs could increase and that FAA would decide to retain some of its ground-based navigation systems. To understand the impact of these possibilities, we asked FAA’s investment analysis group to perform alternative runs of their benefit-cost analysis using the methodology that they followed. The scenarios we requested made the following assumptions: a 20-percent probability that the two existing leased satellites will continue to be leased throughout WAAS’ life cycle and an 80-percent probability that they will be replaced with one, more expensive, dedicated leased satellite; a 50-percent probability that three additional satellites will be needed and a 50-percent probability that four additional satellites will be needed; and a 50-percent probability that 125 VOR units will never be decommissioned and a 50-percent probability that 650 VOR units will never be decommissioned.
DOT’s guidance, dated April 9, 1997, directs departmental staff to include passenger time savings in benefit-cost analyses. The guidance notes that a controversy exists over whether small increments of time savings, such as a few minutes or less, should be valued at the same hourly rate as larger increments. However, it concludes that assuming “a constant value per hour for large and small time savings is probably appropriate.” The Director, FAA’s Office of Aviation Policy and Plans, told us that while only small increments of passenger time savings may result from any one FAA project, more significant—and clearly valuable—time savings may result from aggregating the small increments. Because FAA develops and implements many aviation projects over a number of years, the agency would not know the total impact of these projects on passenger time savings unless all increments were captured in its benefit-cost analyses. An official of the Office of Management and Budget (OMB) told us that her office does not provide specific guidance to federal agencies about the valuation of small increments of passenger time savings. She said that while OMB has not formally endorsed DOT’s April 1997 guidance, OMB’s staff do not have any major concerns with it.
Concerned that passengers might not perceive and value time savings of as little as 30 seconds, we reviewed the economic literature about the validity of using small increments of time and found that no consensus exists.(See app. II for a discussion of the literature.) In the absence of a consensus among experts, we requested that FAA’s investment analysis group perform an alternative run of its January 1998 benefit-cost analysis base case excluding the value of small increments of passenger time savings.
The results shown in table 2 reflect the use of the alternative assumptions compared with those in FAA’s 1998 base case analysis. We found that our alternative cost and decommissioning assumptions alone did not cause much of a decrease in the benefit-cost ratios and net benefits. Excluding small increments of passenger time savings caused a more pronounced decrease. For example, we found that at the high confidence level, net benefits declined by only $0.2 billion—from $2.4 billion or greater using FAA’s base case assumptions to $2.2 billion or greater using our alternative cost and decommissioning assumptions. However, the exclusion of small increments of passenger time savings alone led to a $1 billion decline in net benefits. Nevertheless, when the alternative assumptions are taken together, the system’s benefits still exceed the costs by nearly a 2-to-1 ratio.
section_title: Conclusions
paragraphs: The Secretary’s report adequately discussed the risks FAA faces in achieving the performance and cost goals for the WAAS project. It could have done more, however, to recognize the schedule uncertainties, particularly those related to obtaining the GEO satellites.
More information would help the Congress and the administration in deciding on future investments in the WAAS project. Information on the range of milestones for making the system operational and the probabilities attached to those milestones would aid decisionmakers in determining the timing of the investments. Also, a detailed explanation of FAA’s strategy for leasing GEO satellites would help them in understanding the cost and budgetary implications. Particularly useful would be information on (1) the cost-effectiveness of the hosted and dedicated satellite options and (2) the estimated premium to be paid for a vendor’s financing of the building and launching of the satellites.
Even under our alternative assumptions, WAAS’ benefits clearly outweigh its costs. However, the continued investment in WAAS must compete with other demands on FAA’s capital and operating budgets. When more is known about the likely costs for obtaining GEO satellites and the extent to which the agency may retain existing ground-based navigation aids, an updated benefit-cost analysis would help the Congress and administration in making future investment decisions. The analysis would be more useful if the agency compared an investment in WAAS with alternative uses of FAA’s resources and explained the effects of including small increments of passenger time savings on the benefit-cost ratio and net benefits of the system.
section_title: Recommendation
paragraphs: To assist the Congress in making future funding decisions for the Wide Area Augmentation System project, we recommend that the Secretary of Transportation direct the FAA Administrator to report information to the Congress on the range of milestones for making the initial and full Wide Area Augmentation System operational and the probabilities associated with those milestones; a detailed explanation of the agency’s strategy for leasing geostationary updated benefit-cost analyses, including a comparison with alternative investments of FAA’s resources and an explanation of the effects of including small increments of passenger time savings.
section_title: Agency Comments
paragraphs: We provided a draft of this report to the Departments of Transportation and Defense for review and comment. We met with officials from the Office of the Secretary of Transportation and FAA, including the Director, Communications, Navigation, and Surveillance (CNS) Systems; the Chairman, Satellite Operational Implementation Team; the WAAS Program Manager; and the Manager, CNS/Facility Investment Analysis. We also spoke with the Assistant for GPS, Positioning, and Navigation Policy, Office of the Deputy Under Secretary of Defense (Space). DOT and DOD generally agreed with our draft report’s findings, conclusions, and recommendations. They gave us information and suggestions to help make the report clearer and more accurate. We incorporated their suggestions where appropriate.
DOT expressed concern that the wording in our draft report could leave the impression that we believe FAA improperly calculated the WAAS benefit-cost ratio because of the inclusion of small increments of passenger time savings. The agency noted that DOT’s guidance directs departmental staff to include all increments—large and small. We did not intend to suggest that FAA should not follow DOT’s guidance, and we have added language to the report to clarify this. However, our review of the economic literature found a lack of consensus among experts on the validity of using small increments of passenger time savings and our sensitivity analysis found that the inclusion of small increments was significant for WAAS’ benefit-cost ratio and net benefits. Taken together, these findings argue for informing decisionmakers about the effects of including small increments of passenger time savings when future benefit-cost analyses are conducted for the WAAS project.
section_title: Scope and Methodology
paragraphs: To obtain information for this report, we interviewed (1) officials at FAA headquarters and DOD, including DOD’s National Reconnaissance Office; (2) representatives from Raytheon (previously Hughes Aircraft), the prime contractor on WAAS; and (3) officials from the Mitre Corporation who provide technical advice to FAA. We reviewed agency documentation on the current schedule, life-cycle costs, and performance goals for WAAS. We also reviewed technical reports from the WAAS contractor and outside experts that discussed the risks and challenges facing the project. To identify the potential impact of differing assumptions on the benefit-cost ratio for WAAS, we asked FAA to run alternative analyses.
We performed our review from October 1997 through April 1998 in accordance with generally accepted government auditing standards. We did not assess the reliability of all cost information. However, with regard to satellite costs—the major cost item contributing to increased life-cycle costs—we did satisfy ourselves that the information being used by FAA is in general agreement with those estimates provided by outside sources, such as DOD. Also, while we did not perform an extensive review of the model used to calculate benefit-cost ratios, the model used by FAA is widely recognized as an appropriate economic analysis tool for providing risk-adjusted benefit-cost ratios.
We are sending copies of this report to interested congressional committees, the Secretaries of Transportation and Defense, and the Administrator of FAA. We will also make copies available to others on request.
If you or your staff have any questions or need additional information, please call me at (202) 512-3650 or send email to dillinghamg.rced@gao.gov. Major contributors to this report are listed in appendix III.
section_title: Results of FAA’s Benefit-Cost Analyses, 1994, 1996, 1997, and 1998
paragraphs: The results of the Federal Aviation Administration’s (FAA) benefit-cost analyses of the Wide Area Augmentation System (WAAS) project in 1994, 1996, 1997, and 1998 are summarized in table I.1. On the benefit side, benefits to the government accrue from the reduced maintenance of the existing, ground-based network of navigation aids and the avoidance of capital expenditures for replacing these aids. Benefits to users—the aircraft operators—fall into five categories:
Efficiency benefits derive from having precision landing capability at airports where it does not now exist.
Avionics cost savings reflect how WAAS will enable users to reduce the proliferation of avionics equipment in their cockpits.
Fuel savings reflect the use of less fuel to fly aircraft that carry less avionics equipment.
Safety benefits stem from the reduction in accident-related costs (death, injury, and property damage) because of the availability of WAAS landing signals at airports that presently lack a precision landing capability.
Direct route savings result from the shorter flight times associated with restructured, more direct routes that aircraft can fly because of WAAS.
Table I.1: FAA’s Analysis of the Net Present Value of Benefits and Costs for the WAAS Project, 1994, 1996, 1997, and 1998 Not applicable.
FAA’s September 1997 benefit-cost analysis took a more conservative approach than previous analyses in estimating the benefit-cost ratio. That is, compared with the previous analyses, the assumptions underlying the September study increased the expected costs of WAAS and simultaneously reduced the expected benefits, which resulted in a lower benefit-cost ratio than found in the previous versions of the study. The higher costs in the 1997 analysis were largely due to the inclusion of the costs of decommissioning ground-based navigation systems that were not included in any earlier versions of the study. On the benefit side, several changes in key assumptions led to reduced expected benefits, including (1) a shorter life cycle for the project, (2) a reduction in the assumed “saved” costs from phasing out ground-based navigation systems, (3) a reduction in estimated safety benefits based on the use of more recent accident data,and (4) a reduction in the expected flight time savings resulting from more direct routes. In addition, the high benefit-cost ratio in the 1997 analysis included passenger time savings. The low benefit-cost ratio in the 1997 analysis excluded passenger time savings.
The January 1998 analysis used a different methodology in calculating benefit-cost ratios than did the 1997 analysis. The 1998 analysis assessed how multiple events, such as a combination of numbers of satellites and ranges of satellite costs, could affect the benefit-cost ratio. This analysis then produced benefit-cost ratios at high and low confidence intervals. That is, at the high confidence level, there is an 80-percent chance that the benefit-cost ratio could be between 3.0 and 1.0. Conversely, at the low confidence level, there is a 20-percent chance that the benefit-cost ratio could be between 4.0 and 1.0. This analysis did not, however, exclude passenger time savings. The 1997 analysis assessed how individual events, such as increased satellite costs, could affect the benefit-cost ratio. While this analysis did not assign confidence levels to its benefit-cost ratios in arriving at the 5.2 high estimate and the 2.2 low estimate, it did exclude passenger time savings in the low estimate.
section_title: Issues Regarding the Valuation of Small Increments of Passenger Time Savings
paragraphs: A sizable portion of the calculated benefits of WAAS are from the time aviation passengers are expected to save once the system is in place. However, most of these savings come in small increments of time—a minute or less per passenger trip. Concerned that passengers might not perceive and value time savings of a minute or less, we requested, as a sensitivity analysis, alternative runs of the WAAS benefit-cost analysis that excluded these passenger time savings. We made this request because we found that there is considerable controversy and no consensus in the economic literature about whether travelers perceive and value very small time savings. We do not suggest that these benefits should be excluded from the benefit-cost analysis of WAAS or that FAA should undertake an analysis that is not in accordance with Department of Transportation’s guidance that directs its staff to include small increments of passenger time savings in benefit-cost analyses. However, we believe it is useful to understand how sensitive the benefit-cost results were to the inclusion of small increments of passenger time savings.
This appendix provides information on (1) the value of passenger time in the WAAS benefit-cost analysis and (2) the issues discussed in the economics literature on the value of very small increments of time to travelers.
subsections: section_title: A Significant Portion of the Calculated Benefits of WAAS Are Attributable to Passenger Time Savings
paragraphs: FAA’s most recent WAAS benefit-cost analysis found, as is the case for many transportation improvement projects, considerable benefits attributable to reduced travel time for passengers. In the case of WAAS, about 40 percent of the calculated benefits, or approximately $1 billion in the base case benefit-cost analysis, are due to time savings that would accrue to travelers because of slightly reduced flight times. FAA officials told us that, on average, these time savings would probably be about 30 seconds per flight. FAA’s guidance regarding the valuation of small increments of passenger time savings suggests that there is no reason, based on either empirical findings or theoretical concepts, that these small increments should not be valued at the same per hour rate as larger increments of time savings.
section_title: Few Empirical Studies on Small Increments of Passenger Time Exist; Conceptual Arguments Are Contradictory
paragraphs: We reviewed several studies, including an overview study prepared for FAA, on the issue of the value of small increments of passenger time. As FAA’s analysis points out, there is limited empirical work on this issue. Several studies from the 1970s suggest that travelers may place little value on very small time savings, such as 1 minute. However, the findings of these studies may have limited applicability to WAAS. First, these studies, and most other analyses of this issue, are focused on intracity commuter travel. The nature and characteristics of such travel are very different from intercity air travel and, accordingly, results from the studies may have little applicability to how intercity air travelers value time. Secondly, the study prepared for FAA discusses the considerable methodological problems and limitations in these empirical studies. Because of these problems, and the lack of studies focused on air travel, the empirical literature does not provide definitive evidence about how small increments of time savings are valued by travelers who would benefit from WAAS.
Our review of the conceptual arguments regarding the value of small increments of passenger time revealed a mixed message. There appear to be sound conceptual points on both sides of this debate: Some suggest that small increments of passenger time savings should be valued on a pro rata basis just as larger increments of time are; others suggest that less value should be placed on very small time savings.
Those who argue that small time savings have little value suggest some key reasons. First, people cannot perceive very small time savings, and if they cannot perceive them, they do not value them. Second, even if a savings of, for example, 1 minute is perceived, it will not be of value to a person unless that time can be put to some alternative use. Because it is likely to take some threshold amount of time to have value in an alternative use, very small increments of time cannot be used and are therefore not valued. Moreover, as the amount of time savings increases, more potential uses of that time become available.
Conversely, several conceptual arguments suggest that there is no basis for valuing small time savings at less than their pro rata share of the value of larger time savings. First, some analysts have suggested that even if people do not perceive a time savings, they do place value on it if they put that time to an alternative use. Additionally, the issue of needing a threshold block of time for an alternative use may be true, but this would suggest that people may always have some “spare” time that cannot be used, and if so, very small increments of time may, in some cases, push them over the threshold level and give them a usable block of time. This not only suggests that small increments of time may, in some cases, have considerable value, but it also points out that even spare, or unusable, time will be valued because there is the possibility of time savings from some other source that will meet the threshold for a usable time block. The final argument for valuing even small time increments is that transportation improvement initiatives are somewhat arbitrarily divided into recognized “projects.” That is, across both time and geography, a variety of projects may be providing incremental time savings that may each be only a small amount, but when added together become significant. Hence, it is not appropriate to view the savings of a given project in isolation of other projects that might occur a year later or at a different location.
section_title: Major Contributors to This Report
paragraphs: Amy D. Abramowitz Leslie Albin John H. Anderson, Jr. Robert E. Levin John T. Noto E. Jerry Seigler The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 37050 Washington, DC  20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000  or by using fax number (202) 512-6061, or TDD (202) 512-2537.
Each day, GAO issues a list of newly available reports and testimony.  To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone.  A recorded menu will provide information on how to obtain these lists.