STARS is designed to replace FAA's automated radar terminal system, which is composed of 15-to 25-year-old controller workstations and supporting computer systems. According to FAA, this system is prone to failures, is maintenance intensive, and requires long repair times. The system also has capacity constraints that restrict the agency from making required safety and efficiency enhancements. Automated radar terminal systems are located at 180 Terminal Radar Approach Control facilities (TRACON) and allow FAA controllers to separate and sequence aircraft near airports. STARS equipment (see fig. 1) is also expected to provide the platform needed to make system enhancements that would increase the level of air traffic control automation and improve weather display, surveillance, and communications. In addition, STARS is expected to permit FAA to consolidate some TRACONs and replace all Digital Bright Radar Indicator Tower Equipment systems. In September 1996, FAA signed a contract with Raytheon Corporation and, as mentioned, currently plans to acquire as many as 171 STARSs. In producing STARS, Raytheon intends to rely fully on commercially available hardware and, to a large extent, on commercially available software. Some original software development will still be required. In August 1996, the contractor projected that 124,000 new lines of software code will need development to meet FAA's requirements. This estimate was revised in December 1996 to 140,000 new lines of code. STARS is an outgrowth of the troubled Advanced Automation System acquisition. As originally designed, the terminal segment of this system, known as the Terminal Advanced Automation System, would provide controllers in TRACONs with new workstations and supporting computer systems. However, in June 1994, the FAA Administrator ordered a major restructuring of the acquisition to solve long-standing schedule and cost problems. These schedule delays were up to 8 years behind the original schedule, and estimated costs had increased to $7.6 billion from the original $2.5 billion estimated in 1983. Specifically, regarding terminal modernization, the Administrator canceled the Terminal Advanced Automation System and expanded the STARS project to include all terminal facilities. In April 1996, FAA established a new acquisition management system, as directed by the Congress. Included in this system is the concept of life-cycle management, which is intended to be a more comprehensive, disciplined full-cost approach to managing the acquisition cycle, from analysis of mission needs and alternative investments through system development, implementation, operation, and, ultimately, disposal. Under this new system, decisions related to resource allocation (mission and investment) are made by FAA's Joint Resources Council, which is composed of associate administrators for operations and acquisition and other key executives. Decisions associated with program planning and implementation are made within Integrated Product Teams (IPT). IPTs are responsible for bringing together all essential elements of program implementation, including scheduling, allocation of funding, and the roles and responsibilities of stakeholders. To ensure successful program implementation, the acquisition management system dictates that these issues be resolved before contracts are awarded. IPTs also generate schedule and cost baselines, which the Joint Resources Council authorizes the teams to operate under. Team members include representatives from FAA units responsible for operating and maintaining air traffic control equipment and other stakeholders in the acquisition process. To achieve the implementation schedule approved by the Joint Resources Council in January 1996, FAA will have to obtain commitment from key stakeholders, resolve scheduling conflicts between STARS and other terminal modernization efforts, and overcome difficulties in developing the system. FAA is aware that these issues pose a risk for STARS and has begun several risk mitigation initiatives. While such actions are encouraging, it is too early to tell how effective they will be. FAA's schedule for developing and implementing STARS by its January 1996 approved baseline is shown in table 1. Figure 2 shows FAA's plans for ordering, delivering, and operating STARS. FAA intends to begin operating STARS at only three TRACONs before fiscal year 2000. Operation increases after this time, with FAA expecting to operate 55 additional systems in fiscal year 2002. FAA has yet to obtain commitment from all key stakeholders responsible for ensuring that STARS equipment is properly installed. FAA's new acquisition management system stresses that IPTs need to reach agreement before contracts are awarded. Such agreement is necessary to ensure that all stakeholders' roles are defined and agreed upon, facilities are ready to receive STARS, and all other equipment necessary for the operation of STARS is in place. In the past, poor coordination among key stakeholders has caused schedule delays in other modernization projects at FAA. The IPT for STARS has yet to obtain commitment to the STARS schedule from the entire Airway Facilities Service--a key stakeholder. Located in headquarters and regions, maintenance technicians who work for the Airway Facilities Service are responsible for installing and maintaining air traffic control equipment. FAA's current schedule anticipates that STARS will be installed at most sites using a turnkey concept whereby the contractor, not FAA employees, will install the equipment. This concept presumes that a significant level of regional resources will still be required to support and oversee contractor installation. IPT officials told us that while Airway Facilities Service officials at headquarters have committed to the turnkey concept, regions' commitment is incomplete. IPT and Airway Facilities Service officials told us that a process has been established to ensure regions' understanding and obtain their commitment. As part of this process, the IPT has begun regional briefings and has formed implementation teams to gain regions' commitment on turnkey issues. In addition, the IPT has yet to obtain commitment to the STARS schedule from the Professional Airways Systems Specialists--the technicians' union. Top union officials told us that, as of late February 1997, they have not been briefed on the STARS turnkey concept and have not agreed as to how it will be implemented. The union is concerned that the turnkey installation may jeopardize the job security of its members. IPT officials said that while union representatives have been involved in reviewing vendors' proposals for STARS, the union has not been briefed on the specifics of STARS deployment. Although FAA's Acquisition Management System stresses that all key program implementation issues be resolved before contracts are awarded, the IPT believed that it could obtain the union's commitment at a later date. As required by the union's collective bargaining agreement, in January 1997, the IPT initiated actions to brief the union and obtain its commitment. FAA's schedule for STARS can be jeopardized by scheduling conflicts with other modernization efforts. For example, each year, various TRACONs are scheduled to be renovated or replaced. If STARS equipment is delivered during this time, installation could be delayed. Currently, the IPT is unsure of the number of these potential conflicts. In September 1996, the IPT identified 12 potential scheduling conflicts at the first 45 STARS sites. One month later, the number of conflicts was reduced to four, but the team did not provide us with an explanation for this decrease. We believe that the number of potential conflicts will not be known until the IPT ascertains the readiness of each facility to receive and install STARS equipment. The IPT plans to start conducting site reviews in 1997. Another potential scheduling conflict involves terminal surveillance radars, which track aircraft position and use analog or digital processing and communications to transmit the information to TRACONs. Many existing surveillance radars are not digital, but STARS requires digital processing and communications. FAA plans to replace nondigital Airport Surveillance Radar-7s (ASR-7) with new digital ASR-11s. The agency has not decided yet whether to replace other nondigital radar, ASR-8s, or to digitize them. In January 1997, FAA was concerned that 47 of 98 ASR-7s and -8s might not be upgraded in time to meet the STARS schedule. FAA officials told us that, as of late February, they had reduced the number of potential conflicts from 47 to 10 through efforts to coordinate the STARS and digital radar schedules. According to an IPT official, if digital radar does not provide coverage for a TRACON's entire airspace, FAA may have to delay STARS or reorder the sequence of TRACONs receiving STARS. FAA officials told us that they are taking actions to identify and resolve potential scheduling conflicts. The IPT has developed project guides for the FAA regions receiving STARS. These guides identify possible scheduling conflicts with other modernization efforts. Also, Airway Facilities Service officials told us that as a result of a recent reassessment in December 1996 of the schedule for the first 39 STARSs, FAA was able to avoid potential conflicts by repositioning the order in which TRACONs received STARS. Finally, the Airway Facilities Service is developing a database to assist the IPT in maintaining current planning information. Although STARS depends on the use of commercial off-the-shelf computer hardware and a significant amount of commercially available software, FAA and Raytheon have numerous tasks to accomplish before system development is completed. However, the nature and extent of these tasks are not completely known, and such development inevitably poses continual managerial and technical challenges. As noted in table 1, FAA's schedule calls for software development to proceed in two phases. For the initial phase, the agency expects to complete software testing in September 1998, about 2 years from the time when the contract was awarded. For the second phase, the agency expects to complete testing of the full STARS software in July 1999. As an example of the challenge that software development poses for FAA, as recently as December 1996, FAA and Raytheon were discussing (1) how the system would provide specific functions and (2) whether certain functions would be needed, and if so, whether the functions would be included in the equipment with initial- or full-system capability. According to Raytheon officials, these discussions ended with FAA and Raytheon coming to closure on all of the 28 issues needing resolution. As a result, some 16,000 lines of additional software code--beyond the planned 124,000 lines of new code--must be written. Of the 140,000 lines of code, about 138,000 are for flight data processing, training, and maintenance functions, and 2,000 are to fulfill safety requirements, such as warning controllers when aircraft are not maintaining proper separation or minimum safe altitudes. Raytheon officials believe the additional code development will not affect their ability to meet the original milestones. All new code will have to be tested in conjunction with the nearly 840,000 lines of existing STARS software code. If potential difficulties in developing and testing the system are realized, initial implementation of STARS--particularly at the three TRACONs targeted for operation before fiscal year 2000--will likely be delayed. FAA's life-cycle cost baseline has the potential to increase--from $2.23 billion, the level approved by the Joint Resources Council in January 1996, to as much as $2.76 billion. This possible increase is attributable to expected higher costs for operating and maintaining STARS equipment. FAA expects the estimate for facilities and equipment costs to remain stable for the immediate future. FAA's January 1996 facilities and equipment cost baseline is $940 million. During 1996, this baseline was reviewed by the IPT. Through September 1996, the IPT was estimating that the baseline could increase to $1.18 billion. At that time, the IPT (1) estimated higher expected costs for software development; (2) estimated higher expected implementation, technical support, and maintenance costs because of the addition of necessary equipment; and (3) included costs for communications because the baseline estimate overlooked them. In December 1996, the IPT assessed the STARS costs on the basis of the signed contract with Raytheon. As a result, the IPT determined, that while some cost elements will increase, other elements will decrease. Specifically, significantly lower costs for hardware--key components were $40,000 less per unit than what FAA had estimated--will enable the STARS project for the present time to stay within the original baseline. Table 2 shows the differences in cost elements between the original cost baseline and the IPT's December 1996 assessment. FAA's January 1996 operations cost baseline is $1.29 billion. However, based on a September 1996 analysis, FAA staff identified a potential $529 million increase that could revise the baseline to $1.82 billion. FAA officials told us that this increase occurred, in part, because the agency overlooked maintenance costs in the initial estimates. Also, the officials attributed the increase to FAA's deploying more STARS equipment than originally planned. IPT officials told us that on the basis of more current information from the contractor, operations and maintenance costs are expected to be significantly closer to the $1.29 billion baseline estimate than the $1.82 billion figure. The officials could not, however, provide us with an updated cost estimate or detailed support for their views. The IPT officials told us that they are reviewing the latest cost estimates and expect to brief the Joint Resources Council on any potential changes to the baseline in March 1997. Separate and distinct from STARS life-cycle costs are two additional costs that FAA will incur to make STARS operational. First, FAA will have to prepare the TRACONs for the delivery of STARS equipment. FAA officials estimate that the agency will incur at least $18 million in costs to get the first 46 TRACONs and related facilities ready to accept the STARS equipment. Roughly half of this amount is for asbestos removal; the balance is for power upgrades and building improvements. FAA has yet to develop estimates for readying the remaining sites. Second, FAA will incur costs for upgrading radars. FAA plans to modernize the existing analog ASR-8 radars that provide data to its TRACONs. Because the implementation of STARS is approaching, FAA is faced with an immediate decision between digitizing these existing analog radars or replacing them with new digital radars. FAA officials estimate that the 20-year life-cycle costs for modifying and digitizing all the ASR-8s will be $459 million and for replacing them will be $474 million. According to FAA officials, the estimated cost difference between digitizing existing radars and buying new radars is minimal because of the higher costs of maintaining older analog equipment. The agency is continuing to refine these cost estimates, and it expects to decide later this year on which option to select. We provided the Department of Transportation with a draft of this report for its review and comment. We met with FAA officials, including the IPT leader for Terminal Air Traffic Systems Development; the Program Director for National Airspace System Transition and Implementation; and representatives of FAA's Air Traffic and Airway Facilities Services. FAA was concerned about our use of the $1.82 billion estimate for operations and maintenance costs. The estimate came from a September 1996 study done by FAA's Program Analysis and Operations Research staff. FAA told us that this estimate was preliminary and should not be reported as a basis for evaluating the STARS project. While FAA acknowledged that there may be some cost growth in the STARS project, it did not anticipate growth as large as we reported. We continue to include the September 1996 estimate in this report. This estimate was developed by experienced cost analysts, including a member of the STARS IPT, and was the only documented estimate available since the official baseline was approved in January 1996. Furthermore, FAA could not provide us with a more current estimate or detailed support for its views on why the September 1996 analysis may have overstated the cost estimate for operations and maintenance. FAA also expressed concern about the way the draft report characterized the extent to which key stakeholders were committed to the implementation schedule, which relies heavily on the use of the turnkey concept. We revised the report to recognize that (1) while regions' commitment is incomplete, Airway Facilities Service officials at headquarters have committed to the turnkey concept and (2) FAA has established a process, including the formation of implementation teams to ensure regions' understanding and obtain their commitment on turnkey issues. However, because the turnkey concept will affect regional resources and employees' responsibilities, FAA agreed that the potential lack of regions' commitment is a risk that must be mitigated throughout the implementation of STARS. To obtain information for this report, we interviewed officials at FAA headquarters, its New England Regional Office in Burlington, Massachusetts, its New York Regional Office in Jamaica, New York, and its William J. Hughes Technical Center in Pomona, New Jersey. We reviewed agency documentation on current schedule and life-cycle costs for STARS. We reviewed guidelines pertaining to system acquisition, compared FAA's actions to the guidance, and identified key issues that could affect the success of the STARS project. To identify any labor issues that could affect the scheduled deployment, we interviewed union officials with the Professional Airways Systems Specialists. We conducted our review from July 1996 through January 1997 in accordance with generally accepted government auditing standards. However, we did not assess the reliability of the process used to generate cost information. We are sending copies of this report to the Secretary of Transportation, the Administrator of FAA, and other interested parties. We will also make copies available to others on request. Please call me at (202) 512-3650 if you or your staff have any questions about this report. Major contributors to this report are listed in appendix I. John H. Anderson, Jr. Gregory P. Carroll Robert E. Levin Peter G. Maristch John T. Noto The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Summary:
Pursuant to a congressional request, GAO reviewed the Federal Aviation Administration's (FAA) acquisition planning to date, focusing on the extent to which: (1) the schedule estimate for the Standard Terminal Automation Replacement System (STARS) is attainable; and (2) cost estimates to make STARS operational are likely to change. GAO noted that: (1) the STARS schedule, which calls for implementation of 171 air traffic control facilities between December 1998 and February 2005, is attainable only if FAA is successful in its efforts to mitigate certain risks; (2) specifically, FAA will need to obtain commitment by key stakeholders to the STARS schedule, resolve schedule conflicts between STARS and other modernization efforts, and overcome difficulties in developing system software that could delay implementing STARS; (3) FAA is aware that these issues pose a risk for STARS and has begun several risk mitigation initiatives; (4) while such actions are encouraging, it is too early to tell how effective they will be; (5) FAA's cost estimate for STARS has the potential to increase; (6) FAA's total cost estimate for STARS is $2.23 billion; (7) FAA approved this estimate in January 1996, however, a September 1996 analysis by agency officials pointed to potential cost increases that could drive the total cost estimate to as much as $2.76 billion; (8) this possible increase is attributable to expected higher costs for operating and maintaining STARS equipment; (9) FAA officials are continuing to revise the STARS cost estimate and now believe that cost increases may be significantly lower; and (10) at this time, however, FAA could not provide GAO with an updated estimate.