Year 2000 Computing Crisis: FAA Must Act Quickly to Prevent Systems Failures
Released: 1998-02-04T00:00:00

Letter

FAA Depends on Information Processing to Fulfill Its Mission

In ensuring a safe, secure, and efficient airspace system that contributes to national security and the promotion of U.S. airspace, FAA administers a wide range of aviation-related programs, such as those to certify the airworthiness of new commercial aircraft designs, inspect airline operations, maintain airport security, and control commercial and general aviation flights.

Integral to executing each of FAA’s programs are extensive information processing and communications technologies. For example, each of FAA’s 20 en route air traffic control facilities, which control aircraft at the higher altitudes between airports, depends on about 50 interrelated computer systems to safely guide and direct aircraft. Similarly, each of FAA’s almost 100 flight standards offices, responsible for inspecting and certifying various sectors of the aviation industry, is supported by over 30 mission-related safety database and analysis systems. Because of the complexity of these systems supporting FAA’s mission, many of them are unique to FAA, not off-the-shelf systems that could be readily maintained by vendors.

FAA also has numerous, complex information processing exchanges with various external organizations, including airlines, aircraft manufacturers, general aviation pilots, and other government agencies, such as the National Weather Service (NWS) and the Department of Defense. Over the years, these organizations and FAA have built vast networks of interrelated systems. For example, airlines’ flight planning systems are linked to FAA’s Enhanced Traffic Management System, which monitors flight plans nationwide, controls high-traffic situations, and alerts airlines and airports to bring in more staff during busy periods. As another example, FAA facilities rely on weather information from NWS ground sensors, radars, and satellites to control and route aircraft.

It is easy to see, then, that should FAA systems not be Year 2000 compliant, the domino effect would be far-reaching. In fact, representatives of major airlines are concerned that even if their own systems are ready for the millennium, they could not fly until FAA’s systems were Year 2000 compliant.

FAA’s Year 2000 Awareness, Assessment Work Incomplete; Extent of Problem Unknown

To assist agencies in resolving the Year 2000 problem, we have prepared a guide that discusses the scope of the challenge and offers a structured, step-by-step approach for reviewing and assessing an agency’s readiness to handle this challenge. The guide describes in detail five phases, each of which represents a major Year 2000 program activity or segment. The first phase, awareness, entails gaining executive-level support and sponsorship and ensuring that everyone in the organization is fully aware of the issue. During this phase a Year 2000 program team is also established, and an overall strategy developed. The second phase, assessment, entails assessing the likely Year 2000 impact on the enterprise, identifying core business areas, inventorying and analyzing the systems supporting those areas, and prioritizing their conversion or replacement. Contingency planning is also initiated, and the necessary resources identified and secured.

FAA recognizes that the upcoming change of century poses significant challenges. It began Year 2000 problem awareness activities in May 1996, and within 3 months had established a Year 2000 product team and designated it the focal point for Year 2000 within FAA. A Year 2000 steering committee was also established. Since then, the product team and steering committee have conducted various awareness activities and have briefed FAA management. In September 1996 the product team issued the FAA Guidance Document for Year 2000 Date Conversion.

Yet FAA was late in designating a Year 2000 program manager and its initial program manager recently retired. FAA has not yet selected a permanent replacement and needs to fill this position as soon as possible. Further, its strategic plan—defining program management responsibilities and providing an approach to addressing the millennium challenge—has yet to be made final. A draft of this plan was provided to the Administrator on December 1, 1997, and we understand that it is now being revised. Until an official agencywide strategy is available, FAA’s executive management will not have the approved road map they need for achieving Year 2000 compliance. The lack of a formal agencywide strategy also means that FAA’s program manager position lacks the authority to enforce Year 2000 policies. As a result, each line of business within the agency will have to decide if, when, and how to address its Year 2000 conversion, irrespective of agency priorities and standards.

Additionally, FAA’s inventory of all information systems and their components is still evolving. According to a Year 2000 program official, FAA’s inventory of 741 systems was completed on December 29, 1997. However, we have found that the inventory changed on at least three occasions since then and, by January 23, 1998, had reached 769 systems.

Other crucial tasks include an assessment of the criticality of the systems in the inventory, and deciding whether they should be converted, replaced, retired, or left as is. On January 30, 1998, we were told by a Year 2000 program official that all outstanding systems assessments were to be received that day, but that review and validation of these assessments would continue during February. Assessing the likely severity of systems failures is crucial as well, yet FAA only recently began to examine the likely impact of Year 2000-induced failures; this assessment is due to be presented to FAA management this month, February 1998.

Without the thorough definition of a program’s scope and requirements that only such inventorying and assessment can provide, cost estimates are uncertain at best, as the agency acknowledges. FAA’s current Year 2000 program cost estimate of $246 million will likely change once the agency more accurately identifies its inventory and determines how it will go about making its systems Year 2000 compliant.

On the basis of our discussions with FAA personnel, it is clear that FAA’s ability to ensure the safety of the National Airspace System and to avoid the grounding of planes could be compromised if systems are not changed. FAA’s organization responsible for air traffic control reported that 34 of the 100 mission-critical systems it initially assessed were likely to result in catastrophic failure if they were not renovated. FAA plans to renovate all of these systems. As of January 30, 1998, assessments of another 140 mission-critical air traffic control systems were continuing.

The Host Computer System: Critical Information Processing Link

As FAA completes its systems assessments, it faces difficult decisions about how to renovate, retire, or replace its date-dependent systems. One of the most significant examples is FAA’s Host Computer System—the centerpiece information processing system in FAA’s en route centers—which runs on IBM mainframe computers. Key components of the Host include its operating system, application software, and microcode—low-level machine instructions used to service the main computer. While FAA officials expressed confidence that they have resolved any date dependencies in the Host’s operating system and application software, IBM reported that it has no confidence in the ability of its microcode to survive the millennium date change because it no longer has the skills or tools to properly assess this code. IBM has therefore recommended that FAA purchase new hardware.

Given these concerns, FAA—in an attempt to help ensure success and minimize risk—is considering moving in two directions simultaneously: It is continuing its assessment of the microcode with a plan to resolve and test any identified date issues, while at the same time preparing to purchase and implement new hardware, called Interim Host, at each of its 20 en route centers before January 1, 2000. Yet the purchase of new hardware carries its own set of risks—risks that FAA must mitigate in a short period of time. These are at least fourfold.

First, Lockheed Martin, currently the Host software support contractor, will be responsible for porting the existing Host operating system and application software to the new hardware. This software conversion requires extensive testing to ensure that air traffic control operations are not affected. Unexpected problems in testing and certifying the new system for use in real-time operations may also become apparent.

Second, the Interim Host will have to be deployed concurrently with FAA’s new Display System Replacement (DSR), compounding the risk of delays and problems. When upgrading parts of a safety-critical system such as the Host and DSR, it is simpler and safer to upgrade one part at a time.

Third, deploying the Interim Host to 20 en route centers in less than 2 years will be very difficult. As a point of reference, FAA’s Display Channel Complex Rehost took almost 2 years to deploy to just five centers.

Fourth, by moving quickly to purchase the Interim Host, FAA may not be purchasing a system that best meets its long-term needs. For example, alternative mainframe systems may provide more communications channels—something the Host currently depends on peripheral systems to provide.

External Organizations Also Concerned About FAA Year 2000 Compliance

External organizations are also concerned about the impact of FAA’s Year 2000 status on their operations. FAA recently met with representatives of airlines, aircraft manufacturers, airports, fuel suppliers, telecommunications providers, and industry associations to discuss the Year 2000 issue. At this meeting participants raised the concern that their own Year 2000 compliance would be irrelevant if FAA were not compliant because of the many system interdependencies. Airline representatives further explained that flights could not even get off the ground on January 1, 2000, unless FAA was substantially Year 2000 compliant—and that extended delays would be an economic disaster. Because of these types of concerns, FAA has now agreed to meet regularly with industry representatives to coordinate the safety and technical implications of shared data and interfaces.

Little Time Remains for Critical Renovation, Validation, and Implementation Activities, Placing January 1, 2000, Readiness at Risk

One result of delayed awareness and assessment activities is that the time remaining for renovation, validation, and implementation can become dangerously compressed. Renovation, validation, and implementation activities are the three critical final phases in correcting Year 2000 vulnerabilities. Renovation involves converting, replacing, or eliminating selected systems and applications. Validation entails testing, verifying, and validating all converted or replaced systems and applications, and ensuring that they perform as expected. Implementation involves deploying, operating, and maintaining Year 2000-compliant systems and components. Contingency plans are also implemented, if necessary.

FAA has started to renovate some of the systems it has already assessed. However, because of the agency’s delays in completing its awareness and assessment activities, time is running out for FAA to renovate all of its systems, validate these conversions or replacements, and implement its converted or replaced alternatives.

FAA’s delays are further magnified by the agency’s poor history in delivering promised system capabilities on time and within budget, which we have reported on in the past. FAA’s weaknesses in managing software acquisition will also hamper its renovation, validation, and implementation efforts.

Given the many hurdles that FAA faces and the limited amount of time left, planning for operational continuity through the turn of the century becomes ever more urgent. To ensure the ability to carry out core functions, such planning defines assumptions and risk scenarios, operational objectives, time frames, priorities, tasks, activities, procedures, resources, and responsibilities. Such planning also lays out the specific steps and detailed actions that would be required to reestablish functional capability for mission-critical operations in the event of prolonged disruption, failure, or disaster. We plan to issue a guide later this month, in exposure draft form, to assist agencies in ensuring business continuity by performing necessary contingency planning for the Year 2000 crisis.

Structured, Rigorous Approach Can Reduce Level of Risk, but Urgent Action Essential

FAA’s delays to date put the agency at great risk. The coming millennium cannot be postponed, and FAA will continue to be hamstrung until all inventorying and assessments have been completed. Once the degree of vulnerability has been determined, a structured, five-phase approach with rigorous program management—such as that outlined in our assessment guide—can offer a road map to the effective use of available resources, both human and financial.

But time is short. Should the pace at which FAA addresses its Year 2000 issues not quicken, and critical FAA systems not be Year 2000 compliant and therefore not be ready for reliable operation on January 1 of that year, the agency’s capability in several essential areas—including the monitoring and controlling of air traffic—could be severely compromised. This could result in the temporary grounding of flights until safe aircraft control can be assured. Avoiding such emergency measures will require stronger, more active oversight than FAA has demonstrated in the past.

Our report being released today makes a number of specific recommendations to increase the likelihood that FAA systems will be Year 2000 compliant on January 1 of that year. In summary, we recommend that the Secretary of Transportation direct that the Administrator, FAA, take whatever action is necessary to expedite overdue awareness and assessment activities. At a minimum, this would include issuing a final FAA Year 2000 plan providing the Year 2000 program manager with the authority to enforce Year 2000 policies and outlining FAA’s strategy for addressing the date change; assessing how its major business lines and the aviation industry would be affected if the Year 2000 problem were not corrected in time and using these results to help rank the agency’s Year 2000 activities; completing inventories of all information systems and their components, completing assessments of all inventoried systems to determine criticality and whether the system will be converted, replaced, or retired; determining priorities for system conversion and replacement based on establishing plans for addressing identified date dependencies; developing plans for validating and testing all converted or replaced crafting realistic contingency plans for all business lines to ensure the continuity of critical operations; and developing a reliable cost estimate based on a comprehensive inventory and completed assessments of the various systems’ criticality, and how their needs for modification will be addressed.

Officials of both FAA and the Department of Transportation generally agreed with our findings, conclusions, and recommendations. FAA’s CIO stated that FAA recognizes the importance of addressing the Year 2000 problem and plans to implement our recommendations.

This concludes my statement, and I would be pleased to respond to any questions that you or other Members of the Subcommittees may have at this time.

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.