FAA Computer Systems: Limited Progress on Year 2000 Issue Increases Risk Dramatically
Released: 1998-02-05T00:00:00

Letter

Background

FAA is responsible for ensuring a safe, secure, and efficient airspace system that contributes to national security and the promotion of U.S. airspace. To fulfill these key missions, FAA administers a wide range of aviation-related programs, such as those to certify the airworthiness of new commercial aircraft designs, to inspect airline operations, to maintain airport security, and to control commercial and general aviation flights.

Integral to executing each of FAA’s programs are extensive information processing and communications technologies. For example, each of FAA’s 20 en route air traffic control facilities, which controls aircraft at the higher altitudes between airports, depends on about 50 interrelated computer systems to safely guide and direct aircraft. Similarly, each of FAA’s almost 100 flight standards offices, responsible for inspecting and certifying various sectors of the aviation industry (e.g., commercial aircraft, repair stations, mechanics, pilot training schools, maintenance schools, pilots, and general aviation aircraft), are supported by over 30 mission-related safety database and analysis systems. Because of the complexity of the systems supporting FAA’s mission, most of these systems are unique to FAA and not off-the-shelf systems that can be easily maintained by system vendors.

FAA also has numerous, complex information processing interactions with various external organizations, including airlines, aircraft manufacturers, general aviation pilots, and other government agencies, such as the National Weather Service (NWS) and the Department of Defense. Over the years, these organizations and FAA have built vast networks of interrelated systems. For example, airlines’ flight planning systems are linked to FAA’s Enhanced Traffic Management System, which monitors flight plans nationwide, controls high traffic situations, and alerts airlines and airports to bring in more staff when there is extra traffic. As another example, FAA facilities rely on weather information from NWS ground sensors, radars, and satellites to control and route aircraft.

FAA is headed by an administrator, who is supported by the chief counsel, assistant administrators responsible for each of its five staff offices, and associate administrators responsible for each of its seven lines of business. The chief counsel and five staff offices handle crosscutting management functions (e.g., system safety), while the seven lines of business are dedicated to specific services (e.g., airport funding). For the purpose of Year 2000 program planning, FAA refers to all of these 13 organizations as “lines of business”. Figure 1 provides a visual summary of FAA’s management structure, highlighting the 13 lines of business.

Objective, Scope, and Methodology

In assessing actions taken by FAA to address the Year 2000 problem, our objective was to determine the effectiveness of FAA’s Year 2000 program, including the reliability of FAA’s Year 2000 cost estimate.

To satisfy this objective, we reviewed and analyzed key FAA documents, including (1) Year 2000 guidance and draft performance plan documents, (2) Year 2000 memorandums and cost estimation worksheets, (3) minutes of FAA’s Year 2000 steering committee, and (4) the lines of businesses’ Year 2000 project plans, monthly status reports, and systems assessment documentation. We also reviewed the agency’s Internet Web sites for Year 2000 information and newsletter articles, plus FAA’s documentation of various Year 2000 briefings. We used our Year 2000 assessment guide to assess FAA’s readiness to achieve Year 2000 compliance.

To supplement the analyses noted above, we interviewed FAA’s Year 2000 product team members, the Year 2000 program manager, FAA’s deputy chief information officer, and representatives from each of FAA’s 13 lines of business.

We performed our work at the Federal Aviation Administration in Washington, D.C., the Mike Monroney Aeronautical Center in Oklahoma City, Oklahoma, and the William J. Hughes Technical Center in Atlantic City, New Jersey. Our work was performed from February through November 1997, in accordance with generally accepted government auditing standards. We requested comments on a draft of this product from the Secretary of Transportation or his designee. On January 6, 1998, we obtained oral comments from Transportation and FAA officials, including representatives from the Office of the Secretary of Transportation and FAA’s Chief Information Officer. Their comments are discussed in the “Agency Comments and Our Evaluation” section of this report.

High Risk of Year 2000 Complications Can Be Reduced Through Structured Approach and Rigorous Program Management

On January 1, 2000, computer systems worldwide could malfunction or produce inaccurate information simply because the date has changed. Unless corrected, such failures could have a costly, widespread impact. The problem is rooted in how dates are recorded and computed. For the past several decades, systems have typically used two digits to represent the year—such as “97” for 1997—to save electronic storage space and reduce operating costs. In such a format, however, 2000 is indistinguishable from 1900. Software and systems experts nationwide are concerned that this ambiguity could cause systems to malfunction in unforeseen ways, or to fail completely.

Correcting this problem will not be easy or inexpensive, and must be done while such systems continue to operate. Many of the government’s computer systems were developed 20 to 25 years ago, use a wide array of computer languages, and lack full documentation. Systems may contain up to several million lines of software code that must be examined for potential date-format problems.

The enormous challenge involved in correcting these systems is not primarily technical, however, it is managerial. Agencies’ success or failure will largely be determined by the quality of their program management and executive leadership. Top agency officials must understand the importance and urgency of this undertaking, and communicate this to all employees. The outcome of these efforts will also depend on the extent to which agencies have institutionalized key systems-development and program-management practices, and on their experience with such large-scale software development or conversion projects. Accordingly, agencies must first assess their information resources management capabilities and, where necessary, upgrade them. In so doing, they should consider soliciting the assistance of other organizations experienced in these endeavors.

To assist agencies with these tasks, we have prepared a guide that discusses the scope of the challenge and offers a structured, step-by-step approach for reviewing and assessing an agency’s readiness to handle the Year 2000 problem. The guide describes in detail the following five phases, each of which represents a major Year 2000 program activity or segment:

Awareness. This phase entails defining the Year 2000 problem, gaining executive level support and sponsorship, and ensuring that everyone in the organization is fully aware of the issue. Also, during this phase, a Year 2000 program team is established and an overall strategy is developed.

Assessment. This phase entails assessing the Year 2000 impact on the enterprise, identifying core business areas, inventorying and analyzing the systems supporting the core business areas, and prioritizing their conversion or replacement. Also, during this phase, contingency planning is initiated and the necessary resources are identified and secured.

Renovation. This phase deals with converting, replacing, or eliminating selected systems and applications. In so doing, it is important to consider the complex interdependencies among the systems and applications.

Validation. This phase deals with testing, verifying, and validating all converted or replaced systems and applications and ensuring that they perform as expected. This entails testing the performance, functionality, and integration of converted or replaced systems, applications, databases, and interfaces in an operational environment. Implementation. This phase entails deploying and implementing Year 2000 compliant systems and components. Also, during this phase, data exchange contingency plans are implemented, if necessary.

FAA Year 2000 Awareness Activities Not Yet Completed

Institutional Year 2000 awareness is the first step in effectively addressing the Year 2000 problem. FAA has initiated awareness activities, including conducting a Year 2000 problem awareness campaign, drafting a Year 2000 agencywide plan, issuing a Year 2000 guidance document for project-level plan development, and establishing a program management organization. However, FAA fell behind in other key awareness activities, such as finalizing the agency’s Year 2000 strategy, in part due to its late designation of a program manager.

FAA Has Initiated Year 2000 Activities

FAA recognizes that the upcoming change of century poses significant challenges to the agency. It began Year 2000 problem awareness activities in May 1996, and during the ensuing 3 months established a Year 2000 product team and designated it as the focal point for Year 2000 issues within FAA. Also, FAA established a Year 2000 steering committee. Since then, the Year 2000 product team and steering committee have (1) sponsored an awareness day and held assessment and testing practices workshops, (2) set up Web pages and published a newsletter article to provide information on the Year 2000 problem, and (3) briefed FAA’s management on the agency’s Year 2000 problem. In addition, in September 1996, the product team issued the FAA Guidance Document for Year 2000 Date Conversion. This guide was intended to assist the lines of businesses within FAA in planning for the conversion of their computer systems to handle processing of dates in the year 2000 and beyond.

FAA Late in Designating Year 2000 Program Manager, Leading to Delays in Year 2000 Plan

It is essential that agencies appoint a Year 2000 program manager and establish an agency-level program office to manage and coordinate Year 2000 program activities. The problem and solutions involve a wide range of dependencies among information systems: the need to (1) centrally develop or acquire conversion and validation standards, inspection, conversion, and testing tools, (2) coordinate the conversion of crosscutting information systems and their components, (3) establish priorities, and (4) reallocate resources as needed.

However, FAA did not establish a program manager who had responsibility for Year 2000 program management until July 1997. This contributed to key awareness activities being delayed. Specifically, FAA experienced delays in establishing the agencywide Year 2000 plan needed for timely initiation and effective execution of the key awareness and assessment phase activities.

Because FAA was slow to designate a program manager, it is only now finalizing its agencywide plan for achieving Year 2000 compliance. The September 1997 draft of this document outlines the FAA strategy and management approach to address the Year 2000 century date change. Specifically, it defines the Year 2000 program management structure and responsibilities; adopts the five-phase management process, including the awareness, assessment, renovation, validation, and implementation phases that are being used throughout the government to manage and measure agencies’ Year 2000 programs; calls for awareness and assessment activities to be completed provides for completion of the three remaining program phases—renovation, validation, and implementation; and establishes performance indicators and reporting requirements.

FAA’s Year 2000 project manager provided a draft of this plan to the Administrator on December 1, 1997, but does not have any estimate as to when this document will be signed by the Administrator and made final.

Without an official agencywide Year 2000 strategy, FAA’s executive management is without a road map for achieving Year 2000 compliance.

Further, the lack of an approved strategy means that FAA’s program manager lacks the authority to enforce Year 2000 policies. As a result, each line of business will have to decide if, when, and how to address its Year 2000 conversion, irrespective of agency priorities and standards. This reinforces our existing concerns with FAA’s CIO not being in the proper place in the organization to develop, maintain, and enforce information technology initiatives. We have repeatedly recommended that FAA adopt a management structure similar to that of the department-level CIOs as prescribed in the Clinger-Cohen Act. The Department of Transportation (DOT) and FAA have disagreed with this recommendation because they believe that the current location of the CIO, within the research and acquisition line of business, is effective. We disagree. FAA’s CIO does not report directly to the Administrator and thus does not have organizational or budgetary authority over those who develop air traffic control systems or the units that maintain them. Further, the agency’s long history of problems in managing information technology projects reflects the need for change.

FAA Does Not Know the Full Extent of Its Year 2000 Problem Because It Has Not Completed Assessment Activities

FAA has not completed key assessment activities, placing it at enormous risk of not achieving Year 2000 compliance by January 1 of that year. Specifically, FAA has not (1) assessed the severity of its Year 2000 problem and (2) completed the inventory and assessment of its information systems and their components. Also, while FAA has initiated other key assessment phase activities on individual projects, it has not completed determining priorities for system conversion and replacement, developing plans for addressing identified date dependencies, developing validation and test plans for all converted or replaced systems, addressing interface and data exchange issues among internal and external systems, and initiating the development of business continuity plans in case systems are not corrected in time. FAA states that it expects to complete assessment phase activities by the end of January 1998.

FAA Does Not Know the Likely Full Impact of Its Year 2000 Problem

Developing and publishing a high-level assessment of the severity of the Year 2000 issue provides executive management and staff with a broad overview of the potential impact the century change could have on the agency. Such an assessment provides management with valuable information on which to rank the agency’s Year 2000 activities, as well as a means of obtaining and publicizing management commitment and support for necessary Year 2000 initiatives.

Unfortunately, FAA has only begun to assess the severity of the impact of Year 2000-induced failures. The Year 2000 Financial Oversight Team, established in October 1997, has been tasked with identifying the impact of Year 2000 failures on FAA’s operations, programs, and priorities. This assessment will be focused primarily on key mission critical systems and is to be provided to FAA management in February 1998.

On the basis of our discussions with FAA personnel, it is clear that FAA’s ability to ensure the safety of the National Airspace System and to avoid the grounding of planes could be compromised if systems are not changed. For example, the Host Computer System, the centerpiece information processing system in FAA’s en route centers, relies on the date to determine which day of the week it is when the system is initialized. This information triggers the use of different prescheduled flight plans. That is, carriers use different flight plans and times on a Monday than they do on a Saturday. Because January 1, 2000, is a Saturday, and January 1, 1900, was a Monday, uncorrected date dependencies could lead to the Host using incorrect flight plans. This could result in delayed flights. While FAA officials stated that they believe this problem has been solved, they acknowledged that other unforeseen problems may exist because they have not yet completed assessments of the impact of the Year 2000 problem.

External organizations are also concerned about the impact of FAA’s Year 2000 status on their operations. FAA recently met with representatives from airlines, aircraft manufacturers, airports, fuel suppliers, telecommunications providers, and industry associations to discuss the Year 2000 issue. At this meeting, participants raised the concern that their own Year 2000 compliance was irrelevant if FAA was not compliant because of the many system interdependencies. Airline representatives further explained that flights could not even get off the ground on January 1, 2000, unless FAA is substantially Year 2000 compliant—and that would be an economic disaster. Because of these types of concerns, FAA has now agreed to meet regularly with industry representatives to coordinate the safety and technical implications of shared data and interfaces.

Inventory and Assessment of Information Systems Are Incomplete

An agencywide inventory and assessment of information systems and their components provides the necessary foundation for detailed Year 2000 program planning. A thorough inventory ensures that all systems are identified and linked to a specific business area or process, and that all agencywide, crosscutting systems are considered while detailed assessments determine (1) the criticality of the various systems and (2) how they should be handled (through conversion, replacement, retirement, or no remedial action).

According to FAA’s April 1997 Year 2000 guidance document, it expected to have completed inventories of its computer systems and components by May 31, 1997. However, FAA still had not finished them when we completed audit work in November 1997. This inventory did not contain all systems, support software, firmware, telecommunications equipment, and desktop computers. According to FAA’s November 15, 1997, quarterly Year 2000 report to DOT, its inventory included 619 systems. These systems comprise approximately 18,000 subsystems and 65 million lines of software code. In commenting on a draft of this report, a Year 2000 program official told us that the inventory of systems was completed on December 29, 1997, with other inventory items expected to be completed later.

In addition, FAA has not completed assessing (1) the criticality of the computer systems in its inventory or (2) how the systems should be handled. Of the 619 systems in its inventory when it reported to DOT on November 15, FAA identified 329 as mission-critical, 278 as nonmission-critical, and 12 as undetermined, meaning that they have not yet been categorized as mission-critical or nonmission-critical. These numbers will likely continue to grow as the inventory nears completion.

In mid-November, FAA provided data to DOT on the number of mission critical systems it had assessed and whether it had determined that they should be replaced, retired, left alone, or converted to Year 2000 compliancy. DOT requested validation of these assessments and refined these numbers for its November quarterly report to the Office of Management and Budget (OMB). In that report, DOT stated that FAA had completed assessments of only 84, or about 25 percent, of its 329 mission-critical systems. That is, FAA had not determined how to handle its remaining 245 mission critical systems.

Of the 84 completed assessments, DOT reported that 34 are Year 2000 compliant, 8 are to be replaced with new compliant applications, 2 are being retired, and 32 are being repaired. The remaining 8 systems are in the process of being certified compliant. Figure 2 summarizes the completed assessments’ results.

FAA reported that assessment of the remaining mission-critical systems will continue through the end of January 1998.

Other Key Assessment Phase Activities Are Likewise Incomplete

Other key assessment phase activities include determining priorities for system conversion and replacement, developing plans for addressing identified date dependencies, developing validation and test plans for all converted or replaced systems, addressing interface and data exchange issues among systems, and developing contingency plans for continuing operations in case systems are not corrected in time. FAA has just begun these activities.

In October 1997, FAA established a Year 2000 Financial Oversight Team with responsibilities for determining priorities for system conversion and replacement, and recommending sources for funding Year 2000 activities to FAA management. At the conclusion of our audit work, Year 2000 program officials told us that they provided preliminary recommendations to FAA management in December 1997, with final recommendations to follow in February 1998.

Also, FAA’s draft Year 2000 plan calls for each of the 13 lines of business to (1) develop plans for addressing identified date dependencies, (2) develop plans for validating and testing all converted or replaced systems, (3) address interface and data exchange issues among systems, and (4) develop a realistic contingency plan, including establishing manual or contract procedures, to ensure the continuity of core processes. To date, most of the lines of business have developed plans for addressing identified date dependencies. Some of these plans include requirements for testing converted or replaced systems, addressing interface and date exchange issues, and developing contingency plans; other plans, however, do not address these items. Regardless, not all of these plans have been finalized. The program manager told us that she was working with the responsible organizations and planned to finalize their plans by the end of December 1997. At the conclusion of our review, these plans had still not been finalized.

Delays in Completing Awareness and Assessment Leave Little Time for Critical Renovation, Validation, and Implementation Activities

Renovation, validation, and implementation activities are the three critical final phases in correcting Year 2000 vulnerabilities. FAA has started the renovation process for some of the systems with completed assessments. However, because of the agency’s delays in completing its awareness and assessment activities, time is running out for FAA to renovate its systems, validate these conversions or replacements, and implement its converted or replaced systems.

FAA’s delays are further magnified by the agency’s poor history in delivering promised system capabilities on time and within budget. FAA’s weaknesses in managing software acquisition will also hamper its renovation, validation, and implementation efforts.

Given the many hurdles FAA faces and the limited amount of time left, planning for business continuity becomes ever more urgent for FAA so that its mission-critical business processes and supporting systems continue to function through the millennium. Such business continuity planning defines the assumptions and risk scenarios, business service objectives, time frames, priorities, tasks, activities, procedures, resources, responsibilities, and the specific steps and detailed actions for re-establishing functional capability for mission critical business processes in the event of prolonged disruption, failure, or disaster.

Until Assessments Are Complete, Cost Estimates Will Remain Incomplete

Reliable program cost estimates require a thorough and complete definition of a program’s scope and components. However, FAA has yet to completely define the scope of its Year 2000 program. As noted above, FAA has not yet completed its inventory of systems or its assessment of which systems are critical and how to handle them. As a result, the current Year 2000 program cost estimate of $246 million will likely change once FAA has a better handle on its inventory and determines how to handle the various systems. FAA acknowledges the uncertainty of its current cost estimate due to incomplete inventory and assessment information.

Even after assessments are completed and estimates finalized, FAA’s cost estimates could still be of questionable reliability. We have previously reported on FAA’s weaknesses in reliably estimating the cost of its software-intensive Air Traffic Control projects and recommended that FAA correct its weak cost estimating practices by institutionalizing defined estimating processes. FAA agreed with our recommendation and has initiated efforts to improve its processes.

Regardless of the eventual cost estimate, uncertainty surrounds funding of FAA’s Year 2000 activities. For example, only $18 million of the $246 million is currently in the fiscal year 1998 budget. At the same time, however, OMB has stated that, because of DOT’s disappointing progress on the Year 2000 issue, it has established a rebuttable presumption that it will not fund any DOT requests for information technology investments in the fiscal year 1999 budget unless they are directly related to fixing the Year 2000 problem. Further, according to a December 4, 1997, briefing to the Administrator, FAA has been directed by OMB to reprogram existing funds from lower priority projects to its Year 2000 program.

Conclusions

Should the pace at which FAA addresses its Year 2000 issues not quicken, and critical FAA systems not be Year 2000 compliant and therefore not be ready for reliable operation on January 1 of that year, the agency’s capability in several essential areas—including the monitoring and controlling of air traffic—could be severely compromised. This could result in the temporary grounding of flights until safe aircraft control can be assured.

Avoiding such emergency measures will require strong, active oversight. Yet an approved strategy containing detailed plans, milestones, and valid cost estimates—all vital considerations—is still lacking. This is due, at least in part, to incomplete assessment of agency vulnerabilities. Such incomplete assessment is cause for concern. It means that FAA has no way of knowing at this time how serious its Year 2000 date software-coding problem is—or what it will cost to address it.

FAA’s delays to date are very troubling. Given the rapidly approaching millennium, such delays are no longer acceptable. Until all inventorying and assessments have been completed—set for the end of January 1998—FAA will not be able to effectively or efficiently marshal the available resources, both human and financial, that will be needed to do the job. Once the degree of vulnerability has been determined, a structured approach—such as that provided in our assessment guide—can offer a road map as to the effective use of such resources. Unless critical renovation, validation, and implementation activities are completed in time, and sound contingency plans are available, FAA risks not successfully navigating the change to the new millennium.

Recommendations

Urgent action is imperative to improve the management effectiveness of FAA’s Year 2000 program and thus the likelihood of its success. Accordingly, we recommend that the Secretary of Transportation direct that the Administrator of FAA, take the actions necessary to expedite the completion of overdue awareness and assessment activities. At a minimum, the Administrator should finalize an agencywide plan which provides the Year 2000 program manager the authority to enforce Year 2000 policies and outlines FAA’s strategy for addressing the Year 2000 date change; assess how its major business lines and the aviation industry would be affected if the Year 2000 problem were not corrected in time, and use the results of this assessment to help rank the agency’s Year 2000 activities, as well as a means of obtaining and publicizing management commitment and support for necessary Year 2000 initiatives; by January 30, 1998, complete inventories of all information systems and their components, including data interfaces; by January 30, 1998, finish assessments of all systems in FAA’s inventory to determine each one’s criticality and to decide whether each system should be converted, replaced, or retired; determine priorities for system conversion and replacement based on establish plans for addressing identified date dependencies; develop validation and test plans for all converted or replaced systems; craft Year 2000 contingency plans for all business lines to ensure continuity of critical operations; and make a reliable cost estimate based on a comprehensive inventory and completed assessments of the various systems’ criticality and handling needs.

Agency Comments and Our Evaluation

DOT and FAA officials provided oral comments on a draft of this report. These officials generally concurred with the report’s findings, conclusions, and recommendations. FAA’s Chief Information Officer (CIO) stated that FAA recognizes the importance of addressing the Year 2000 issue and plans to implement our recommendations.

The CIO stated that FAA’s administrator had not yet signed the agencywide Year 2000 plan and that its Year 2000 program manager retired at the end of December 1997. FAA plans to hire a new acting program manager from outside the agency. Given the limited amount of time left to address Year 2000 issues, delays in finalizing the agencywide plan and the turnover of senior management further risk FAA’s chance of success.

Representatives from the Air Traffic Services (ATS) line of business, the organization responsible for operational air traffic control systems, commented that their organization has made significant progress in addressing the Year 2000 problem and that they do not have the problems that FAA has overall. ATS officials stated that they have in place a Year 2000 project plan, repair process and standards, and a quality assurance plan for system renovations. While we acknowledge these steps by ATS, the pace of progress must increase if ATS and FAA are to address the Year 2000 problem in time.

FAA officials also offered some specific comments directed to particular language in the draft report. These comments have been incorporated into the report where appropriate.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we will not distribute it until 30 days from its date. At that time, we will send copies of this report to the Chairmen and Ranking Minority Members of the House and Senate Committees on Appropriations, and their Subcommittees on Transportation; the Subcommittee on Aviation, Senate Committee on Commerce, Science and Transportation; and the Subcommittee on Aviation, House Committee on Transportation and Infrastructure. We are also sending copies to the Secretary of Transportation, the Administrator of the Federal Aviation Administration, and other interested congressional committees and subcommittees. Copies will also be made available to others upon request. Please contact me at (202) 512-6253 or by e-mail at willemssenj.aimd@gao.gov if you have any questions concerning this report. Major contributors to the report are listed in appendix I.

Major Contributors to This Report

Accounting and Information Management Division, Washington, D.C.

Atlanta Field Office

Glenda C. Wright, Senior Information Systems Analyst The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.