Defense Computers: Air Force Needs to Strengthen Year 2000 Oversight
Published: 1998-01-16T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO reviewed the Air Force's program for solving its year 2000 computer systems problem, focusing on the: (1) status of the Air Force's efforts to oversee its Year 2000 program; and (2) appropriateness of the Air Force's strategy and actions for ensuring that the problem will be successfully addressed.
section_title: What GAO Found
paragraphs: GAO noted that: (1) as with other military services, the Air Force is taking a decentralized approach to year 2000 correction--that is, it is relying heavily on its components to identify and correct year 2000 problems affecting their own systems; (2) however, in providing oversight for this effort, the Air Force must ensure that all of its systems have been accounted for and that component actions are successful; (3) it must also be well-positioned to make the resource tradeoff decisions that are inevitable in any year 2000 effort and to address conflicts between component approaches toward identifying and correcting interfaces; (4) further, it must be able to provide additional resources, such as testing facilities, that may be necessary to correct and validate systems; (5) the Air Force has taken a number of positive actions toward fulfilling its year 2000 oversight responsibilities; (6) for example, it is taking inventory of its systems and prioritizing them for conversion or replacement, and it has issued extensive guidance on dealing with the year 2000 problem; (7) it has also established a year 2000 working group comprised of focal points from the components which aims to eliminate duplicative efforts, share resources, and track component progress; (8) at the same time, the Air Force has not yet adequately addressed several critical issues that would ensure that it is well-positioned to deal with the later, and more difficult, phases of year 2000 correction; (9) GAO's review revealed that some components are failing to plan for the testing phase of their year 2000 effort and develop contingency plans; (10) GAO also found that some components are taking conflicting approaches toward determining the actual impact or the program status of their system interfaces; (11) if components and the Air Force do not promptly address and take consistent action on these issues, they may well negate any success they may have in making systems within their control year 2000 compliant; and (12) while the Air Force has enlisted the services of the Air Force Audit Agency to help address some of these concerns, this work needs to be backed by comprehensive and continued Air Force oversight in order to ensure that it can address unforeseen problems and delays in the next, more difficult phases.

section_title: Letter
paragraphs: This report presents the results of our review to date of the Air Force’s program for solving its Year 2000 computer systems problem. If this problem is not corrected in time, Air Force computer systems could malfunction or produce incorrect information. The impact of these failures could be widespread, costly, and debilitating to important Air Force warfighting and military support missions.
We performed this work as part of our review of the Department of Defense’s (DOD) Year 2000 computer systems efforts for the Chairman, Senate Committee on Governmental Affairs; the Chairman and Ranking Minority Member of the Subcommittee on Government Management, Information and Technology, House Committee on Government Reform and Oversight; and the Honorable Thomas M. Davis, III, House of Representatives. Our objectives were to evaluate (1) the status of the Air Force’s efforts to oversee its Year 2000 program and (2) the appropriateness of the Air Force’s strategy and actions for ensuring that the problem will be successfully addressed. This letter summarizes the concerns we raised and provides recommendations for addressing these issues.
subsections: section_title: Scope and Methodology
paragraphs: In conducting our review, we assessed the Air Force’s Year 2000 efforts against our own Year 2000 Assessment Guide. This guide addresses common issues affecting most federal agencies and presents a structured approach and a checklist to aid in planning, managing, and evaluating Year 2000 programs. The guidance, which is consistent with DOD’s Year 2000 Management Plan and the Air Force’s own Year 2000 management approach, describes five phases—supported by program and project management activities—with each phase representing a major Year 2000 program activity or segment. The phases and a description of each follows.
Awareness - Define the Year 2000 problem and gain executive-level support and sponsorship. Establish a Year 2000 program team and develop an overall strategy. Ensure that everyone in the organization is fully aware of the issue.
Assessment - Assess the Year 2000 impact on the enterprise. Identify core business areas and processes, inventory and analyze systems supporting the core business areas, and prioritize their conversion or replacement. Develop contingency plans to handle data exchange issues, lack of data, and bad data. Identify and secure the necessary resources.
Renovation - Convert, replace, or eliminate selected platforms, applications, databases, and utilities. Modify interfaces.
Validation - Test, verify, and validate converted or replaced platforms, applications, databases, and utilities. Test the performance, functionality, and integration of converted or replaced platforms, applications, databases, utilities, and interfaces in an operational environment.
Implementation - Implement converted or replaced platforms, applications, databases, utilities, and interfaces. Implement data exchange contingency plans, if necessary.
During our review, we concentrated primarily on the Air Force’s efforts to oversee its Year 2000 program during the awareness and assessment phases. We focused our review on Year 2000 work being carried out by (1) DOD’s Office of the Assistant Secretary of Defense for Command, Control, Communications, and Intelligence (OASD/C3I)—which is responsible for promulgating DOD guidance on Year 2000 and providing assistance to Defense components, (2) Air Force headquarters, including the Air Force Communications and Information Center (AFCIC)—which is responsible for day-to-day management and supervision and for issuing Air Force Year 2000 policy and guidance, (3) Air Force Communication Agency, the designated Air Force Year 2000 program office, and (4) selected program offices managed by the Air Force Materiel Command’s Aeronautical Systems Center at Wright-Patterson Air Force Base, Ohio.
To assess OASD/C3I efforts in providing Year 2000 support to the Air Force, we met with the Acting Assistant Secretary of Defense for Command, Control, Communications and Intelligence, the Principal Director for Information Management, the Director for Information Technology, and other senior staff responsible for Year 2000 issues. We reviewed the office’s Year 2000 guidance and other documentation on Year 2000 funding, reporting, and date format requirements.
To assess Air Force headquarters efforts to manage and oversee the Year 2000 computer problem, we (1) met with the Air Force Communications and Information Center officials and Year 2000 focal points, (2) obtained and analyzed documents issued by these offices that describe organizational structure and responsibilities for carrying out the Air Force Year 2000 program, and (3) reviewed the Air Force’s Year 2000 Guidance Package to assess the level of guidance, roles, and responsibilities, and target milestone dates for the Year 2000 effort.
Further, we obtained and analyzed the Air Force Year 2000 inventory data to determine (1) the number of systems owned and operated by Air Force organizations and (2) the status of Air Force systems in their Year 2000 efforts, proposed strategy, and the number of systems reporting to be compliant. We reviewed pertinent Year 2000 program documentation such as Defense and Air Force guidance and management directives, working group minutes, status reports, and cost and schedule data.
We performed our work primarily at the Air Force Materiel Command, Wright-Patterson Air Force Base, Ohio; Headquarters Air Force at the Pentagon, Washington, D.C.; and the Office of Assistant Secretary of Defense for Command, Control, Communications and Intelligence at Arlington, Virginia. We conducted our work from July 1996 through August 1997 in accordance with generally accepted government auditing standards. We received written comments on a draft of this report from the Chief Information Officer for the Department of the Air Force. His comments are discussed in the “Agency Comments and Our Evaluation” section and are reprinted in appendix II.
section_title: Background
paragraphs: Most of the Air Force’s automated information systems and embedded weapon systems are vulnerable to the Year 2000 problem, which is rooted in the way dates are recorded and computed in automated information systems. For the past several decades, systems have typically used two digits to represent the year, such as “97” representing 1997, in order to conserve on electronic data storage and reduce operating costs. With this two-digit format, however, the Year 2000 is indistinguishable from 1900, or 2001 from 1901, etc. As a result of this ambiguity, system or application programs that use dates to perform calculations, comparisons, or sorting may generate incorrect results when working with years after 1999.
Should Air Force computer systems fail on the morning of the Year 2000, Air Force operations at all levels could be affected by the incorrect processing of data, as well as corrupted databases, or even massive system failures. In turn, this could result in such problems as delays in supply shipments, faulty inventory forecasts, unreliable budget estimates, and erroneous personnel-related information. Moreover, the problem could adversely affect critical warfighting functions such as combat, communications, command and control, intelligence, surveillance, reconnaissance, and air traffic control.
Like the other military services, the Air Force has adopted DOD’s Year 2000 management strategy, which calls for centralized oversight with decentralized execution of Year 2000 correction. In February 1995, the Air Force designated the Air Force Communication Agency (AFCA) as the focal point for Year 2000 efforts, with responsibility for (1) coordinating Year 2000 efforts being carried out by its 9 major commands, 36 field operating agencies, and 3 direct reporting units, (2) ensuring that components completed Year 2000-related tasks on time, (3) developing Year 2000 guidance, (4) collecting and reporting progress and inventory-related data, and (5) chairing the Air Force Year 2000 working group which is comprised of representatives from components.
In April 1997, the Air Force established a Year 2000 program office at AFCA. The program office is currently staffed with 24 full-time personnel and it reports to the Air Force Communications and Information Center (AFCIC). AFCIC, which was established in April 1997 due to a Headquarters Air Force reorganization, was tasked with responsibility for implementing Year 2000 policy and programmatic changes across the Service. AFCIC also reports to the Office of the Chief Information Officer and it has assigned three full-time staff members to oversee the Air Force’s Year 2000 Program. Appendix I illustrates the Air Force’s Year 2000 organizational structure and describes the complexity involved in carrying out Year 2000 efforts at the command level.
Early in its Year 2000 effort, the Air Force introduced a five-phased management approach for addressing the Year 2000 problem, which was later adopted by DOD and the Federal Government CIO Council’s Year 2000 Subcommittee. According to Air Force officials, if properly implemented, this phased approach will enable the Air Force to achieve its goal of having every mission-critical system compliant by December 1998. The five phases and their supporting program and project management activities are consistent with those identified in our Year 2000 Assessment Guide, which draws heavily on the best practices work of the CIO Council’s Year 2000 Subcommittee.
In addition to following the five-phase approach, our guidance addresses common issues affecting most federal agencies and provides a checklist to aid them in planning, managing, and evaluating their year 2000 programs. Also, because the Year 2000 is a massive and complex management challenge, our guidance recommends that agencies plan and manage a Year 2000 program as a single large information system development effort and promulgate and enforce good management practices on the program and project levels.
To comply with DOD’s current Year 2000 funding mandate, the Air Force does not plan to provide system/program managers with any additional funds to manage and fix the Year 2000 problem. Rather, system/program managers have been directed to reprioritize or reprogram previously budgeted funds (primarily operational & maintenance (O&M) funds) to fix Year 2000 problems.
section_title: Current Status of Air Force Year 2000 Efforts
paragraphs: The Air Force estimates there are 2,944 automated information systems and weapons embedded systems in its inventory and that the majority of these systems will have to be either renovated, replaced, or retired before January 1, 2000. Of the 2,944 systems, 550 (about 19 percent) are considered to be mission-critical systems, that is, they directly support wartime operations. As of September 4, 1997, the Air Force reported that all of its 2,944 systems completed the awareness phase, 33 percent were in the assessment phase, 32 percent in renovation, 17 percent in validation, 12 percent were in implementation, and 6 percent will be decommissioned by December 1999. As of September 1997, the Air Force estimated that it will cost about $405 million to successfully complete its Year 2000 program. Table 1 details the status of Air Force systems according to their mission impact.
The Air Force has taken a number of positive steps to ensure that its personnel are fully aware of the impact should Air Force systems not be compliant at the turn of the century. For example, in November 1995, the Air Force established a Year 2000 working group comprised of focal points from each major command, field operating agency, and direct reporting unit. This group has focused on such matters as sharing lessons learned, eliminating duplicative efforts, sharing resources, and tracking component progress. In the same month, the Air Force released an Air Force-wide impact assessment survey to all major commands, field operating agencies, and direct reporting units for the purpose of obtaining a rough order-of-magnitude of the Year 2000 problem throughout the Air Force. The results of this survey indicated that the Air Force Year 2000 problem would be significant and that it required immediate and sustained management attention.
The Air Force has also addressed a number of steps associated with the assessment phase of Year 2000 correction, including the following.
Developing a comprehensive Air Force-wide system inventory, which will include information on information systems, weapons systems, and infrastructure-related devices that could be affected by the Year 2000 problem.
Prioritizing systems for conversion or replacement according to their mission impact.
Tasking the Air Force Software Technology Support Center at Hill Air Force Base, Utah, to evaluate in-house and vendor tools and services that could be used to identify and fix Year 2000 problems.
Creating a dedicated Year 2000 database, which contains system inventory-related information as well as information on component progress. Issuing a Year 2000 Guidance Package for senior managers and Year 2000 points-of-contact, which (1) explains how to prepare individual project management plans and develop Year 2000 strategies, (2) includes milestones and exit criteria for Year 2000 tasks, (3) provides a flowchart illustrating the five-phase resolution process, and (4) provides cost estimating formulas. This package is continually updated to reflect new managerial, technical, legal and other Year 2000-related developments.
Developing a checklist to assist system managers in ensuring that their systems are compliant for the Year 2000, which covers (1) the identification of systems and interfaces, (2) assessment of date usage by the systems, and (3) compliance testing, among other subjects.
Directing each major command and field operating agency to appoint Year 2000 certifiers to ensure that all systems belonging to the components have completed the necessary steps to become Year 2000 compliant.
subsections: section_title: The Assessment Phase Is Running Behind Schedule
paragraphs: The Air Force originally anticipated that it would complete the assessment phase of its Year 2000 effort in May 1997. It acknowledged that approximately 66 percent of its systems did not meet this deadline and it subsequently revised the deadline to October 1997. However, as of September 4, 1997, about 33 percent of its systems had still not been assessed. With less than 26 months remaining before the Year 2000 deadline, this will add pressure on the Air Force to renovate, validate, and implement systems as quickly as possible. According to an industry expert, June 1997 is apt to be the latest point in time to start fixing systems and to have a reasonable probability of finishing before year 2000. The Air Force’s Year 2000 guidance, as well as GAO and OMB’s Year 2000 guidance, call for a similar completion date. In addition, according to the Gartner Group—an independent contractor hired by Defense to provide Year 2000 technical support primarily in the areas of scheduling and cost estimating—no more than 26 percent of an organization’s total Year 2000 effort should be spent in the awareness and assessment phases. Our analysis shows that the Air Force has used nearly 46 percent of its available time to complete these two phases. While Air Force officials acknowledge that the assessment phase is taking longer than expected, they do not believe it will significantly affect their Year 2000 program because system and program managers have already begun to fix systems identified with Year 2000 problems.
One reason for the delay in completing the assessment phase is that it has taken longer than anticipated to develop a complete systems inventory. Before its Year 2000 effort, the Air Force did not have a comprehensive servicewide system inventory. As such, it could not readily determine the magnitude (much less the cost to fix) of the Year 2000 problem servicewide when it began the assessment phase. While its inventory now contains 2,944 systems, the Air Force is still expanding it to include information on infrastructure-related devices, such as elevators, traffic control and security devices, telephone switching systems, and medical equipment. These devices rely on either microprocessors or microcontroller chips that may be vulnerable to Year 2000 problems. In addition, the Air Force is contending with slow and incomplete reporting by system and program managers. As a result, it has revised reporting requirements to facilitate better reporting on the part of its components.
Furthermore, the Air Force must still resolve discrepancies between its inventory and recent findings by the Air Force Audit Agency. In June 1997, the Audit Agency identified over 6,000 information systems that were not included in the Air Force inventory (which contained 2,543 systems at the time this audit was conducted). These additional systems included 1,600 mission-critical systems. The Air Force is currently reconciling its database to the audit findings.
The Air Force has recently enlisted the Air Force Audit Agency to help evaluate component progress in completing the assessment phase. The agency will determine whether selected components have (1) completed timely assessments, (2) addressed all system interfaces, (3) accomplished mandatory system certifications, (4) prioritized and scheduled required renovations, and (5) developed contingency plans.
section_title: Key Issues Need Prompt Attention for the Air Force to Solve Its Year 2000 Problem
paragraphs: Even though the Air Force is entering the next phases of its Year 2000 correction effort, it has yet to complete several critical assessment steps, which are designed to ensure that it is well-positioned to deal with the later, and more difficult, phases of Year 2000 correction. These include (1) recalculating its $405 million cost estimate, based on actual assessment data, so that it can make informed choices about information technology priorities, (2) ensuring that interfaces are properly accounted for, (3) ensuring that components are developing contingency plans, and (4) ensuring that components are adequately prepared for the testing phase. The Air Force Audit Agency audit should help the Air Force complete these steps; however, this work will be carried out only at selected sites and it will not provide the comprehensive and continued oversight that is needed to ensure that the Air Force can handle unforeseen problems and delays.
subsections: section_title: Full Cost of Year 2000 Problem Needs to Be Determined
paragraphs: As DOD’s Year 2000 Management Plan and our Year 2000 Assessment Guide state, the primary purpose of the assessment phase is to gather and analyze the information in order to determine the size and scope of the problem. Among other things, this enables an agency to estimate the cost of its Year 2000 effort in terms of dollars and work years, and, in turn, to make informed choices about information technology priorities and whether other system development efforts should be deferred or canceled so that resources can be freed up to solve the Year 2000 problem. The Air Force, however, has not yet fully defined the scope of its Year 2000 problem or refined cost estimates, using actual assessment data, in order to gauge what resources are needed for correction. The need to take immediate action in this regard is critical, given that some organizations are already discovering that they do not have sufficient funding to correct their systems.
Currently, the Air Force expects to spend about $405 million from fiscal year 1997 through 1999 to fix its Year 2000 problem. Table 2 breaks down the estimated cost by fiscal year.
According to AFCIC officials, the cost estimate was calculated using the Gartner cost formula, which recommends multiplying $1.10 by the lines of code contained in the agency’s automated information systems and $8.00 by the lines of code for weapon systems. The Gartner method is helpful in developing a rough estimate of what it will cost to resolve the problem early in the Year 2000 effort. However, according to a directive from Defense’s Chief Information Officer as well as Year 2000 consultants, agencies should refine their cost estimates as they progress through the assessment phase and into the later Year 2000 phases to factor in the actual resources they believe are needed to renovate and implement their systems. According to DOD’s Year 2000 Management Plan, these can include:
The age of the systems being corrected. Age can have a significant impact on the cost of correction since older code tends to be less structured and thus harder to understand and correct than newer code.
The Year 2000 strategy that the program is pursuing. Strategies that involve keeping the two-digit code, for example, are much less expensive than those that involve changing the two-digit code to a four-digit code.
The degree of documentation that is available on the system and its understandability and the availability of source code.
The skill and expertise of in-house programmers.
Projected engineering costs.
Labor hours required to fix systems.
Testing requirements.
The September estimate still used the Gartner formula and did not take into account other factors that can have a significant impact on the cost of correction including those identified in DOD’s Year 2000 Management Plan. Air Force officials acknowledged that the $405 million estimate is a rough figure. They planned to re-estimate costs at some point after the assessment phase is completed. Costs should be continuously reestimated through the assessment and subsequent Year 2000 phases. By waiting to refine its cost estimates, the Air Force will be delaying the availability of information needed to make informed resource trade-off decisions.
In fact, trade-off issues and other funding disputes, which call for the need to develop more accurate cost estimates, have already surfaced in some Air Force programs. For example, one aircraft weapon system program found that correcting the Year 2000 problem in ground software equipment that is used to program the aircraft’s operational avionics software for navigation and weapons delivery would cost $42 million more than what was budgeted for routine maintenance of the aircraft. In August 1997, the program office reported that it fixed the problem for about $300,000 using a temporary workaround. However, according to a program office official, because the existing equipment consists of old IBM mainframes and outdated Jovial code it will have to be replaced eventually—and likely at a higher cost—in order to support future planned aircraft enhancements such as Joint Direct Attack Munition and Joint Standoff Weapon.
In addition, the Air Force estimates that it will cost between $70 million and $90 million to fix telephone switches throughout the Service. This estimate is not included in the $405 million total Air Force Year 2000 cost estimate. The Air Force is currently in a dispute with the contractor that supplied the switches over who is responsible for Year 2000 correction. At the same time, Air Force components have not budgeted funds to fix their telephone switches. Since then, and according to AFCIC officials, the Air Force has begun to address this funding issue through its normal corporate funding process.
section_title: System Interfaces Need More Attention
paragraphs: It is critically important during the Year 2000 effort that agencies protect against the potential for introducing and propagating errors from one organization to another and ensure that interfacing systems have the ability to exchange data through the transition period. According to our Year 2000 Assessment Guide, to address the issue of interfaces, agencies should (1) identify their internal and external interfaces, (2) determine the need for data bridges and filters, (3) notify outside data exchange partners of their interface plans, (4) test their interface correction strategies, and (5) develop contingency plans that address the possibility of failing to receive data from an external source or receiving invalid data. DOD’s Year 2000 Management Plan places responsibility on component heads or their designated Year 2000 points of contact to document and obtain system interface agreements in the form of memorandums of agreement or the equivalent.
Since October 1996, the Air Force has participated in six high-level DOD Year 2000 interface workshops, including finance, intelligence, command and control, communications, logistics, and weapons systems. However, to date, the Air Force has not been tracking (1) how its components are going about identifying their interfaces, (2) how they plan to correct interfaces, and (3) whether they are instituting memorandums of agreement in order to communicate their interface plans to their data exchange partners. It is important for the Air Force to immediately begin tracking these issues since individual components are embarking on varying—and possibly conflicting—approaches to addressing interfaces. Moreover, others have not yet addressed the interface issue.
For example, none of the five weapon system program offices we surveyed had fully determined the actual impact or program status of their system interfaces. One program office told us that it did not plan to do so until the Air Force prescribed a uniform approach to interfaces. In addition, we found other weapon system program approaches to identifying their interfaces to be considerably different. For example, the F-22 weapon systems program formally requested its development contractor, in writing, to assess the impact of the Year 2000 problem on the aircraft. This assessment would include identification of interfaces and an evaluation on whether they pose a Year 2000 problem. By contrast, the F-16 program office planned to informally contact its subcontractors to identify the status of interfaces and Year 2000 issues for on-board components of the aircraft that the program office does not directly manage. For components that the program office directly manages, it plans to informally request that its contractor assess Year 2000 problems and identify the status of interfaces. However, that assessment will not be documented as the F-22 program office’s assessment will be. Clearly, the second approach will provide the Air Force with less assurance that all interfaces have been accounted for than the first approach.
Without centralized oversight over the identification and correction of interfaces, there is a chance that some systems and interfaces, for which ownership is unclear, may not be identified and corrected. In addition, there is also a higher risk that conflicting interface solutions will be implemented without the data bridges that are necessary to ensure that information can still be transferred. For example, one system manager may choose to fix a system by expanding its date and year, while another may choose to keep the two-digit format and use procedural code or sliding windows as a strategy for becoming Year 2000 compliant. According to current Defense guidance, either fix is acceptable, but both parties need to know of the potential conflict so that they can install the data bridge.
AFCIC plans to recommend that responsible system/program managers prepare interface memorandums of agreement, which describe the method of interface and assign responsibility for accommodating the exchange of data. If implemented, these agreements could ensure that information can be transferred even when components take conflicting approaches to their interfaces. At the time of our review, however, none of the five program offices we visited had prepared such agreements, and the Air Force was not tracking whether these or comparable agreements were being instituted.
section_title: Air Force Is Not Ensuring Components Are Planning for Testing
paragraphs: Our Year 2000 Assessment Guide calls on agencies to develop validation strategies and test plan, and to ensure that resources, such as facilities and tools, are available to perform adequate testing. This planning should begin in the assessment phase since agencies may need over a year to adequately validate and test converted or replaced systems for Year 2000 compliance and since the testing and validation process may consume over half of the Year 2000 program resources and budget.
At the time of our review, however, the Air Force was not ensuring that components were developing test plans. It was also not assessing the need for additional testing resources, even though it acknowledged that these resources would be in demand. Instead, AFCIC officials told us that they are relying heavily on system/program managers to organize, plan, and manage the necessary resources to test Year 2000 fixes. Our review showed that more attention is needed in this area. For example, none of the five program offices we surveyed had completed a master Year 2000 test plan.
Due to the complexities and risks involved with testing, components that are not currently planning their testing strategies run a high risk of not completing the Year 2000 effort on time. This is because components must not only test the year 2000 compliance of individual applications, but also the complex interactions between scores of converted or replaced computer platforms, operating systems, utilities, applications, databases, and interfaces. Moreover, in some instances, components may not be able to shut down their production systems for testing and thus have to operate parallel systems implemented on a year 2000 test facility. Components may also find that they need computer-aided software testing tools and test scripts to help prepare and manage test data, automate comparisons of test results, and schedule tests. AFCIC officials themselves believe that there is a good chance that adequate test facilities may not be available to conduct joint interoperability testing involving systems that interface with one another. For these reasons, it is critical that Air Force headquarters ensure that components are taking time now to assess their testing needs and that the Air Force is well-positioned to provide components with additional testing facilities and tools.
In August 1997, the Air Force working group began to address this testing issue in part by directing its components to identify and develop an inventory of existing testing facilities that could support Year 2000 testing of selective platforms such as Unisys and IBM. This effort is ongoing.
section_title: Required Contingency Plans Not Being Prepared
paragraphs: DOD’s Year 2000 Management Plan and our Year 2000 Assessment Guide call on agencies to develop realistic contingency plans during the assessment phase for critical systems and activities to ensure the continuity of their core business processes. Contingency plans are important because they identify the manual or other fallback procedures to be employed should some critical systems miss their Year 2000 deadline or fail unexpectedly even after they are found to be compliant. Contingency plans also establish a series of checkpoints that allow the agency to identify performance problems early enough to correct them.
The Air Force itself has acknowledged that components need to develop contingency plans and it has directed system/program managers to prepare, at a minimum, contingency plans for all mission-critical systems. It has also incorporated this requirement into its assessment phase exit criteria.
However, the Air Force has not been tracking the extent to which components have prepared plans for mission-critical functions/systems. Without greater oversight over the preparation of such plans, some components may fail to adequately plan for contingencies without the Air Force’s knowledge. In fact, at the time of our review, none of the five system program offices we surveyed had prepared contingency plans. Officials from these offices told us that contingency plans are not needed because they believed that their systems did not require extensive Year 2000 work and thus their corrections would be made before the Year 2000 deadline expired. In addition, they did not believe that contingency planning was cost-effective.
All Air Force organizations need to be engaged in contingency planning since there is no guarantee that the corrections they will make will be completed on time or be free of unforeseen problems. As such, according to DOD’s Year 2000 Management Plan, components, at a minimum, need to (1) analyze the impact of a system failure, (2) identify alternative activities—including manual or contract procedures—to be employed should critical systems fail to meet their Year 2000 deadline, and (3) identify procedures and responsibilities for implementing such alternatives. Furthermore, given the dangers associated with not having contingency plans, we believe the Air Force headquarters’ oversight responsibility must involve ensuring that all components are planning for contingencies for mission-critical systems.
section_title: Conclusions
paragraphs: To its credit, the Air Force has recognized that virtually every computer system it operates is vulnerable to the Year 2000 problem, it has raised the awareness of the Year 2000 problem among system owners, and it has begun assessing the Year 2000 impact on Air Force systems. However, the Air Force is unnecessarily putting its Year 2000 program at risk of failure because it has not yet refined cost estimates based on actual assessment data, fully examined resource trade-offs, and ensured strong and continuous oversight for interface, testing, and contingency planning issues. Because these steps are designed to ensure that organizations are well-positioned to deal with the more difficult stages of Year 2000 correction, neglecting any one of them can seriously endanger the Air Force’s ability to meet its Year 2000 deadline. Given its role in national security, and its interdependence with other military organizations, the Air Force cannot afford this risk.
section_title: Recommendations
paragraphs: We recommend that the Secretary of the Air Force immediately require that the Air Force ensure its cost estimates factor in the actual resources it believes are needed to renovate and implement systems so that the Service can make informed resource trade-off decisions and ensure that this estimate is periodically refined throughout the Year 2000 program.
We also recommend that the Secretary ensure that an approach is developed to continuously track how components are going about identifying interfaces, how they plan to correct interfaces, and whether they are instituting memorandums of agreement.
In addition, we recommend that the Secretary ensure that components are developing test plans and identifying the need for additional testing resources and design an approach to obtain any needed testing resources that are identified by Air Force components.
Finally, we recommend that the Secretary act to ensure that components have prepared contingency plans for their mission-critical systems.
section_title: Agency Comments and Our Evaluation
paragraphs: In written comments on a draft of this report, the Office of the Air Force Chief Information Officer agreed with all of our recommendations to improve the Air Force’s Year 2000 program. In response to our recommendations, the Air Force agreed to update its cost estimates as it progresses through the remaining Year 2000 phases and include actual resources needed to renovate and implement systems so that it can make informed resource trade-off decisions. The Air Force also agreed to place greater management attention on identifying system interfaces and improve reporting practices to ensure that interface corrections are properly accounted for and can be readily tracked. In addition, the Air Force agreed to have major commands and product centers outline and prioritize their test requirements to ensure that testing resources will be available when needed.
The Air Force pointed out that it is working with components to develop Year 2000 contingency plans as part of the renovation and validation phases. In addition, the Air Force plans to open servicewide crisis response centers around August or September 1999 to deal with critical systems that will not be Year 2000 compliant by January 1, 2000. The Air Force is taking steps to ensure that contingency plans will be prepared on each noncompliant system identified and be made readily available to the crisis response centers. The full text of Air Force’s comments is provided in appendix II.
This report contains recommendations to you. The head of a federal agency is required by 31 U.S.C. 720 to submit a written statement on actions taken on these recommendations to the Senate Committee on Governmental Affairs and the House Committee on Government Reform and Oversight not later than 60 days after the date of this report. A written statement also must be sent to the House and Senate Committees on Appropriations with the agency’s first request for appropriations made more than 60 days after the date of this report.
We appreciate the courtesy and cooperation extended to our audit team by Air Force officials and staff. We are providing copies of this letter to the Chairman and Ranking Minority Members of the Senate Committee on Governmental Affairs; the Subcommittee on Oversight of Government Management, Restructuring and the District of Columbia, Senate Committee on Governmental Affairs; the Subcommittee on Defense, Senate Committee on Appropriations; the Senate Committee on Armed Services; the Subcommittee on Government Management, Information, and Technology, House Committee on Government Reform and Oversight; the Subcommittee on National Security, House Committee on Appropriations; and the House Committee on National Security. We are also sending copies to the Honorable Thomas M. Davis, III, House of Representatives; the Deputy Secretary of Defense; the Acting Assistant Secretary of Defense for Command, Control, Communications and Intelligence; the Air Force Chief Information Officer, Department of Defense; and the Office of Management and Budget; and other interested parties. Copies will be made available to others on request.
If you have any questions on matters discussed in this letter, please call me at (202) 512-6240 or John B. Stephenson, Assistant Director at (202) 512-6225. Major contributors to this report are listed in appendix III.
section_title: Air Force Year 2000 Organizational Structure
paragraphs: As figure I.1 below indicates, the size and complexity of the Air Force’s organization structure will pose a significant management challenge. Year 2000 management and oversight efforts will have to be coordinated among 9 major commands, each with complex and diverse organizational structures of their own, 3 direct reporting units, and 36 field operating agencies.
Figure I.2 provides an example of just one command’s organizational structure. To understand the complexity involved in carrying out Year 2000 efforts at the command level, consider the following: the Air Force Materiel Command employs about 112,000 personnel; the command manages about 1,700 computer applications and embedded systems;
175 of these systems cover 21 various types of aircraft, including the F-22 and F-16 fighters, the B-1 and the B-2 bombers, and C-17 cargo plans;
410 of these systems are business applications;
266 of these systems are applications covering command, control, communications and intelligence activities;
915 of these systems are base-level owned and operated applications, such as local area networks and medical systems; and the Air Force Materiel Command alone has about 50 Year 2000 points-of-contact.
Air Force Chief of Staff 11th Wing, Bolling AFB,  Washington, D.C.
subsections: section_title: Figure I.2: The Air Force Materiel Command Organizational Structure
paragraphs: Air Force Office of  Scientific Research,  Bolling AFB,  Washington, D.C.
section_title: Comments From the Department of the Air Force
section_title: Major Contributors to This Report
subsections: section_title: Accounting and Information Management Division, Washington, D.C.
section_title: Chicago/Dayton Field Office
paragraphs: Robert P. Kissel, Jr., Senior Evaluator Steven M. Hunter, Senior Evaluator Robert G. Preston, Senior Evaluator The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 37050 Washington, DC  20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000  or by using fax number (202) 512-6061, or TDD (202) 512-2537.
Each day, GAO issues a list of newly available reports and testimony.  To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone.  A recorded menu will provide information on how to obtain these lists.