Defense Computers: Improvements to DOD Systems Inventory Needed for Year 2000 Effort
Released: 1997-08-13T00:00:00

Letter

Scope and Methodology

DISA was included in our review because of its unique role in DOD’s information processing and the Year 2000 process. DISA is responsible to the ASD/C3I for maintaining DIST as the Department’s enterprise inventory database and its primary tool for performing oversight of the Year 2000 correction efforts. In assessing DIST’s effectiveness in facilitating Year 2000 efforts, we interviewed DIST managers and a representative from the contractor. Since the services have different approaches to entering data in DIST, we spoke to officials at various organizational levels regarding ease of use and how they are entering information.

In addition, we analyzed the contents and capabilities of DIST to gauge its accuracy, performance, reliability, and usefulness as a Year 2000 enterprise inventory database. In conducting this analysis, we relied on our previous work on DIST which was conducted as part of a review on Defense’s migration strategy—a DOD effort focused on improving and modernizing automated information systems. We also reviewed Air Force and Army comparisons of DIST inventories against their own inventories. In addition, we assessed whether DIST conformed to system inventory-related guidance included in our Year 2000 Assessment Guide, and DOD’s Year 2000 Guidance Package and Year 2000 Management Plan. We specifically focused on the Assessment Phase of the Year 2000 process described below, during which agencies are to develop an enterprise inventory. We conducted our work from November 1996 through July 1997 in accordance with generally accepted government auditing standards.

The Department of Defense provided written comments on a draft of this report. These comments are discussed in the “Agency Comments and Our Evaluation” section and are reprinted in appendix I.

Background

Under DOD’s Year 2000 Management Plan, DISA is responsible for enhancing and maintaining DIST as a Year 2000 enterprise inventory tool. In February 1997, we published the Year 2000 Computing Crisis: An Assessment Guide, which addresses common issues affecting most federal agencies and presents a structured approach and a checklist to aid them in planning, managing, and evaluating their Year 2000 programs. The guidance is consistent with DOD’s Year 2000 Management Plan. The guide describes five phases—supported by program and project management activities—with each phase representing a major Year 2000 program activity or segment. The phases and a description of what each entails follow.

Awareness: Define the Year 2000 problem and gain executive-level support and sponsorship. Establish a Year 2000 program team and develop an overall strategy. Ensure that everyone in the organization is fully aware of the issue.

Assessment: Assess the Year 2000 impact on the enterprise. Identify core business areas and processes, inventory and analyze systems supporting the core business areas, and rank their conversion or replacement. Develop contingency plans to handle data exchange issues, lack of data, and bad data. Identify and secure the necessary resources.

Renovation: Convert, replace, or eliminate selected platforms, applications, databases, and utilities. Modify interfaces.

Validation: Test, verify, and validate converted or replaced platforms, applications, databases, and utilities. Test the performance, functionality, and integration of converted or replaced platforms, applications, databases, utilities, and interfaces in an operational environment.

Implementation: Implement converted or replaced platforms, applications, databases, utilities, and interfaces. Implement data exchange contingency plans, if necessary.

In addition to following the five phases described, a Year 2000 program should also be planned and managed as a single, large information system development effort. Agencies should promulgate and enforce good management practices at the program and project levels.

System Inventories Are Integral to Correcting the Year 2000 Problem and Managing Information Technology Resources

As discussed in our Year 2000 Assessment Guide, agencies need to ensure that they have complete and accurate enterprisewide inventories of their information systems during the assessment phase of the Year 2000 correction effort. This inventory helps the agency analyze the systems supporting its core business processes and rank its conversion or replacement based on key factors, such as business impact and the anticipated date the systems would experience Year 2000-related date problems.

The inventory also plays a very critical role in the later stages of the Year 2000 process, which include renovation, validation, and implementation. For example, the inventory can be used in monitoring the status of each system included in DOD’s Year 2000 efforts, assessing whether the most mission-critical systems are receiving appropriate attention, determining needs for testing facilities, and identifying areas that may require additional resources. The inventory can also assist in identifying and coordinating interfaces between and among systems. Even if all systems within one organization were made Year 2000 compliant, an external interfacing system on which the system is dependent for data or information processing can still introduce and propagate Year 2000-related errors.

Having an accurate and reliable enterprisewide systems inventory is also fundamental to having a good information technology investment process. In today’s environment of rapidly changing information technology and the demands for government organizations to operate effectively and more efficiently, agencies need to ensure that their information technology projects are being implemented at acceptable costs, within reasonable and expected time frames, and are contributing to tangible, observable improvements in mission process. In order to make the kinds of trade-off decisions that would produce these benefits, good visibility into their information system environment is indispensable. The enterprisewide inventory of information systems provides this visibility. In addition, Defense will need a reliable and complete system inventory in order to successfully implement the recently passed Clinger-Cohen Act of 1996, which aims to ensure that agencies strengthen their information technology investment processes. Among other things, this act requires that agencies (1) provide their senior managers with timely and accurate information on system costs and (2) have the capability to meet performance requirements, timeliness, as well as other conditions.

As discussed in our Year 2000 Assessment Guide, system inventories serve as a useful Year 2000 decision-making tool, by offering added assurance that all systems are identified and linked to a specific business area or process, and that all enterprisewide cross boundary systems are considered. Thus, good inventories include information for each system on (1) links to core business areas or process, (2) systems platforms,languages, and database management systems, (3) operating system software and utilities, (4) telecommunications, (5) internal and external interfaces, (6) systems owners, and (7) the availability and adequacy of source code and associated documentation.

Importance of DIST for DOD’s Year 2000 Efforts

Defense has designated the Defense Integration Support Tools database to be the departmentwide automated information systems inventory for use in making information technology decisions and managing the Year 2000 effort. DIST was originally designed to track Defense migration systems for the Corporate Information Management initiative but has evolved into a multipurpose tool. DIST presently contains over 9,000 systems and has a total capacity of 40,000. Each system is provided with its own identification number and should be accompanied by a host of informative data elements, including information on hardware platforms, operating systems, applications languages, communications, and interfaces.

Early in its Year 2000 effort, DOD recognized the value of having a reliable enterprisewide system inventory and the potential beneficial role its DIST database could have in the initiative. For example, in November 1996, the Under Secretary for Defense (Comptroller) and the Assistant Secretary of Defense for Command, Control, Communications and Intelligence issued a joint memorandum to senior Defense managers stating that they considered DIST to be “the backbone tool for managing the Department’s Information Technology investment strategies, identifying functional information systems interfaces and data exchange requirements, and managing the efforts to fix the Year 2000 problem.”

In its Year 2000 Management Plan, Defense reaffirmed that DIST will be the official repository for the DOD components and added that the reason components are required to report every quarter on their systems and are encouraged to report significant progress on their systems is “to give DOD the visibility necessary to ensure a thorough and successful transition to Year 2000 compliance for all DOD systems.” It also stated that this reporting “will also keep other functional , that your systems interface with or exchange data with, informed as to the status of your Year 2000 compliance progress.” Finally, Defense noted that the DIST needed to be up-to-date so that it could keep the Congress informed on the Department’s efforts to achieve Year 2000 compliance.

DOD Recognizes DIST Data Integrity Problems

Defense has recognized that DIST is currently not a reliable and accurate management tool that can have a beneficial impact on the Year 2000 effort or on other initiatives to improve and manage information systems. As a result, the ASD/C3I and DISA have undertaken initiatives to improve the reliability of DIST data and to increase their user friendliness. These efforts will address a wide range of problems associated with data integrity and the ability of users to have direct and quick access to the database.

During our review, DOD officials and users told us that updating DIST was traditionally a low priority for the services and components largely because DIST is an antiquated and labor-intensive system. A number of officials also told us that they have grown frustrated with DIST because it contains erroneous data and that they are now reluctant to use DIST because they do not have confidence in the accuracy or reliability of the data it contains. Our analysis of DIST as well as comments by officials in DOD components have revealed significant data integrity problems associated with DIST’s ability to transfer information to other information systems. The following examples below illustrate the magnitude and range of problems pervading the database.

DIST managers, service-level Year 2000 teams, and component Year 2000 teams acknowledge that the database contains duplicate, outdated, and erroneous information. The Air Force’s Year 2000 team compared its own Year 2000 database to DIST and found over 1,100 systems that were shown on DIST but not on its database. The Army’s Year 2000 team found a discrepancy of over 200 systems when it compared its system inventory to the DIST. The Army team also stated that it does not trust the data in DIST and that it would continue to update and rely on its own Year 2000 database instead of DIST. Air Force, Army, and DIST Year 2000 focal point representatives agree that until DIST is purged of duplicate, outdated, and erroneous information, the service-level databases contain the most accurate inventories for those agencies.

Many systems in DIST do not have complete status and descriptive information. Each entry in DIST is supposed to include over 140 data elements, such as name, size, system manager, software, hardware, and interfaces. But for many systems, managers responsible for the systems have merely entered “placeholder” information, that is, the bare minimum of information required to get the system into the database. In some cases, this may mean that only the system name appears in the database. At present, DIST contains an undetermined amount of these incomplete entries. However, a February 1997 Defense analysis of migration systems listed in DIST illustrated that there are high levels of incomplete data. The analysis, which was conducted on the 223 migration systems included in DIST, found that

55 percent of the migration systems did not identify interfaces with other systems,

77 percent did not disclose the computer installations where the system

68 percent did not indicate the computer hardware on which the system

61 percent did not disclose the system software, and

26 percent did not identify the organization responsible for the system.

When we analyzed DIST as part of our review of Defense’s migration effort, we also found that the database contained a high number of inaccurate system implementation and termination dates. For example, for three functional areas—clinical health, civilian personnel, and transportation—DIST showed that 92 legacy systems were terminated by April 1996, while functional managers told us that only 43 had actually been terminated. And, DIST showed that 53 legacy systems were scheduled for future termination, but functional managers told us 91 were slated for termination.

Our migration review also found that DOD had not ensured that the data definitions used in DIST were fully compatible with data maintained in other Defense information systems that track and report on systems. Without standard definitions and formats, data cannot be easily transferred to DIST from other systems that may be used by the DOD Principal Staff Assistants, program managers, and other decisionmakers.

Although DOD has progressed in populating the DIST database, component officials told us that they have been confused about what is to be entered. Since Year 2000 efforts began, for example, components were unsure what qualifies as a system. The Office of the ASD/C3I has just recently addressed the issue in a memo and its DOD Year 2000 Management Plan. The plan now states that mission-critical systems, migration systems, legacy systems, systems with an annual operating budget over $2 million, and any system that interfaces with the previous criteria must be reported to DIST. All other systems must be accounted for in a “one-line entry” to the ASD/C3I office. This new criteria will prompt DOD systems managers to revisit their Year 2000 project plans and apply this new criteria for reporting.

Component and service officials indicated that inputing information into DIST is time consuming and difficult, and the rules for entering and updating data are unclear. For example, database tables that would provide information on hardware manufacturers, series, and models are not up-to-date. Yet, as late as May 1997, no new entries on these hardware data elements were allowed to be made to the database. Also, while DOD components are required to enter Year 2000-related information on weapons systems into DIST, the database itself was not designed to apply to weapon systems or embedded systems. Without guidance on what data elements are applicable to what type of system, it is difficult to decide what information to enter on weapon systems and embedded systems.

Component and service officials indicated that DIST cannot be easily queried and does not provide timely feedback. For example, components and services cannot directly query DIST for information. Instead, they have to request that a query be made by DIST managers. The lack of user friendliness and querying capabilities has compounded the level of distrust in DIST by service and component-level managers responsible for addressing the Year 2000 problem and further diminished the incentive to keep the database updated.

DIST also does not contain key scheduling and tracking information, such as when critical systems within the services’ and components’ Year 2000 programs will be in the various phases and whether a system is behind schedule. Managers of interfacing systems need to know this information to coordinate key Year 2000 activities such as the start of system renovation, testing, and implementation of the modified system and to determine, as well as whether software bridges will be necessary.

Because the data in DIST are incomplete, inaccurate, and difficult to use, a number of Defense components and military services have developed and are relying on their own system inventories to manage and oversee their Year 2000 efforts. During our review, however, officials from the Navy informed us that they will be using DIST for their Year 2000 efforts because they do not have a servicewide inventory of their own.

DOD Efforts to Address DIST Problems

DIST managers are planning to implement new releases in September and October 1997 to make DIST a more user friendly tool and enable the services and components to directly query the database. They are also planning to increase the accuracy of the tool by developing a purging methodology to validate the data in DIST.

The new DIST releases, which DISA has made partially available and plans to make fully available by October 1997, are designed to make it easier to input changes into DIST through the use of such features as on-line help pages, navigational buttons, and expanded tables on hardware and software types. The new versions are also designed to make it easier to send and receive database information. While the services and components will be able to directly query the database for some types of information, they will not be able to enter or obtain data related to the Year 2000 problem, such as progress-related information that we believe is necessary for effective system management and departmentwide oversight of Year 2000 program status.

The purging methodology is the first step of a systematic program of improving the quality and accuracy of DIST data. Its purpose is to identify duplicate, inactive, and incomplete data. DIST managers cautioned that the purge has to be done carefully. While some older systems may be obsolete, they may be attached to smaller, feeder systems which are not obsolete. These smaller systems may not be readily identifiable on the database. Other systems that may appear obsolete on the database may actually be older legacy systems with no recent updates.

At the end of January of 1997, DIST officials told us that it would take 90 days just to determine the methodology for the purge. However, as of July 1997, the methodology to purge the DIST database and ensure the validity of information it contains had not been completed. DISA officials told us that their inability to obtain funds to make the needed improvements was the reason for delays in completing DIST modifications. Although the ASD/C3I recently provided $2.5 million in funding for the upgrades, this delay has resulted in the database not being valid and usable for managing corrective actions while most of DOD is in the assessment phase, a phase which the Department as a whole planned to complete during June 1997. DOD’s unwillingness to fund needed improvements to DIST until recently is inconsistent with both its previously stated importance of DIST to DOD’s Year 2000 program, and the ability of DIST to be the primary tool of DOD’s future information technology efforts.

Efforts to improve DIST may be further slowed by the failure of the military services and their components to input information on all of their systems into the database. The DOD Comptroller and the ASD/C3I recognized that earlier calls for the services and components to enter information into DIST did not succeed in completing the inventory. Consequently, they have set deadlines for entering this information and warned the services and components that if their systems were not entered into the database, they would risk losing funding for them. However, this deadline has been changed several times—from January 15, 1997, to March 5, 1997, to April 18, 1997. A DISA spokesperson recently reported that a new deadline would be established because they have not completed the DIST upgrade. Accordingly, as the June 1997 deadline for completion of the Year 2000 assessment phase for the Department passed, the database still remained incomplete. We believe that if DIST improvement efforts are not expedited, the inventory will be of little use to the services and components during the remaining critical stages of the Year 2000 correction efforts as well.

The potential consequences of not having this inventory for the assessment phase and the remaining phases of the Year 2000 effort are significant. First, without having a complete and reliable DIST during the assessment phase, DOD organizations that plan to use DIST would not have it as a management tool for ranking systems based on their importance to their mission and, in turn, ranking systems for correction. Many DOD components can utilize their own inventories, assuming they are accurate and reliable, to do this, but the Navy will not be able to since it does not have a servicewide inventory and it was planning to use DIST for this purpose. Second, the Department as a whole will be constrained in its ability to ensure that all systems owned by the military services and components are being made Year 2000 compliant. While the Department can use individual service and component inventories for this purpose, there is a chance that some systems which fall between the boundaries of ownership of the components may not be reflected in any inventory. Third, without an enterprisewide inventory, Defense cannot adequately ensure that all interfaces are properly identified and corrected. Fourth, for DIST to be an effective enterprise inventory, it is necessary to add data fields that provide DOD, the components, and the individual organizations with a much needed mechanism to track the progress of both the overall program and, if necessary, individual programs. Such a mechanism is needed to quickly identify schedule delays, enact timely corrective measures, and if necessary, trigger contingency plans. Finally, in not having a single, enterprisewide inventory, the Department will not be able to readily identify areas that may need additional resources, such as testing facilities.

Conclusions

The concerns we raised above demonstrate that if immediate attention is not given to ensuring that DIST is reliable, complete, and accurate, the Department’s Year 2000 efforts will be at risk of failing. In addition, without a good enterprisewide system inventory, Defense will not be in a position to make the trade-off decisions necessary to ensure that information technology projects are being implemented at acceptable costs, within reasonable and expected time frames, and are contributing to tangible, observable improvements in mission process. Given the fact that Defense has a major effort ongoing to improve its information systems, and that the Year 2000 problem will likely call on the Department to divert resources from other information technology-related initiatives, decisive action is needed to provide the resources and schedule priorities needed to accomplish DIST improvements, and to ensure that the currency and accuracy of DIST information is maintained in the future.

Recommendations

In order to ensure that DIST can be effectively used for Year 2000 efforts, we recommend that you direct your staff assigned to oversee implementation of the DOD Year 2000 Management Plan and the Director of the Defense Information Systems Agency to ensure that all duplicate, inactive, and incomplete entries be identified and expedite development and implementation of the purging methodology, expand Year 2000 information included in DIST for individual systems to include key program activity schedules that managers of interfacing systems need to ensure that their system interfaces are maintained during the renovation phase. This expansion should also include information that will enable the Office of the ASD/C3I, component, and organizational-level Year 2000 program officials to quickly identify schedule delays, promptly correct them, and if necessary, trigger contingency plans.

After the new criteria for reporting information systems are applied by system managers, we recommend that your staff, and the Director of DISA, in conjunction with the services and components, act to ensure that the DIST database is kept up-to-date and accurate, identify instances of noncompliance so that responsible command organizations can take corrective actions, and move forward with any other initiatives needed to make DIST an effective management tool.

Agency Comments and Our Evaluation

The Department of Defense provided written comments on a draft of this report. These comments are summarized below and reprinted in appendix I. The Assistant Secretary of Defense for Command, Control, Communications and Intelligence concurred with our recommendations. In concurring with our recommendations Defense stated that it planned to perform statistical sampling of DIST data to validate accuracy, and that it would rely on the DOD Inspector General to validate DIST data accuracy during its Year 2000 audits. It stated that the services and components were responsible for entering their automated information systems into DIST or be at risk of losing funding for their systems. Also, DISA has instituted a data quality program for DIST which includes purging of duplicative and obsolete data and will assist users in completing systems entries as necessary.

These actions will help to enable DIST to become an effective tool for both DOD management oversight and for the components day-to-day management of the department’s Year 2000 system correction efforts and beyond. However, in order to ensure complete validation of DIST, we believe that the Office of the ASD/C3I and DISA need to supplement these actions with efforts that involve fully comparing service inventories (and command inventories in the case of the Navy) to DIST and reconciling differences identified. Further, these offices must play a more active role in ensuring that data fields necessary to track Year 2000 progress are included in DIST upgrades and that this information is also reconciled with the services and components specific Year 2000 project status databases.

We appreciate the courtesy and cooperation extended to our audit team by your representatives and DISA officials and staff. Within 60 days of the date of this letter, we would appreciate receiving a written statement on actions taken to address these recommendations. We are providing copies of this letter to the Chairman and Ranking Minority Member of the Senate Committee on Governmental Affairs; the Chairmen and Ranking Minority Members of the Subcommittee on Oversight of Government Management, Restructuring and the District of Columbia, Senate Committee on Governmental Affairs, and the Subcommittee on Government Management, Information and Technology, House Committee on Government Reform and Oversight; the Honorable Thomas M. Davis, III, House of Representatives; the Secretary of Defense; the Deputy Secretary of Defense; the Acting Under Secretary of Defense (Comptroller); the Director of the Defense Information Systems Agency; and the Director of the Office of Management and Budget. If you have any questions on matters discussed in this letter, please call me at (202) 512-6240 or Carl M. Urie, Assistant Director, at (202) 512-6231.

Comments From the Department of Defense

Major Contributors to This Report

Accounting and Information Management Division, Washington, D.C.

Atlanta Regional Office

Chicago/Dayton Field Office

Kansas City Regional Office

George L. Jones, Senior Information Systems Analyst David R. Solenberger, Senior Evaluator The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.