The military services have a long history of interoperability problems during joint operations. For example, the success of the Persian Gulf war in 1991—a major joint military operation—was hampered by a lack of basic interoperability. The current certification requirement was established to help address these problems. The Joint Staff’s Director for C4 systems (J-6) is assigned primary responsibility for ensuring compliance with the certification requirement. DISA’s Joint Interoperability Test Command is the sole certifier of C4I systems. According to Joint Staff guidance, commanders in chief, the services, and DOD agencies are required to adequately budget for certification testing. They can either administer their own tests with Test Command oversight or ask the Test Command to administer them. Certification is intended to help provide the warfighter with C4I systems that are interoperable and to enable forces to exchange information effectively during a joint mission. Specifically, certification by the Test Command is confirmation that (1) a C4I system has undergone appropriate testing, (2) the applicable requirements for interoperability have been met, and (3) the system is ready for joint use. However, while a system may pass certification testing, it may not have been tested against all systems with which it may eventually interoperate. This is because some systems with which they must interoperate become available later and commanders sometimes use systems in new ways that were not envisioned during testing. DOD guidance requires that a system be tested and certified before approval to produce and field it. Depending on the acquisition category and dollar threshold of the program, the approval authority may be the Under Secretary of Defense (Acquisition and Technology), with advice from the Defense Acquisition Board; the Assistant Secretary of Defense (Command, Control, Communications, and Intelligence), with advice from the Major Automated Information System Review Council; or the DOD component head (such as the commander in chief of a unified combatant command, the head of a military service, or a DOD agency head). A DOD Directive established the Military Communications Electronics Board to provide guidance on interoperability issues referred to it by the Secretary of Defense and the Chairman of the Joint Chiefs of Staff. The Board addresses interoperability issues through two subpanels: (1) The Interoperability Improvement Panel monitors C4I interoperability issues surfaced by the commanders in chiefs, military services, and DOD agencies and (2) The Interoperability Test Panel resolves testing disputes (such as appeals of Test Command certification decisions made by commanders in chief, military services, and DOD agencies). The Test Panel may waive the certification requirement to support developmental efforts, demonstrations, exercises, or normal operations. The waiver is not intended to be permanent, and is typically granted for 1 year. Commanders in chief, services, and DOD agencies are generally not complying with the certification requirement. As a result, we found instances in which existing, newly fielded, and modified systems are not certified for interoperability. Test Command analysis showed that a significant number of existing C4I systems had not been submitted for certification as required. According to Test Command officials, as of December 1997, the DOD Defense Integration Support Tool database of C4 systems listed about 1,000 systems that may exchange information with another system. In addition, there are about 1,176 unclassified intelligence systems, according to the Office of the Assistant Secretary of Defense, C3I. Test Command officials said they did not know precisely how many of these systems require certification. Nor did the Office of the Assistant Secretary of Defense know which intelligence systems would require certification because they were unable to determine which of these systems were outdated (i.e., legacy systems), stand alone systems, or one-service-only systems. While the Test Command has generally certified increasingly more systems during the past 4 years, officials acknowledged that “they have not even begun to scratch the surface” of the universe of systems that may require testing and certification. During fiscal years 1994 through 1997, the Test Command certified 149 C4I systems. According to Test Command officials, DOD’s Defense Integration Support Tool database attempts to list all C4 systems and other mission critical systems, but it does not contain all C4 systems or indicate whether the systems have been certified. According to DISA documentation, the purpose of the Defense Integration Support Tool is to support a DOD-wide information management requirement for data collection, reporting, and decision support in areas such as planning and interoperability. After discussions with DOD officials regarding this issue, DOD has recently included certification status as part of the database and, as of January 1998, 44 systems reflected this information. We recently reported in two separate reports that the Defense Integration Support Tool database is incomplete and inaccurate. In response to our October 1997 report, DOD acknowledged that this database is its official automated repository and backbone management tool for DOD’s inventory of systems. Accordingly, DOD said that it had begun to take major actions to enhance the database by instituting a validation and data quality program to ensure that the database contains accurate and complete data. DOD further stated that it would closely monitor this program to ensure that the data quality is at the highest level as required for reports to senior Defense managers and the Congress. Since this database is an important management tool, it is essential that it be complete and accurate. In several instances, new systems have been fielded without consideration of the certification requirement. Two recently fielded Air Force systems—a weather prediction system and a radar system—were not tested for certification by the Test Command, despite June 1996 memorandums from the Joint Staff stating that the service must plan for testing to ensure compliance with interoperability guidelines. Further, since 1994, the Assistant Secretary of Defense (Command, Control, Communications, and Intelligence) has approved three of nine major automated information systems for production and fielding that had not been certified for interoperability. For example, the recently fielded Defense Message System was not certified by the Test Command. Test Command officials stated that the system has undergone some interoperability testing but, because of shortfalls, was not certified. A decision was made to field the system while the shortfalls are resolved. Test Command officials believe the system will eventually be certified. No newly developed systems purchased through the Command and Control Initiatives Program were tested by the Test Command. (This program allows commanders in chief to purchase low-cost improvements to their command and control systems.) According to DISA officials, DISA had assessed these systems’ interoperability requirements and reminded the users to submit the systems for testing. In addition, during the last 3 years, no systems purchased through the Advanced Concept Technology Demonstrators program were tested and certified. (This program allows a new capability to be quickly developed, purchased, and exercised in the field before an acquisition commitment is made.) According to Test Command officials, previously certified systems that were later modified are not consistently submitted for recertification as required. Although Test Command officials do not know the exact number of modified systems that require recertification, they are aware of several systems—such as the Navy’s AEGIS shipboard weapon system and the Air Force’s Airborne Warning and Control System. Joint Staff officials believe that, although the certification requirement is outlined in several DOD and Joint Staff guidance documents, some system managers are unaware of it. In a study chartered by J-6 and completed in January 1996, only 12 of 424 (less than 3 percent) surveyed acquisition managers and Defense System Management College students knew about the DOD and Joint Staff interoperability requirements. The study team found that this lack of knowledge prevented users from placing interoperability in the initial requirements documents and acquisition managers from building interoperability into approved programs. As a result, the Joint Staff began an effort in 1996 to better educate system managers about the requirement. However, the study points out that education is not a panacea for all interoperability problems. Our analysis showed that some DOD organizations, although aware of the requirement, did not submit fielded systems for testing. For example, some program managers did not submit their modified systems for certification because they believed their design, although fielded, was not mature enough for testing. The program managers did not seek a waiver for their systems and ignored the certification requirement. Test Command officials told us that they lack the authority to compel program managers to bring their systems in for testing and must rely on the managers’ cooperation. In addition, in fiscal year 1995, only three intelligence systems were certified by the Test Command. Because Test Command officials believed that DOD’s intelligence community was ignoring the certification requirement, in 1996 the Command negotiated an agreement with DOD’s Intelligence Information Systems Management Board (which has responsibility for a portion of intelligence systems) to facilitate better participation in the certification process. In fiscal year 1997, the number of intelligence systems tested and certified increased to 14. Test Command officials believe that the increase is a direct result of the agreement. Further, according to Test Command officials, DOD officials do not always budget the resources needed for interoperability testing as required by Joint Staff guidance. In certain cases, the services do not budget sufficient funds to cover secondary C4I systems that are used to test the primary C4I system for interoperability because the services cannot afford to pay for all the testing DOD policy requires. For example, the services are required to provide secondary systems for 10 tactical data link interoperability tests a year. In this case, however, according to a Test Command official, the Army budgets for only seven or eight tests a year. The services are responsible for acquiring systems that satisfy service-unique requirements, and this responsibility sometimes takes precedence over satisfying joint interoperability requirements.