The Department budgets billions of dollars each year to purchase and repair spare parts and has established various programs to help ensure product quality throughout the acquisition and repair processes. According to Navy and Defense Logistics Agency quality assurance officials, although it is normal in manufacturing processes that some deficient parts will be delivered to the end users, the rate of deficiencies is to be managed and controlled to acceptable levels. In fact, according to these officials, spare parts quality is of greater importance in today's environment of increased deployments, extended preventive maintenance cycles, and just-in-time deliveries of parts. A Defense Logistics Agency joint-service directive for quality deficiency reporting regulates Department activities. The directive establishes a Department-wide system for the military services and other activities to report, investigate, and correct the causes of individual problems and identify trends and recurring deficiencies. The system is to be used to document the quality of spare parts delivered for use in the maintenance and repair processes and is the key program for documenting quality deficiency data Navy-wide. When deficient parts are delivered and are detected by end users, such as military supply or maintenance personnel, these users are required to report the details of the deficiencies under a uniform set of guidelines. Each military service and agency manages its own quality deficiency reporting program. The Navy's Product Quality Deficiency Reporting Program is designed to document and report unsatisfactory material, initiate corrective action to fix or replace deficient items, grant credit or compensation for items, and take actions to prevent recurrence. The program applies to all Navy activities. Navy activities report quality deficiency data both internally and to a broader program, known as the Product Data Reporting and Evaluation Program, where it is managed as one of several databases. This broader program is a Navy-wide automated system for collecting data on the quality of material and products furnished by contractors. The quality deficiency report is also one of several records used in the Navy's Red/Yellow/Green Program, another Product Data Reporting and Evaluation Program application used to reduce the risk of receiving nonconforming products and late shipments. The Assistant Secretary of the Navy for Research, Development, and Acquisition has program authority and sets policy, and the Naval Sea Systems Command administers the Product Quality Deficiency Reporting Program. Navy program executive officers, program managers, and commanders of the naval systems commands are to process quality deficiency reports for their systems and equipment in accordance with instructions and to ensure people are properly trained in reporting deficiencies. Quality deficiency reporting data are generally sent to a screening point, such as the Navy's Inventory Control Point, for review and forwarding to an appropriate item manager or action point to determine the causes and who is responsible for the deficiencies. Once a determination is made and final disposition occurs, the quality deficiency reporting database is to be updated to reflect the results. The Navy's Product Quality Deficiency Reporting Program has been largely ineffective in gathering the data needed for analyses so that Navy managers can accurately report on and correct deficiencies in the quality of spare parts being provided to end users, including maintenance personnel in field activities. Specifically, the program data were incomplete and of limited value because they were underreported, did not include information on parts that failed prematurely, and omitted key data on the causes of failures. To a large extent, the program's ineffectiveness can be attributed to limited training and incentives to report deficiencies, lack of management emphasis, and competing priorities for the staff resources needed to carry out the program. In addition, a contributing cause of the program's ineffectiveness may be a lack of Navy-wide visibility into program results. Many deficiencies in the quality of new and rebuilt parts occur that have not been reported to the program. As a result, program data are unreliable and can be misleading in determining the significance of deficient parts and conducting trend analyses. Using information gathered under the program, the Navy produces an annual report summarizing quality deficiency data, such as the number of deficiency reports, the value of deficient material, and related information on new and reworked material, including rebuilt parts. Data system managers said this report is the most comprehensive attempt to collect and analyze Navy-wide quality deficiency data. The May 2000 report showed that during calendar years 1997-99 the number of deficiency reports averaged about 6,500 per year, representing all parts that were reported as deficient at the time of installation by the Naval Sea Systems Command, the Naval Air Systems Command, and the Naval Supply Systems Command. Each report identified one or more deficient parts to be evaluated for possible quality problems. During our visits to a number of ship and aircraft maintenance and operating units, we were told that not all deficiencies on parts that failed at the time of installation were reported, and we found that the estimated extent of underreporting varied. For example, one ship maintenance unit that handled munitions systems reported most deficiencies. However, an aircraft squadron commander said his unit documented quality deficiency problems and knew of the reporting requirements, but the unit rarely reported these problems to the quality deficiency program because there was no incentive to do so. In addition, Navy supervisor of shipbuilding officials at one facility told us that the full-time position dedicated to quality deficiency reporting had been eliminated and the work was reassigned to another staff member as a collateral duty. As a result, the number of quality deficiency reports dropped from about 200 each year in calendar years 1997 and 1998 to 34 in calendar year 2000. Also, Navy officials said several major Navy program offices have their own internal systems for handling parts quality deficiencies and do not share this information with the Navy's quality deficiency reporting program. While these examples are not intended to reflect the experience of all Navy units, they indicate how the data gathered by the program can vary among units based on factors other than the actual numbers of quality deficiencies encountered by maintenance personnel. Various unit commanders indicated that their efforts to report spare parts quality deficiencies were limited because accurate and complete data needed for the reporting process were not available on the failed parts and increased resources would be needed to make meaningful improvements in the availability of these data. Due to the underreporting, the interpretation of program data can be misleading. For example, in a March 2001 two-page report to the Congress on how the Department planned to improve its quality assurance program, the Principal Deputy Under Secretary of Defense, Acquisition, Technology, and Logistics stated that the Department used product quality deficiency reports as a key method to measure the quality of the products it purchased. The Department stated that it had no evidence of a systemic quality problem and that a decreasing number of investigated quality deficiency reports recorded in recent years may indicate that product quality is getting better. However, after we discussed our findings with the Department about the underreporting of quality deficiencies in the Navy, the Department stated in its report that if user feedback is not input into the system and investigated, due to reduced staffing, this metric fails to be a valid measure. According to Navy and Defense Logistics Agency quality assurance officials, the trend in the number of deficiencies reported may reflect the level of resources dedicated to filling out and analyzing deficiency reports rather than the actual number of deficiencies found by maintenance personnel. The Navy does not attempt to use the Product Quality Deficiency Reporting Program to collect a major category of quality deficiencies-- those involving parts that failed prematurely, that is, after some operation but before the end of their expected design life. Premature failure data are important to determine if there are problems with particular parts or suppliers of parts. Consequently, not knowing the full extent of deficiencies can prevent meaningful analyses of quality problems on a systemwide basis. Under the May 1997 joint-service directive on quality deficiency reporting, the Navy could capture data on premature equipment failures, including parts, as part of its program, as the Army, the Air Force, and the Marine Corps agreed to do. Although the joint-service directive states that it is applicable to a broad range of deficiencies, including premature failures, the Navy did not agree to this application. It has instead used deficiency reporting to identify mainly new or newly reworked parts that fail, as specified in a 1993 version of the joint-service directive. Navy quality assurance officials said the program was limited to capturing data on those parts because some type of warranty might be involved or they might obtain replacement parts or reimbursements from the suppliers. However, after we discussed this issue with Assistant Secretary of the Navy, Research, Development, and Acquisition officials, they said that premature failures should be included in the Navy program. They said that in April 2001 they approved the Navy's use of a new draft version of the joint-service directive that includes the requirement to report premature failures as quality deficiencies. While the Navy has not captured data on premature parts failures under the Product Quality Deficiency Reporting Program, Naval Air Systems Command officials said they have attempted to identify these deficiencies through their Engineering Investigation Request Program. However, they said that resource shortages have limited this program mainly to the analysis of mission-critical and safety-of-life requests for engineering investigations, and that as a result, many premature parts failure requests were not analyzed. Without validation and analysis of the causes of these failures, managers are not in a position to take corrective or preventive action. Furthermore, Naval Air Systems Command-wide data on engineering investigation requests were not available, and reports have not been combined with quality deficiency reports to provide managers with a systemwide view of spare parts problems. According to Navy and Defense Logistics Agency quality assurance officials, premature parts failures are an important aspect of quality, and without this data the Navy is not in a position to completely identify its problems with parts and suppliers. Premature parts failures could indicate problems in design, manufacture, or installation that, if not corrected, could lead to unanticipated parts shortages and increased costs. Quality deficiency reporting data that were collected under the program have been of limited value because they frequently lacked key information on the causes of the parts failures and identification of whom was responsible. Navy and Defense Logistics Agency quality assurance officials said that identifying whether a contractor is responsible for deficiencies is important for obtaining credits or refunds and preventing recurring problems. We reviewed the Navy's systemwide database of quality deficiencies for the Naval Air, the Naval Sea, and the Naval Supply Systems Commands. We reviewed the number of reports submitted during calendar years 1997-99 (19,124 reports) and determined their status as of September 20, 2000. We found that in most cases the causes of problems had not been identified in the database and responsibility for the deficiencies had not been determined. Specifically, we found that about 72 percent (13,675) of the reports did not identify the specific causes for the failures, which is information needed for effective corrective and preventive actions, and about 70 percent (13,287) of the reports did not identify who--a private contractor or a Navy or other government activity--was responsible for the deficiencies. According to Navy and Defense Logistics Agency quality assurance officials, analyzing product quality deficiencies to determine causes and responsibilities for failures is often difficult, time-consuming, and staff intensive. They said that the information needed to fill out the deficiency form is not always available, the deficient part might have been damaged or misplaced, or the part may not be forwarded for analysis to determine the cause of the deficiency. Also, a supplier might have provided a replacement part and not identified the cause of the problem. They said problems were more likely among deployed units. Officials at units we visited identified similar problems. According to an Assistant Secretary of the Navy, Research, Development and Acquisition official, when the Navy screening points receive a quality deficiency report, they work with the originators to provide the missing descriptive data and decide if an in-depth analysis is needed. If an analysis is needed, other problems may arise such as a lack of access to contractor information, which can prevent a complete analysis of the cause of a quality deficiency. The need for completed quality deficiency reporting to identify cause and responsibility for the deficiencies is important to achieving program results and is especially important for the Navy and the Defense Logistics Agency to obtain some type of credit for deficient parts, where allowed by the parts contract. Credit can involve a contractor providing replacement parts or monetary consideration. Also, determining responsibility is the first step in providing feedback to the supplier that deficient material has been provided, manufacturing or repair processes must be reviewed, and corrective actions may be necessary. Without such a determination, neither credit nor quality parts may be obtained. According to Navy quality assurance officials, this information is also needed for other Navy programs. For example, the Navy's Red/Yellow/Green Program is designed to help reduce the risk of receiving products that do not conform to requirements and late shipments. This program uses quality deficiency reports as one of the key inputs to the evaluation, but only when such reports show that a specific contractor was found to be responsible for the deficiencies in parts. However, since the quality deficiency data we examined were so often incomplete, many of the reports lacked the needed information. According to Navy and Defense Logistics Agency quality assurance officials, the lack of complete and reliable quality deficiency data limits their usefulness in identifying suppliers of deficient parts. In addition to the problems of incomplete and underreported data, concerns about using the data to determine the extent of spare parts quality deficiencies are twofold. First, to be meaningful, the number of quality deficiencies identified should be compared with the total number of parts used, but the data to compute such a deficiency rate are often not available. Second, even if a deficiency rate could be determined, criteria for determining what constitutes a reasonable deficiency rate for a particular part have not been established for the quality deficiency reporting program as it has been, for example, for the Red/Yellow/Green Program. Navy officials at many levels--maintenance personnel, program officials, quality assurance managers, and Command-level staff--indicated that program ineffectiveness is to some extent the result of a lack of action by Navy management to emphasize the importance of the program to the Navy. In addition, Navy officials identified the following causes of program ineffectiveness: Limited training to supply, maintenance, and key command personnel explaining reporting procedures, types of quality deficiencies to be reported, and benefits to the Navy to be derived from the program. Limited incentives and competing priorities for available staff resources to fill out the quality deficiency reports and do all their other work as well. Lack of Navy-wide visibility into the results being derived from the program. According to Navy quality assurance officials, some training that covers filling out the standardized quality deficiency reporting form has been given, but it has not been effective in encouraging additional quality deficiency reporting. Navy and Defense Logistics Agency quality assurance officials said that their sailors and their commanders do not receive training on the importance of the quality deficiency reporting program to the Navy. Without such training, reporting quality deficiencies does not become a priority within units. Officials within the Naval Systems Commands told us that compliance with quality deficiency reporting has diminished because fewer resources have been available to carry out tasks that are deemed more essential. They indicated that if maintenance personnel must choose between repairing needed equipment and filling out quality deficiency reports, they choose the repair work to support their primary mission. Limited incentives exist at the unit level to encourage compliance with program reporting requirements because financial credits for deficient parts are often not returned to the reporting units, so unit commands may not see the credits as an incentive to expend resources for quality deficiency reporting. They also said that new automation and decision support tools might help facilitate the deficiency reporting and analysis processes and therefore improve compliance with reporting requirements. Navy and Defense Logistics Agency quality assurance officials said that clearly reported program results could stimulate greater management emphasis and staff support for the program. However, while the program is designed to provide a basis for reporting, correcting, and preventing spare parts quality deficiencies, the results are not always measured or clearly reported. For example, a Navy-wide annual report shows that during the 3-year period 1997-99, quality deficiency reports submitted from the Naval Air, Sea, and Supply Systems Commands identified about $466 million in rejected material, mostly on aircraft parts. The meaning of these data, however, is unclear because information is not available on what portion of the rejected material was investigated, was found deficient, and yielded some type of credit or reimbursement from contractors to the government. Also, the data did not include any reporting of parts design modifications, manufacturing changes, or other actions taken based on quality deficiency reporting in order to prevent recurrence of problems. Without such reporting of results, program weaknesses such as incomplete data may not appear to be important issues to Navy and Department managers. According to Navy quality assurance officials, although one Navy instruction governs the analysis of quality deficiency reports, the level of execution can vary significantly among units, depending on the program emphasis assigned and resources made available. For example, Navy Inventory Control Point officials who manage ships' quality deficiency reporting told us they track the quality deficiencies and identify costs avoided and recovered from contractors. Their internal metrics for fiscal years 1998 through 2000 identified several million dollars in replacement items, refunds, and credits received from contractors that they said were directly attributable to quality deficiency reports. They said that cost avoidance is an important measure of results and should be tracked and reported Navy-wide along with other metrics. However, they said that such reporting is not required and the extent of reporting varies significantly among the Naval Air, Sea, and Supply Systems Commands and their subordinate units. Without an effective Navy-wide program to document feedback from users of parts on the quality deficiencies they encounter, Navy managers lack the data needed to fully assess the extent and seriousness of spare parts quality problems. Furthermore, to the extent that the data are incomplete or are not analyzed to determine the causes of and accountability for deficiencies, managers cannot effectively correct quality problems, address supplier issues, or ensure high quality when buying or rebuilding spare parts. Such activities are important because they can affect the safety, readiness, mission performance capabilities, and support costs of military forces. The Navy's Product Quality Deficiency Reporting Program has been largely ineffective in meeting these management needs due to weaknesses in program implementation, including insufficient training, limited incentives and automation support, competing priorities for staff resources, and a lack of Navy-wide measurement of program results. A stronger quality deficiency reporting program would better enable management to take corrective and preventive actions that over time can result in both mission performance improvements and cost reductions. Given the importance of high quality spare parts to safety, readiness, mission performance, and support costs, and the role that an effective Product Quality Deficiency Reporting Program can play in helping to ensure high quality, we recommend that the Secretary of Defense direct the Secretary of the Navy to increase the program's levels of (1) training, describing what quality deficiencies to report, how to report them, and why it is important to the Navy; (2) incentives, including financial credits back to the reporting unit where appropriate to encourage participation; (3) automation support, to simplify and streamline reporting and analysis; and (4) management emphasis provided to the program, as necessary, to determine the causes, trends, and responsibilities for parts failures and achieve greater compliance with joint-service requirements, including reporting on parts that fail before the end of their design life, and require program officials to measure and periodically report to the appropriate Defense and Navy managers the results of the program in such areas as actions taken to correct parts quality deficiencies, prevent recurrences, and obtain credits or reimbursements from suppliers for deficient products. In written comments on a draft of this report, Department of Defense concurred with our recommendations. It stated that the Navy would initiate the recommended enhancements to the Navy Product Quality Deficiency Reporting Program no later than September 15, 2001. In addition, the Navy will provide a status update to the Office of the Secretary of Defense on accomplishments and remaining challenges no later than March 15, 2002. The Department's comments are presented in their entirety in appendix I. To determine whether the Navy's Product Quality Deficiency Reporting Program has been effective in gathering the data needed for analyses, correction, and prevention of deficiencies in spare parts, we analyzed the completeness of the quality deficiency data contained in the Navy's Product Deficiency Reporting and Evaluation Program database. These data cover aircraft, ships, and other spare parts in the Navy, including parts managed for the Navy by the Defense Logistics Agency. For comparison purposes, we also analyzed data from the Navy's Maintenance and Material Management System, which is designed to track maintenance support data on how often a part failure is involved. We discussed the causes of underreporting of quality deficiencies, the lack of reporting of premature failures, the omission of data in deficiencies being reported, and the causes of program weaknesses with Department, Navy, and Defense Logistics Agency officials. Specifically, work was conducted at headquarters offices, including the Office of the Under Secretary of Defense (Acquisition, Technology, and Logistics); the Office of the Assistant Secretary of the Navy (Research, Development, and Acquisition); and the Naval Air Systems, the Naval Sea Systems, and the Naval Supply Systems Commands. Work was also conducted at the Naval Inventory Control Point activities in Mechanicsburg and Philadelphia, Pennsylvania, who have assumed responsibility for quality deficiency reporting functions. In addition, work was done at Navy field activities in different geographical areas, including Lemoore and San Diego, California; Norfolk, Virginia; and Portsmouth, New Hampshire. The Marine Corps aircraft quality deficiency reporting is managed by the Naval Air Systems Command and was included in our review. The Marine Corps operates a separate program for ground equipment that was not included in this review. We conducted our work between August 2000 and June 2001 in accordance with generally accepted government auditing standards. We are sending copies of this report to the Secretaries of Defense and the Navy; the Commandant of the Marine Corps; the Director, Defense Logistics Agency; the Director, Office of Management and Budget; and appropriate congressional committees. We will also make copies available to others upon request. Please contact me at (202) 512-8412 if you or your staff have any questions regarding this report. Key contributors to this report were Allan Roberts; Lionel Cooper; Gary Kunkle; Jean Orland; Lawson Gist, Jr.; Robert B. Brown; and Nancy Ragsdale. Army Inventory: Parts Shortages Are Impacting Operations and Maintenance Effectiveness (GAO-01-772, July 31, 2001). Navy Inventory: Parts Shortages Are Impacting Operations and Maintenance Effectiveness (GAO-01-771, July 31, 2001). Air Force Inventory: Parts Shortages Are Impacting Operations and Maintenance Effectiveness (GAO-01-587, June 27, 2001). Major Management Challenges and Program Risks: Department of Defense (GAO-01-244, Jan. 2001). High-Risk Series: An Update (GAO-01-263, Jan. 2001). Defense Inventory: Improved Management Framework Needed to Guide Navy Best Practice Initiatives (GAO/NSIAD-00-1, Oct. 21, 1999).

Summary:
The Department of Defense (DOD) budgets billions of dollars each year to purchase and repair the spare parts needed to maintain its weapons systems and support equipment. The quality of the spare parts can greatly determine if the Department's investment of funds is effective, efficient, and economical. This report examines the Navy's Product Quality Deficiency Reporting Program and the extent to which the program has gathered the data needed for the analysis, correction, and prevention of deficiencies in spare parts. GAO found that data on parts defects identified at the time of installation were underreported. Data on parts that failed after some operation but before their expected design life were not collected as part of this program. In the quality reports GAO reviewed, some key information was omitted on the cause of the parts' failures and some reports did not identify who was responsible for the defects. To a large extent, the program's ineffectiveness can be attributed to lack of management, limited training and incentives to report deficiencies, and competing priorities for the staff resources needed to carry out the program.