Depot Maintenance: Management Attention Required to Further Improve Workload Allocation Data
Released: 2001-11-09T00:00:00

Letter

Background

OSD has issued guidance to the military departments for reporting public- private workload allocations required by 10 U.S.C. 2466. The guidance is consistent with 10 U.S.C. 2460, which defines depot maintenance and repair. The guidance requires the comprehensive reporting of all work associated with the overhaul, upgrade, or rebuilding of parts, assemblies, and subassemblies and the testing and reclamation of equipment, regardless of the source of funds or the location at which maintenance is performed. It also requires the reporting of software maintenance, interim contractor support, and contractor logistics support, to the extent work performed in these categories is depot maintenance.

In recent years, the Department of Defense has implemented acquisition and logistics policy initiatives that have shifted depot maintenance workloads from the public to the private sector. We recently reported that between 1987 and 2000, the public sector’s share of depot maintenance work declined by 6 percent, while the private sector’s share increased by 90 percent. As the military departments move closer to the 50-percent ceiling for private sector work, with the Air Force exceeding the ceiling, the accuracy of the collection and aggregation of 50-50 data becomes increasingly important. The data in the prior-years report are important because they provide the best indicators the military departments have of the current public-private sector allocations. While we have said that the future-year data provide a rough estimate, the data are the Department’s only predictor to indicate that management attention may be needed to avoid potential compliance problems.

Summary of 50-50 Data Reported by DOD in February and April 2001 for Prior and Future Years

Table 1 provides a consolidated summary of DOD’s two reports to the Congress on depot maintenance public and private sector workload allocations, dated February 1, 2001 (prior- years) and April 1, 2001 (future- years). The amounts shown are actual obligations incurred for depot maintenance work in fiscal years 1999 and 2000 and projected obligations for fiscal years 2001-2005 based on the defense budget and service funding baselines. The percentages show the relative allocations between the public and private sectors.

Military Departments Had Mixed Results Complying With 50-50 Prior-Years Requirement

The Department had mixed results complying with the 50-50 prior-years requirement for fiscal years 1999 and 2000. Based on our review, the Army met the requirement for both 1999 and 2000, while the Air Force met the requirement in 1999 but exceeded it in 2000, issuing a national security waiver as provided for in the statute. Although we identified some errors in the Army and Air Force prior-years data, improved guidance and data collection efforts and validation and reviews by their audit agencies indicate that the Army and Air Force processes provide a sufficient basis for evaluating compliance. Further, in the Army’s case, the errors were not material in the context of the 50-50 requirement because they would not be sufficient to cause it to exceed the limitation. With respect to the Air Force, the cumulative errors further support the need for the Air Force waiver. However, because of data reliability issues and concerns about management controls and data validation, there is insufficient support for us to determine the Navy’s compliance.

Army Met 50-50 Requirement in Fiscal Years 1999 and 2000

Based on our review, the Army met the 50-50 requirement in both 1999 and 2000, with about a 53 to 47 percent public-private sector split in both fiscal years 1999 and 2000. Our review of the Army’s prior-years 50-50 report, internal reviews by auditors, and improved guidance and direction of the data collection effort indicate that the Army process provides a sufficient basis for determining that it met the 50-50 requirement during this 2-year period. While we identified errors that would increase private sector costs, about 1 percent in fiscal year 2000 above that projected by the Army, this would not be material to the 50-percent requirement since the Army would still be about 3 percent below the ceiling. Table 2 shows our adjustments to the Army data to correct for the errors we identified and the resulting impacts on the public-private sector allocations.

The errors and their impacts are as follows: One reporting activity did not report about $24 million in government- furnished material costs in fiscal year 2000. According to OSD and Army guidance, the costs of government-furnished material supporting work performed by contractors are to be reported as private sector costs. Other Army activities we reviewed properly reported government-furnished material costs.

While we determined that the reliability of the Army data was enhanced by the work of the Army Audit Agency, the Army had not incorporated all the adjustments recommended by the Army Audit Agency before the 50-50 data were submitted to OSD and subsequently to the Congress. Auditors reviewed more than one-half of the reported dollars submitted to Army headquarters for fiscal year 2000, identifying errors in about 5 percent of the items. Army officials revised the 50-50 data to correct for about $70 million of the errors identified; but one activity—citing time constraints—did not make another $21.9 million in adjustments for errors before the Army submitted its report to OSD. Adjusting for these errors adds $20.4 million to the private sector total and $1.5 million to the public sector total.

We identified several other small errors that in total would add $5.8 million to the public sector in fiscal year 1999 and subtract $4.6 million from the private side in fiscal year 2000. These were attributable to officials using budgeted requirements instead of actual obligations, double counting, and other mistakes.

As previously noted, we recognize there are some systemic problems with DOD’s financial data. However, based on the work of the Army Audit Agency and our work, we believe the Army process provides a sufficient basis for determining that it met the 50-50 requirement during the 2-year prior-year period of fiscal years 1999 and 2000.

Air Force Met 50-50 Requirement in Fiscal Year 1999 but Exceeded It in 2000, Issuing a Waiver

Based on our review, the Air Force met the 50-50 requirement in fiscal year 1999, with a 54 to 46 percent public-private split. However, the 48 to 52 percent public-private sector split in fiscal year 2000 required that the Secretary of the Air Force issue a national security waiver and notify Congress. We identified some reporting weaknesses that increased the amount the Air Force exceeded the ceiling in fiscal year 2000 from the reported 1 percent to about 2 percent; and correcting for these weaknesses resulted in increasing the private sector share in both fiscal years 1999 and 2000. Consequently, the weaknesses were not material in the context of meeting the 50-50 requirement since correcting for these weaknesses did not cause the Air Force to exceed the limitation in fiscal year 1999 and it had already reported exceeding the limitation in fiscal year 2000.

While recognizing some problems in the data, the Air Force data review process added to the reliability and the credibility of the reported data and, together with our work, serves as a basis for determining compliance with the 50-50 requirement. The 50-50 data reported to Congress included adjustments made as a result of reviews by the Air Force Audit Agency, Air Force Materiel Command, and Air Force Headquarters. The auditors identified adjustments amounting to 4.3 percent of the total workload. This rate is higher than last year’s and follows several years of gradually declining adjustment rates. The higher rate was mainly attributed to rather large errors in one acquisition program, late posting of a cash transfer into the working capital fund, and high turnover in the staff assigned to collect data.

Table 3 identifies the changes to the reported Air Force report after adjusting to correct weaknesses we identified.

The reporting weaknesses and their impacts are as follows: As in past years, Air Force officials continue to adjust the 50-50 data for general and administrative expenses associated with managing depot maintenance contracts. These amounts are for overhead and salary costs incurred by government personnel charged with administering depot maintenance contracts funded through the working capital fund. Air Force headquarters and Materiel Command officials subtract these amounts ($57.9 million in fiscal year 2000) from the private sector costs—where they are accounted for within the working capital fund—and add them to the public sector costs for 50-50 purposes. Air Force officials told us that they believe these costs should be reported as part of the public sector since government employees incur them. Although this type of cost is not specifically addressed, OSD 50-50 guidance requires that the costs for all factors of production—labor, material, parts, indirect, and overhead—associated with a particular repair workload should be counted in the sector accomplishing the actual maintenance. For example, contract maintenance on depot plant equipment used by government employees to repair items should be counted as public sector costs because they are incurred by the government in producing the repair. Thus, consistent with our prior assessments, we continue to believe that it is appropriate to count the general and administrative costs associated with administering depot maintenance contracts as part of the private sector costs of doing business. Accordingly, in table 2 we reversed the Air Force adjustments to again report them as private sector costs.

We also identified unreported contractor logistics support costs totaling about $2.3 million in both fiscal years 1999 and 2000. The Joint Stars program office reported most of its depot maintenance dollars appropriately but did not report technical data support costs. Additionally, one office in the Special Operations Directorate did not report contract maintenance costs that should have been reported based on Air Force 50-50 guidance.

As previously noted, we recognize there are some systemic problems with DOD’s financial data, but the Air Force has made improvements in the 50-50 reporting data processes since the requirement to submit the report to Congress was imposed. Based on the past improvement efforts and the more recent work of the Air Force Audit Agency and us, we believe the Air Force provided a sufficient basis for determining that the Air Force met the 50-50 requirement during fiscal year 1999 and exceeded it in fiscal year 2000, but submitted a waiver as provided by statute.

50-50 Waiver Issued for Fiscal Year 2000

As authorized under section 2466, on January 11, 2000, the Air Force notified the Congress that the Secretary was waiving the applicability of the 50-percent limitation on private sector contracting for fiscal year 2000 for reasons of national security. The Air Force explanation for the waiver was based primarily on the need to use temporary contracts to support transitioning workloads from closing depots. However, we reported previously that the temporary contracts represented only a minor share of the Air Force contract workload. We noted that the more significant factors were previous Air Force actions, such as the increase of long-term depot maintenance contracts from $600 million in 1996 to $1.1 billion in 2000 and the transfer of about half of the workload from two closing depots to the private sector, had increased the private sector share from 36 percent in fiscal year 1991 to the 50-percent ceiling in fiscal year 2000. These actions left Air Force officials little flexibility to use emergency contracts without exceeding the ceiling for contract depot maintenance work. We also pointed out that while the Air Force projected that it would exercise management changes to remain within the 50-percent limitation in fiscal year 2000 and beyond, it was uncertain whether it would be successful in these efforts.

Process and Data Weaknesses Preclude Determination of Compliance in Navy

Because of weaknesses in the Navy’s reporting processes and data, we were unable to determine whether the Navy complied with the 50-percent requirement for the 1999 and 2000 prior-years report. The Navy’s report, as presented, indicates that the Navy complied with the requirement in both years, with a 57-43 percent mix in fiscal year 1999 and a 55-45 percent mix in 2000. However, our review of the Navy 50-50 data for fiscal years 1999 and 2000 identified concerns about management controls and data validation processes. Also, Navy leadership chose not to use the Naval Audit Service as a part of this year’s process, and the absence of this review added to our concern about the reliability of the Navy data. The specific process problems, which are discussed in more detail later in our overall analyses of the military services’ data reporting processes, include (1) a decentralized and tiered reporting process that consolidated 50-50 numbers into summary reports with little evidence that the data were checked and validated while passing through the reporting layers and (2) the inability to track and document the estimating methodologies. While we have identified these same issues in the past, Navy leadership has not placed enough emphasis on the 50-50 reporting process to make the improvements required to assure that the data reported provide a sufficient basis for developing the Navy’s 50-50 information in the prior- years report.

For example, the Naval Sea Systems Command, which reported about one- fifth of the total Navy 50-50 amount for this time period, had one coordinating official who received summary data from 37 reporting units. Some units had in turn rolled up data received from as many as 10 subactivities. Individuals generally accepted the data, rolled up totals, and transmitted them up the reporting chain without conducting in-depth critical checks of the data. In most cases, an audit trail to track individual subactivity 50-50 submissions that were subsequently rolled-up into a single program or project unit figure did not exist.

In addition to our concerns about the Navy’s process, we identified one major reporting inconsistency that would impact the public-private sector allocations as shown in table 4.

We determined that two activities within the Naval Sea Systems Command reported inactivation activities inconsistently. One project office reported $650 million in nuclear ship inactivation costs performed mainly at the public shipyards, but another project office did not report about $113.6 million in conventional ship inactivation costs performed mainly in the private sector ($81.8 million in fiscal year 1999 and $31.8 million in fiscal year 2000). Officials from the non-reporting office said they did not know these types of costs were to be reported for 50-50 purposes. It is uncertain how many other activities have similar non-reported costs. DOD’s financial management regulation includes inactivation activities as reportable depot maintenance workloads. The internal Navy guidance specified reporting nuclear ship inactivations but did not mention conventional ships.

In commenting on a draft of this report, Navy officials said that the relatively complex process for nuclear ship inactivations is considered to be equivalent to depot maintenance and repair, while the less-complex process for conventional ship inactivations is not generally considered equivalent to depot-level work. However, we believe that some portion of the conventional ship workload should be reported as depot maintenance, based on follow-up reviews with Navy program officials. OSD is developing additional guidance to clarify this reporting category for future 50-50 reports.

Future-Year Projections Are Not Reasonably Accurate or Useful

The projections of the Army, Air Force, and Navy in DOD’s future-years’ report for fiscal years 2001 through 2005 are not reasonably accurate estimates of the future allocations of public and private sector workloads. The services’ management placed much less emphasis on the future-years data and reports. The reported projections are based in part on incorrect data, questionable assumptions, and some inconsistencies with existing budgets and management plans. Further, our review identified errors, inconsistencies, and other shortcomings. As a result, DOD’s future-years report should be viewed with caution because it does not provide the best data available to DOD decisionmakers and congressional overseers, and the reported data is misleading with regard to how future workloads are likely to be allocated between the public and private sectors. To make the future year 50-50 report a useful management tool would require improved management oversight and direction.

Army Future-Year Projections Are Inaccurate and Do Not Portray Continuing Challenge

While the reported Army future years’ workload allocations show an increasing public sector share, after adjusting the reported numbers to correct for errors and omissions, the net effect is an increase in the projected private sector’s share for each year. In our last year’s report, we noted that the Army faced long-term challenges remaining within the contract ceiling. Army officials said that they had taken action to increase the public sector’s share, not only to deal with the contract workload ceiling but also to more cost-effectively use underutilized Army depot infrastructure. The reported Army numbers suggested that these actions were effective. However, we identified significant problems and omissions in the Army’s reporting of its future-years 50-50 data, the net effect of which significantly increases the projected private sector’s share each year. Whereas the Army reported that public sector workloads were projected to increase in the future years, in actuality, after correcting for errors and omissions, the data show that the Army is substantially closer to the private sector limitation and that the future expected public-private sector allocations are relatively constant. Table 5 summarizes adjustments we made to the reported future year Army numbers.

The errors and their impacts are as follows: One reporting activity made a series of transcription errors that reported thousands of dollars as millions of dollars. These errors overstated public sector workloads by a total of $683.5 million in fiscal years 2003 –2005.

This same activity did not fully report contractor costs for contractor logistics support, which resulted in understating private sector workloads by a total of about $227.1 million in fiscal years 2001–2005.

Two major commands did not report depot costs associated with the Army’s Integrated Sustainment Management program. Officials said they did not report these costs because the Army Materiel Command will eventually be responsible for the reporting. However, a Materiel Command official told us the command was not ready to assume this reporting and did not include the costs of these activities in the 50-50 report. These additional projected costs are estimated to add $24 million per year to the public sector reporting and $56 million per year for the private sector. As we reported in the past, the Army’s move to consolidate maintenance activities under the National Maintenance Program and to perform depot-level maintenance at field locations continues to pose reporting challenges and could result in an underreporting of both public and private sector costs.

As discussed in the section on the prior-years report, one Army reporting activity had not reported the costs of government-furnished material. This material, when supplied to a contractor for use in the maintenance process, should be counted as private sector work. Assuming annual material costs stay constant at the fiscal year 2000 level ($24 million), this activity underreported private sector costs by about $120 million over the 5-year reporting period.

Another activity double-counted $19.9 million dollars of its public sector workload and $36 million of its private sector workload for fiscal year 2004. Estimates for maintenance costs for 7 systems were erroneously entered twice for 2004.

There are some additional non-quantifiable factors that are expected to have major impacts on the Army’s future depot maintenance program and these factors further support our concern (1) with the reasonableness of the Army’s future year projections and (2) that the Army will be challenged in the future to manage its depot maintenance program within the 50-50 ceiling. First, quality of the future-years data is questionable because some funding priorities and plans have substantially changed since the budget figures supporting the 50-50 projections were prepared. Revised plans for extensive repairs of the Patriot Missile and the Apache Helicopter will likely alter depot workload projections and public-private sector percentage allocations.

A second factor of potentially greater impact involves depot maintenance requirements associated with the Army’s recapitalization program.Funding requirements and implementation strategies for the recapitalization program are not fully known at this time and continue to evolve. Consequently, these requirements and expected public-private sector allocations were not fully reflected in the 50-50 outyear projections. At the time of our review, Army records showed that it planned to spend about $15.5 billion on recapitalized systems between fiscal years 2002 and 2007. Army officials said, however, that additional funding of at least $7.6 billion is needed over this period but this has not yet been budgeted.Procedures for managing and coordinating the recapitalization program were finalized in April 2001, and detailed implementation strategies are currently being developed for each of the 21 weapon systems to be recapitalized. While it appears that a significant portion of the program expenditures will be considered depot maintenance-type work, the plans supporting the distribution of workloads between the public and private sectors have not yet been finalized.

Air Force’s Future-Year Projections Are Inaccurate and Problems Will Likely Lead to More Waivers

Although the reported data indicate that the Air Force would breach the 50-percent ceiling in fiscal year 2001 but not in fiscal years 2002- 2005, we identified significant problems and questionable assumptions in the Air Force’s reporting of its future year 50-50 data, which indicate that the future contract work is understated. After adjusting the reported numbers to correct the problems we could quantify, the net effect is an increase in the private sector share projected for each year. In addition, other significant factors cannot be quantified, but indicate further growth in the private sector is likely. Taken together, the Air Force will likely continue to exceed the 50-percent private sector limitation, leading to more waivers such as the second one recently issued for fiscal year 2001. Table 6 shows the reported data, our quantifiable adjustments, and the resulting impacts on allocations. The revised allocations show the Air Force just under the 50-percent ceiling for the future years. Significantly, the reported and revised amounts are both over the 48-percent target for private-sector allocation that Air Force officials established for management purposes to allow for some flexibility under the ceiling if estimates and circumstances change.

The reporting weaknesses, errors, and their impacts are as follows: As discussed in our analysis of the Air Force’s prior-year numbers, the Air Force’s adjustments for general and administrative expenses associated with contracted workload shifts more than $100 million annuallyadding about $50 million to the public sector and subtracting the same amount from the private sector. We think it more appropriate to count these expenses as part of the private sector amounts. They are overhead costs required to manage contract workloads; and the OSD reporting guidance says that the costs for all factors of productionlabor, material, parts, indirect, and overheadassociated with a particular repair workload should be counted in the sector accomplishing the actual maintenance. In table 6 we reverse the contract administration adjustment each year. This adds about $263.1 million to projected private sector work for the reporting period and decreases the public sector by the same amount.

The Air Force data do not fully reflect depot officials’ estimates of the continuing need for temporary contracts and contractor augmentees resulting from work transfers from base closures and contract competitions. At the time the future-years report was submitted, these would have added about $21.1 million to the private sector costs for the fiscal year 2001-2004 reporting period and decreased the public side by the same amount.

The Air Force data do not include the estimated repair costs of new systems and upgrades being reviewed for establishing the source of repair. Most of these items 48 of 66 and representing about 90 percent of the repair cost estimates for all the systems undergoing the revieware currently recommended for private sector repair.

Examples include the C-5 AMP repair, which is expected to add about $57 million in private sector repair costs and the KC-10 reverser fan modification, which is expected to add $41 million in private sector repair costs between fiscal years 2001 and 2005. Although some of these source-of-repair decisions could change, as they stand now, the net effect of the additional workloads would add about another $175 million to the private sector costs for fiscal years 2002 through 2005. In commenting on a draft of this report, Air Force officials pointed out that it is difficult to accurately project the outcome of the source of repair process and that the annual dollar projections are very rough budget estimates. However, all of the data projected in the future-years report are point-in-time estimates and subject to change. Incorporating estimated costs for systems in the source-of-repair process would provide a more comprehensive and useful projection of expected future public-private sector allocations.

We also identified underreported contractor logistics support costs and contractor costs to install modifications totaling about $8.2 million. Program officials at one activity were unaware of the 50-50 reporting requirement, while other offices reported some, but not all, of the depot-related contract costs. Officials in the Towed Decoy program office said they did not know that the costs for installing modifications should be reported as depot maintenance costs. The Joint Stars program office did not report technical data support costs and the Special Operations office did not report contract maintenance costs. As we have reported in the past, it is difficult to identify to what extent costs that should be reported are being reported. However, based on our review this year, we noted improvements in reporting these kinds of contract costs.

In addition to the problems we could quantify, there are some other significant factors which, while they cannot be quantified, are likely to have major impacts on future-year workloads as the Air Force moves closer to execution. Both public and private sector amounts can be affected. However, based on past experience, the Air Force’s practice of placing more depot repair work for new and upgraded systems in the private sector, and the unlikely event that the in-house depots will receive the projected amounts of work currently estimated, it is likely that the Air Force will continue to need to waive the 50-percent limitation in the future, absent corrective action.

One of the non-quantifiable factors is the relative accuracy of cost and budget data reported by the depot maintenance activity group, a part of the Air Force working capital fund. From 70 to 80 percent of the total depot maintenance amounts reported annually by the Air Force are financed through the depot maintenance activity group. Consequently, the relative accuracy of budget projections and accounting records for the activity group will significantly affect the quality and completeness of the Air Force’s 50-50 data. However, our review last year of the depot maintenance activity group identified poor budget estimates, inaccurate pricing, and overoptimistic assumptions about worker productivity and process improvement savings.

In addition, the rates charged for maintenance work and workload assumptions used in estimating fiscal years 2003-2005 requirements did not incorporate price increases for both public and private workloads, nor did they fully reflect expected surcharges and other workload changes. In recent years, operating losses in the depot maintenance activity group have necessitated large cash inflows and surcharges applied to maintenance rates to balance the accounts. For example, a reimbursement of $483 million ($417 million for the public sector and $66 million for the private sector) was spread over fiscal years 2001 and 2002 to make up for accumulated losses in the Depot Maintenance Activity Group operations for fiscal year 2000 and prior years. Performance indicators to date indicate that financial performance problems are continuing in fiscal year 2001 that could require changes in future rates and surcharges. Also, the apparent improvement in the 50-50 ratio for fiscal year 2002 shown in DOD’s report and table 1 in this report is due largely to a 17-percent rate increase on public workloads. However, it is questionable whether the projected public workload program can be funded at the higher rates. Depot officials noted that a price increase of that magnitude would likely result in a reduction in the amount of maintenance the operating commands would be able to fund. They said that budget estimates used to support the projections assumed that 100 percent of the anticipated workload would be accomplished. In reality, actual performance generally does not approach 100 percent, resulting in an overstatement of the reported 50-50 data. Officials at the three Air Force depots said that historically in the year of program execution, there are reductions in the depot maintenance program performed in the public depots and an increase in the amount of contracted workload over what had been projected. An official at one depot said that only about 94 percent of the current program would likely be accomplished in fiscal year 2001 due to operating inefficiencies, parts shortages, budget reductions, and other constraints. Air Force Materiel Command headquarters officials said this lower execution rate can affect both public and private workloads, but it generally has a greater impact in reducing the actual work accomplished in public depots. Changes in the amount of workload that is actually accomplished will have obvious impacts on the dollars and public-private sector allocations reported in the future.

Air Force Issued a Second Waiver for Fiscal Year 2001

The Air Force future-years report shows the Department exceeding the 50-percent ceiling in 2001. On July 31, 2001, the Air Force notified Congress that the Secretary of the Air Force had waived the 50-percent requirement for fiscal year 2001. The waiver determination was justified as necessary for national security because “the Air Force concluded that no significant workload could be moved into the public depots in the near term without increased cost and an adverse effect on readiness.” While we did not analyze the basis for the Air Force’s determination, we agree that those transitioning workloads, whether new or old ones, to a military depot would require increased funding.

The Air Force waiver determination also said that to address future compliance, the Air Force is preparing a long-term strategic plan that will address current capacity shortfalls, as well as new technologies and the associated infrastructure. This long-term depot strategy is supposed to be designed to ensure compliance with the 50-percent limitation.

However, the Air Force promised such a plan last year but has not been successful in developing it. After announcing on January 11, 2000, that the Air Force would exceed the 50-percent limitation last year, the Air Force told interested Congressional members and the Subcommittee on Readiness, Senate Committee on Armed Services, that it was developing a short- and long-term strategy for resolving the 50-50 dilemma. According to Air Force officials, they were unsuccessful in identifying workloads that could be moved into the Air Force depots in the short term and were equally unsuccessful in identifying workload to transfer into the depots in the long term. Program offices said they had entered into long-term contracts with private contractors and they had not budgeted for technical data or depot plant equipment and facilities that might be required before establishing an in-house capability in one of the three remaining Air Force depots. Further, DOD and Air Force acquisition strategy continues to express a preference for long-term contractor logistics support, to include maintenance, supply, and other logistics functions. This year’s waiver determination did not specify a date for the completion of the Air Force’s latest effort in developing a long-term depot strategy that would resolve the Air Force’s current 50-50 imbalance. As of November 2001, it has not been successful in implementing such a plan. Without an approved strategy for increasing the Air Force depots’ workloads and the funding of the resources required to establish new capability, the Air Force will not be able to resolve its 50-50 problem. The Air Force’s long-term depot strategy is now expected to be completed by the end of calendar year 2001, according to officials commenting on our draft report.

Navy and Marine Corps Future-Year Projections Are Inaccurate

We identified two major problems in reporting that, together with our reservations about the reliability of the Navy’s data, lead us to conclude that, like the other services, the Navy’s future year projections are not accurate. The resulting impacts on public-private sector allocations after adjusting for these problems are displayed in table 7.

The problems and their impacts on the allocations are as follows: As discussed in the earlier section on the prior-years 50-50 report, the Navy did not report inactivation costs for conventional ships as depot maintenance costs but reported similar work for nuclear ships. The Navy projects that up to $357 million over the 5-year period covered in the future-years report will be spent on conventional ship inactivation activities, mostly in the private sector. The Navy projections did include about $1.2 billion in nuclear inactivation workloads at public shipyards for this time period. As discussed earlier, the Navy believes that conventional ship inactivation workload is generally not equivalent in complexity to depot-level maintenance. A Navy official said they would use the additional clarifying guidance being developed by OSD in reporting future 50-50 workload allocations.

We also determined that the Navy did not include the costs of repairs on the USS Cole, the target of last year’s terrorist attack. The Congress late in calendar year 2000 appropriated $150 million in fiscal year 2001 funds for these repairs, which are to be accomplished at a private shipyard. Officials said the 2001 supplemental came after the 50-50 report was developed. Officials estimate that about another $93 million will be required to complete repairs on the USS Cole, which was also not reported in the 50-50 data. For display purposes, table 7 shows this additional amount in fiscal year 2002.

Several other issues affect the Navy’s report but are not quantified. For example, the Navy plans call for a substantial increase in submarine depot maintenance workloads associated with a major refueling program during this reporting period. Most of the work is expected—and was reported for 50-50 purposes—to be accomplished at the public shipyards. However, Naval Sea Systems Command officials told us that the plans and depot requirements are not yet firm and that extensive use of contract employees to augment the civilian workforce at the shipyards is anticipated. These contract requirements have not been fully identified and were not included in the 50-50 report. In commenting on a draft of this report, Navy officials stated that as soon as contract requirements for shipyard augmentation are determined, the amounts will be included in the 50-50 reports.

As we reported last year, the Navy is moving to a regional maintenance approach, which has made it difficult to identify and report depot-level work. Initially implemented at Pearl Harbor, this approach combines depot-level and lower levels of non-depot maintenance, changes funding sources, and consolidates financial systems. The Navy has not yet developed a system to discretely track and account for work meeting the definition of depot maintenance. In its absence, the officials used estimates to report 50-50 data. While this is a reasonable approach, actual data will likely cause future estimates to be revised if this program is implemented as planned throughout the Navy.

The Marines Corps projections for fiscal years 2001-2005 were based on a combination of budget formulation figures and straight-line projections. Headquarters Materiel Command Officials agreed that the reported data do not fully reflect the planned decrease in total revenues for this period, the impact of new systems going to the private sector for support, and the anticipated decrease in the public depot workforce. They did not provide an estimate to reflect these changes but said actions are underway to improve future reports.

In its report, we observed that the Navy and Marine Corps are now projecting a substantial shift to more private sector workload over the 50-50 reporting period compared to last year. Comparing the 5 years that the current 50-50 reports and last year’s reports have in common (fiscal years 2000-2004), the Navy is now projecting an additional $2.9 billion in private sector work and a decrease of $1.1 billion in public sector work. Whereas in last year’s reports they were projecting private sector allocations in the 35- to 42-percent range, this year’s reports project a significantly higher 43- to 46-percent range. Navy officials responsible for coordinating and reporting the 50-50 data attributed the increase in private sector amounts to (1) a sharp increase in the private sector wage rates, (2) some shifts in ship maintenance from the public to private sectors to make room for the extensive submarine refueling effort to be accomplished mainly in the public sector, (3) contracts with private shipyards, and (4) changes in cost models and estimating baselines.

Reporting Processes Improved, but Some Problems Remain

While DOD has greatly improved the 50-50 reporting guidance and implementation, opportunities for improvement still exist. We have noted improvements in the process each year, particularly with respect to the Air Force and Army’s use of internal auditors to review data, the Navy’s development of internal guidance, and OSD’s revisions to its reporting guidance in response to our recommendations. At the same time, some problems and concerns persist, including incomplete and inconsistent recordkeeping by the services and the Navy’s inadequate data validation. While the 50-50 process and resulting data will never be perfect, there are still opportunities for DOD to take actions to improve the validity of the process and the reliability of the data. If implemented, these improvements could improve the reasonableness of the 50-50 data as input to the management of the depot maintenance program to attain future compliance with the 50-50 requirement.

Army

For this year’s 50-50 data collection, Army officials added to the already extensive and detailed internal instructions used to supplement the OSD guidance. Army officials cited our report findings and their auditor findings to improve guidance in several areas, including warranties and contractor logistics support. Also, the Deputy Chief of Staff for Logistics held two workshops to prepare for the 50-50 data call and to address our prior-year findings, OSD’s reporting requirements, as well as the changes in the Army supplemental instructions. Command individuals responsible for responding to the data call and coordinating reporting efforts within their respective commands attended the workshops. Nonetheless, we noted that guidance and reporting requirements were not always clearly communicated or understood by potential reporting activities, resulting in some incomplete and missed reporting. The Army’s task in this regard is challenging in that 14 major commands need to be involved, along with numerous reporting levels within each command. Nonetheless, improved communication of the reporting guidance to activities that may not initially recognize that they have reportable maintenance activities should mitigate the problem of incomplete and missed reporting in the future.

The Army Audit Agency reviewed the data collection process at command levels as it was evolving, and problems they identified in the report for fiscal year 2000 were for the most part corrected before the activity reports were sent to Army headquarters. The error rate identified by the auditors was about one-half the rate found last year7.1 percent for fiscal year 2000 versus 15 percent in fiscal year 1999. For fiscal year 2000, Army auditors reviewed about $13 billion and identified adjustments of about $92 million. Auditors attributed the improvement to better guidance, the planning workshops, and overall management efforts. However, Army auditors put little emphasis on reviewing the future-year projections. This year’s review of the future year projections was concerned only with the process and how reporting organizations were determining projections. The Army auditors did not review or spot check the amounts projected for individual items or weapon systems, deciding that a detailed audit was not necessary since they had done a more thorough analysis of the future process and numbers last year without identifying significant errors. Nonetheless, we found significant errors that would likely have been identified by Army auditors if they had spot-checked projected amounts.

Air Force

The Air Force supplemented OSD guidance by adding details on contractor and interim contractor logistics support contracts, partnering, and software maintenance to its internal instructions. For the fourth consecutive year, the Air Force Audit Agency assisted Air Force headquarters and Materiel Command officials in verifying data and validating collection processes, significantly improving the quality and completeness of data before its submission to OSD and the Congress.

However, as in the other services, some Air Force offices did not maintain adequate documentation for reviewing and supporting data. Some offices we visited could not readily reconstruct estimating methodologies and provide source documents for their reported data. For example, F-117 contractor logistics support costs and F-15/F-16 trainer/simulator costs were omitted. Documentation requirements are not only valuable for audit purposes and management review but also as a method of maintaining historical records that can be used in subsequent years. This is especially important given the high turnover of staff performing these functions.

Navy

In response to our prior findings and those of the Naval Audit Service, Navy headquarters compiled and distributed a handbook with guidance to supplement the OSD reporting requirements. This handbook improved the Navy’s process as it included more detailed data collection procedures, a responsibility matrix, and a standard reporting format. Some commands also prepared additional instructions to reporting units. In addition, the headquarters official responsible for coordinating the Navy’s reports conducted on-site reviews at several commands and identified some errors that were corrected before the data was reported to OSD. However, the Navy did not hold a planning meeting to assemble the key staff involved in the 50-50 reporting process from the major commands to discuss and critically analyze procedural guidance. We have found such meetings to be useful in the Air Force and the Army in surfacing problems and concerns and helping to ensure a more consistent approach to data collection.

Furthermore, the Naval Audit Service was not asked to review processes and validate this year’s 50-50 data. Last year, auditors found that Navy guidance, data validation, and documentation lacked the detail to identify, collect, support, report, and document depot-level maintenance between public and private sectors. As a result, the Naval Audit Service concluded that the quality of the data reviewed was inadequate to determine the accuracy and completeness of the prior submission for fiscal year 1999. According to audit officials, by not doing a follow-on review, the accuracy and completeness of the Navy’s 50-50 data remain suspect.

We encountered similar problems during this review. The Navy’s decentralized and tiered reporting process rolls up data from numerous subactivities, consolidating 50-50 numbers into summary reports with little evidence that the data were checked and validated while passing through the reporting layers. In many cases, an audit trail did not exist which was sufficient to track and document the estimating methodologies and the data used to develop the individual subactivity 50-50 submissions that were subsequently rolled up into a single program or major activity amount. Although the Air Force and Army also have multi-layered reporting chains, we found their processes for collecting and verifying information to be generally better, especially their use of audit agencies to provide an effective third party review of the data collection process and to correct errors and validate data before submitting it to OSD and to the Congress.

Record-Keeping Weaknesses Continue to Be a Problem

Our review this year, as in the past, determined that each of the services could better maintain auditable records for documenting data collection methodology, estimating techniques, and final reported results. While some central records are maintained, information and reporting rationales at program offices and maintenance activities are sometimes lacking, and it is difficult for a third party to understand and reconstruct the methodology and verify results. For example, some Air Force reporting activities had not kept consolidated records to document their data collection procedures, estimating methodologies and assumptions, and data sources. In some other instances, key reporting staff had been transferred and, without adequate documentation, new staff could not readily explain nor replicate the results. Also, a Navy major command was in the process of realigning its units for budget purposes; and neither we nor officials responsible for the 50-50 effort could always determine which project unit actually provided the data, the type of depot maintenance being performed, and the class of ship involved. We also noted that the command had developed special budget codes to help identify and track depot funding, but that the codes were not extensively or consistently used. Good records, documentation of processes followed, and identification of data sources used are important not only for audit and management oversight but also for use as a historical record that can be followed by newly assigned staff to assist in data collection and by programs reporting for the first time.

Conclusions

Expanded guidance and the efforts of service audit agencies have improved the prior-years 50-50 report overall, but more so in the Army and the Air Force. Nonetheless, problems still exist, and this is particularly true in the Navy, where inadequate management oversight has resulted in continuing weaknesses in reporting control processes and data validation procedures. These weaknesses make it difficult to verify the reliability of reported data. Correcting reporting accuracy problems in all the services is necessary to provide the Congress and DOD managers assurance that the requirements of 10 U.S.C. 2466 are being met. Also, inaccurate data hinder Defense managers in taking timely actions to meet the statutory requirements and leave the Congress uncertain as to whether legislative requirements are being met.

The future-years report is not accurate or reasonable and is not currently a useful tool for guiding DOD actions or informing the Congress about likely future compliance with section 2466 requirements. The management of the military services placed much less emphasis on ensuring the accuracy or reasonableness of the future-years data. Accurate or reasonable projections are of particular importance for the Air Force, which has now issued two waivers of the limitation on the private sector and is likely to exceed the 50-percent ceiling in future years. The Army also faces increasing management challenges in managing its depot maintenance work within the 50-percent ceiling, but problems with the reliability of the data and the lack of an effective review of the future-years data and process by the Army Audit Agency concealed the impact of the results. Admittedly, projecting the future public-private sector mix is much more difficult and much less precise than quantifying the results of what has already occurred. Depot plans and strategies are still evolving with uncertain impacts on depot workloads. Similarly, repair plans for new and upgraded systems and other logistics programs and initiatives impacting the amount and location of depot maintenance services are not fully known. Yet future–year projections must use the best information available to make the most reasonable estimates. Such information should include the latest budget estimates with reasonable adjustments made as needed.

Although DOD and service guidance for both the prior and future-years reports has been improved over the years, we continue to identify errors, omissions, and inconsistencies in several reporting categories. These were caused in large part by insufficient direction and clarification in the reporting guidance and inadequate management attention. These problem areas include inactivation activities, contractor logistics support, depot maintenance at non-depot locations, government-furnished material, contract general and administrative expenses, incorporation of future repair costs for new systems, and adjustments for expected execution of programmed workload. In addition, record-keeping weaknesses hinder audit and management oversight efforts and do not provide a sound historical record for facilitating future data collection and reporting efforts. With improved management oversight and direction and the implementation of the required corrective actions, the 50-50 report could become a more useful management tool for DOD and the Congress in managing the Department’s depot maintenance program to attain future compliance with the 50-50 requirement.

Recommendations for Executive Action

To improve 50-50 data collection, validation, and reporting processes for prior-years and future years data and thus the reliability and reasonableness of the reported data, and to improve management direction and oversight, we recommend that the Secretary of Defense require the Secretary of the Army (1) identify depot maintenance requirements associated with the recapitalization program, (2) require that the Army Audit Agency review both prior-years and future-years 50-50 data, (3) communicate the reporting requirements to all organizational levels responsible for reporting data, and (4) finalize and issue guidance concerning the reporting of depot maintenance at non-depot locations; the Secretary of the Navy (1) review the management priority accorded the 50-50 reporting process throughout the command structure, (2) implement improved management controls and oversight of the processes used by the individual reporting commands to collect, verify, and report 50-50 data, (3) finalize procedures for accurately identifying and reporting depot maintenance costs at regional and other non-depot locations, (4) prior to issuing the data call for the 50-50 reports due in fiscal year 2002, hold a planning meeting of key officials representing all reporting commands to discuss and agree upon 50-50 data collection processes and guidance, and (5) direct the Naval Audit Service to review 50-50 processes and data to validate the data collection processes and results for both the prior-years and future-years reports; the Assistant Deputy Under Secretary of Defense for Maintenance Policy, Programs and Resources expand and clarify its guidance to (1) specify whether contract general and administrative expenses incurred by government employees and similar types of costs should be counted as part of the public or the private sector, and (2) allow for revisions to budgetary estimates to better reflect known and anticipated changes in workloads, workforce, priorities, and performance execution rates in order to achieve more reasonable projections of depot requirements where historical data indicates that budget data are unrealistic; the Assistant Deputy Under Secretary of Defense for Maintenance Policy, Programs and Resources, in conjunction with the secretaries of the military departments, improve and clarify 50-50 reporting guidance in problem areas noted in this report, including inactivation activities, contractor logistics support, incorporation of future repair costs for new and upgraded systems in 50-50 projections, depot-level maintenance performed at non-depot locations, and inclusion of government-furnished material in contract repair costs; and the secretaries of the military departments reemphasize and expand procedures for maintaining adequate records to document data collection processes, data sources, and estimating methodologies in order to facilitate management oversight and audits, as well as provide an historical record that can be readily used by staff newly assigned to the 50-50 process to annually replicate sound, efficient, and consistent data collection efforts.

Agency Comments and Our Evaluation

DOD generally concurred with our recommendations. However, it did not concur with certain parts of two recommendations. The Department’s specific comments and our evaluation of them are discussed below. Service officials also offered some technical comments that we incorporated in this report where appropriate. DOD’s comments are included as appendix I to this report.

The Department did not concur with two parts of our recommendation addressing Navy 50-50 issues. First, regarding our recommendation that the Navy hold a 50-50 planning meeting, the Navy response said that its new handbook, frequent contacts between the 50-50 manager and key reporting officials, and meetings held internally by reporting organizations accomplish the same purpose. While we agree that these efforts are important and should be continued, an initial planning meeting of key representatives of reporting commands has proven useful in both the Army and the Air Force. For example, service officials have said that these meetings surfaced reporting issues up-front and helped ensure more consistency in reporting processes and results. Thus, we continue to believe the Navy should hold an initial planning meeting. Second, the Navy disagreed with our recommendation that the Naval Audit Service review 50-50 processes and data. The Department’s response noted that the Navy believes sufficient management attention has been given to this process and is confident in the integrity of its data. However, the comments also stated that the Naval Audit Service will be used should the Navy determine that an audit service review is necessary. Because of the value added by audit services in the Army and the Air Force (and by the Navy last year), we continue to believe that the Naval Audit Service should be used to review 50-50 processes and data.

Finally, the Department did not concur with one part of our recommendation regarding clarification of Office of Secretary of Defense 50-50 guidance to the services. Specifically, the Department’s response noted that the counting of contract general and administrative expenses incurred by government employees is unique to the Air Force and additional departmental guidance is not necessary. We continue to believe that this issue should be clarified in the OSD guidance because a substantial amount is involved and the Air Force continues to make this adjustment every year.

Scope and Methodology

To determine whether the military departments met the 50-50 requirement in the prior-years report, we analyzed each service’s procedures and internal management controls for collecting and reporting depot maintenance information for purposes of responding to the section 2466 requirement. We reviewed supporting details (summary records, accounting reports, budget submissions, and contract documents) at departmental headquarters, major commands, and selected maintenance activities. We compared processes to determine consistency and compliance with legislative provisions, OSD guidance, and military service instructions. We selected certain programs and maintenance activities for more detailed review. We particularly examined reporting categories that DOD personnel and we had identified as problem areas in current and past reviews; these areas included interserviced workloads, contractor logistics support, warranties, software maintenance, and depot maintenance at non-depot locations. We evaluated processes for collecting and aggregating data to ensure accurate and complete reporting and to identify errors, omissions, and inconsistencies. We coordinated our work, shared information, and obtained results of the Army and Air Force service audit agencies’ data validation efforts.

To determine whether the future-years projections were based on accurate data, valid assumptions, and existing plans, and represented reasonable estimates, we followed the same general approach and methodology used to review the report on the preceding years discussed above. Although the future-years report is a budget-based projection of expenditures, the definitions, guidance, organization, and processes used to report future data are much the same as for the prior-years report of actual obligations. We discussed with DOD officials the main differences between the two processes and the manner in which the data were derived from budgets and planning requirements and key assumptions made in the outyear data.

For reviews of both 50-50 reports, we performed certain checks and tests, including variance analyses, to judge the consistency of this information with data from prior years and with the future-years budgeting and programming data used in DOD’s budget submissions and reports to the Congress. For example, we compared each service’s 50-50 data reported in February and April 2001 for the period 1999 through 2004 with data reported for these same years in the 50-50 reports submitted in 2000. We found repeated and significant changes, even though the estimates were prepared only about one year apart. This analysis helped us identify large transcription errors and unreported costs that the Army had made which resulted in the data reported to Congress erroneously indicating an increase in the percentage of depot maintenance work assigned to the public sector. Instead, our corrected data shows the Army allocation percentages staying rather constant during this period and closer to the 50-percent ceiling. This analysis also revealed a greater increase in the Navy’s shift to more private sector workload than had been projected last year. Variance analysis showed that congressional and DOD decisionmakers were given quite a different view of the public-private sector workload mix than that presented just last year. During this review we also used to a great extent our prior and ongoing audits in such areas as sustainment planning, depot policies, financial systems and controls, and DOD pilots and initiatives for increasing contractor involvement in maintenance.

Several factors concerning data validity and completeness were considered in our methodology and approach to reviewing the prior and future years’ reports. One key factor is the continuing deficiencies GAO has noted in DOD’s financial systems and reports that preclude a clean opinion on its financial statements and which results in limited accuracy of budget and cost information. Another factor is that documenting depot maintenance workload allocations between the public and private sectors is becoming more complicated by the consolidation of maintenance activities and performance of depot-level maintenance at field locations. This (1) makes it more difficult to identify work that meets the statutory definition of depot maintenance; (2) complicates workload reporting; and (3) results in underreporting of depot maintenance for both the public and private sectors. In addition, many contracts, especially the newer performance-based contracts, do not separately identify maintenance activities or account separately for their costs, which can result in under- and over-reporting of depot maintenance work performed in the private sector.

To review DOD efforts to improve the accuracy and completeness of reports, we discussed with officials managing and coordinating the reporting process their efforts to address known problem areas and respond to recommendations by the audit agencies and us. We compared this year’s sets of instructions with last year’s to identify changes and additions. We reviewed efforts to identify reporting sources and to distribute guidance and taskings. We asked primary data collectors their opinions on how well efforts were managed and data verified. We asked them to identify “pain points” and ideas they had to improve reporting. We reviewed prior recommendations and service audit agency findings to determine whether known problem areas were being addressed and resolved.

We interviewed officials, examined documents, and obtained data at OSD, Army, Navy, and Air Force Headquarters in Washington, D.C.; Army Materiel Command in Alexandria, Virginia; Naval Sea Systems Command in Arlington, Virginia; Naval Air Systems Command in Patuxent River, Maryland; Air Force Materiel Command in Dayton, Ohio; Army Audit Agency in Washington, D.C.; Air Force Audit Agency in Dayton, Ohio; and several operating activities under the military departments’ materiel commands. We conducted our review from February to July 2001, in accordance with generally accepted government auditing standards.

We are sending copies of this report to the Secretary of Defense, the Secretary of the Air Force, the Secretary of the Army, the Secretary of the Navy, the Commandant of the Marine Corps, the Director of Office of Management and Budget, and interested congressional committees. We will make copies available to others upon request and will post the report to GAO’s homepage at www.gao.gov. Key contributors to this report are listed in appendix II.

Appendix I: Comments From the Department of Defense

Appendix II: GAO Contacts and Staff Acknowledgments

GAO Contacts

Acknowledgments

In addition to the above contacts, John Brosnan, Raymond Cooksey, Bruce Fairbairn, Johnetta Gatlin-Brown, Jane Hunt, Steve Hunter, Glenn Knoepfle, Ron Leporati, Fred Naas, Andy Marek, and Bobby Worrell made contributions to this report.

Related GAO Products

Defense Maintenance: Sustaining Readiness Support Capabilities Requires a Comprehensive Plan (GAO-01-533T, Mar. 23, 2001)

Depot Maintenance: Key Financial Issues for Consolidations at Pearl Harbor and Elsewhere Are Still Unresolved (GAO-01-19, Jan. 22, 2001).

Depot Maintenance: Action Needed to Avoid Exceeding Ceiling on Contract Workloads (GAO/NSIAD-00-193, Aug. 24, 2000).

Air Force Depot Maintenance: Budgeting Difficulties and Operational Inefficiencies (GAO/AIMD/NSIAD-00-185, Aug. 15, 2000.)

Depot Maintenance: Air Force Waiver to 10 U.S.C. 2466 (GAO/NSIAD-00-152R, May 22, 2000).

Depot Maintenance: Air Force Faces Challenges in Managing to 50-50 Ceiling (GAO/T-NSIAD-00-112, Mar. 3, 2000).

Depot Maintenance: Future Year Estimates of Public and Private Workloads Are Likely to Change (GAO/NSIAD-00-69, Mar. 1, 2000).

Depot Maintenance: Army Report Provides Incomplete Assessment of Depot-type Capabilities (GAO/NSIAD-00-20, Oct. 15, 1999).

Depot Maintenance: Status of the Navy’s Pearl Harbor Project (GAO/NSIAD-99-199, Sep. 10, 1999).

Depot Maintenance: Workload Allocation Reporting Improved, but Lingering Problems Remain (GAO/NSIAD-99-154, July 13, 1999).

Navy Ship Maintenance: Allocation of Ship Maintenance Work in the Norfolk, Virginia, Area (GAO/NSIAD-99-54, Feb. 24, 1999).

Defense Depot Maintenance: Public and Private Sector Workload Distribution Reporting Can Be Further Improved (GAO/NSIAD-98-175, July 23, 1998).

Defense Depot Maintenance: DOD Shifting More Workload for New Weapon Systems to the Private Sector (GAO/NSIAD-98-8, Mar. 31, 1998).

Defense Depot Maintenance: Information on Public and Private Sector Workload Allocations (GAO/NSIAD-98-41, Jan. 20, 1998).

Outsourcing DOD Logistics: Savings Achievable But Defense Science Board’s Projections Are Overstated (GAO/NSIAD-98-48, Dec. 8, 1997).

Navy Regional Maintenance: Substantial Opportunities Exist to Build on Infrastructure Streamlining Progress (GAO/NSIAD-98-4, Nov. 13, 1997).

Defense Depot Maintenance: Uncertainties and Challenges DOD Faces in Restructuring Its Depot Maintenance Program (GAO/T-NSIAD-97-112, May 1, 1997) and (GAO/T-NSIAD-97-111, Mar. 18, 1997).

Defense Depot Maintenance: DOD’s Policy Report Leaves Future Role of Depot System Uncertain (GAO/NSIAD-96-165, May 21, 1996).

Defense Depot Maintenance: More Comprehensive and Consistent Workload Data Needed for Decisionmakers (GAO/NSIAD-96-166, May 21, 1996).

Defense Depot Maintenance: Privatization and the Debate Over the Public-Private Mix (GAO/T-NSIAD-96-148, Apr. 17, 1996) and (GAO/T-NSIAD-96-146, Apr. 16, 1996).

Depot Maintenance: Issues in Allocating Workload Between the Public and Private Sectors (GAO/T-NSIAD-94-161, Apr. 12, 1994).