GAO

United States General Accounting Office

Report to the Chairman, Subcommittee 

on Readiness and Management Support, 

Committee on Armed Services, U.S. 

Senate

September 1999 DEPOT 

MAINTENANCE

Status of the Navy’s 

Pearl Harbor Pilot 

Project

GAO/NSIAD-99-199

Page 1 GAO/NSIAD-99-199 Depot Maintenance

Contents

Letter 3

Appendixes Appendix I Scope and Methodology 26

Appendix II Description of the Pearl Harbor Pilot Test Plan and 

Its Strengths and Weaknesses 29

Appendix III GAO Contacts and Staff Acknowledgments 42

Related GAO Products 43

Tables Table 1: The Navy's Nine Test Metrics for Evaluating the Pearl Harbor 

Pilot's Effectiveness 7

Table 2: Preliminary Results, Strengths, and Weaknesses of the Pearl 

Harbor Pilot Test Metrics 10

Table 3: Comparison of Ship Maintenance Activities in the Pearl

Harbor, Puget Sound, Portsmouth, and Norfolk Areas 15

Table 4: Navy's Initial Two Metrics to Test Whether the Consolidated

Pearl Harbor NSY & IMF Increased Manpower Utilization and

Lowered Unit Cost of Production 29

Table 5: Metrics Three Through Six to Test Whether the Consolidated

Pearl Harbor NSY & IMF Increased Manpower Utilization and

Lowered Unit Cost of Production 31

Table 6: Metrics Seven Through Nine to Test Whether the Consolidated 

Pearl Harbor NSY & IMF Increased Manpower Utilization and

Lowered Unit Cost of Production 32

Table 7: Baseline, Performance Expectations, and Preliminary

Results of the Test Plan 33

Figures Figure 1: Breakdown of the Naval Shipyards' Fiscal Year 1997

Workloads by Major Customer 17

Figure 2: Comparison of Reported NWCF Revenues by Activity

Group for Fiscal Year 1998 20
Contents

Page 2 GAO/NSIAD-99-199 Depot Maintenance

Abbreviations

CNO Chief of Naval Operations

CSMP Current Ship Maintenance Program

DOD Department of Defense

NAS Naval Audit Service

NAVSEA Naval Sea Systems Command

NIMF Naval Intermediate Maintenance Facility

NSY & IMF Naval Shipyard & Intermediate Maintenance Facility

NWCF Navy Working Capital Fund

PACFLT Pacific Fleet

PHNSY Pearl Harbor Naval Shipyard
Page 3 GAO/NSIAD-99-199 Depot Maintenance

United States General Accounting Office

Washington, D.C. 20548

National Security and

International Affairs Division

B-280524 Leter

September 10, 1999

The Honorable James M. Inhofe

Chairman, Subcommittee on Readiness

 and Management Support

Committee on Armed Services

United States Senate

Dear Mr. Chairman:

In recent years, the Navy has implemented many changes aimed at making 

its fleet support activities more efficient and effective. An underlying 

assumption was that consolidating similar activities within a region could 

eliminate the inefficiencies and redundancies. In 1998, as part of this effort, 

the Navy implemented a pilot project consolidating the management, 

operations, and funding of the Pearl Harbor Naval Shipyard and the Naval 

Intermediate Maintenance Facility in Hawaii. The Navy expects that the 

pilot project, commonly called the Pearl Harbor Pilot, will serve as a model 

for other potential consolidations by confirming that integrating shipyard 

and intermediate activities can result in greater efficiency and lower overall 

unit costs.

As you requested, we examined the Navy's progress in implementing this 

pilot project. This report discusses (1) the preliminary results of the Pearl 

Harbor Pilot on improving performance of maintenance activities, (2) the 

usefulness of the pilot as a model for future consolidations, and (3) issues 

related to financial and organizational structures for such consolidations. 

Our scope and methodology are described in appendix I.

Results in Brief Although the Pearl Harbor Pilot is not complete, preliminary results have 

been mixed, showing either improvements, no improvements, or 

insufficient data to determine results. Where data are available, overall 

indications are that the pilot has the potential to improve maintenance 

activities in Hawaii. While the Navy's pilot test plan calls for evaluating 

performance using nine metrics, data has been gathered for only five to 

compare performance under the consolidated operation with performance 

under the preconsolidation organizations. Preliminary results for two 

metrics indicate improvements that meet or exceed the Navy's 

expectations, and two others indicate improvements that fall slightly short 
B-280524

Page 4 GAO/NSIAD-99-199 Depot Maintenance

of the Navy's expectations, while one metric indicates no improvement. 

Other positive results include increasing workforce flexibility by 

integrating nearly 4,000 workers from two work centers into a single 

workforce. Consequently, the number of workers assigned daily to the 

excess labor shop has dropped from about 200 to below 10. Furthermore, 

the Navy has reduced the maintenance infrastructure by 11 buildings 

(114,131 square feet) and plans to eliminate another 6 buildings (24,907 

square feet). At the same time, other anticipated efficiencies have yet to be 

realized. For example, not all the industrial plant equipment has been 

consolidated and the projected number of overhead workers has not been 

moved to direct maintenance positions to improve productivity.

The Pearl Harbor Pilot is likely to serve only as a general model for future 

consolidation efforts because of unique aspects of ship maintenance 

activities in Hawaii, such as the close proximity of facilities and the large 

portion of fleet-funded work than in other locations. Likewise, because of 

this uniqueness, the pilot provides only a general indication that future 

consolidations elsewhere will result in similar efficiencies. Nevertheless, it 

does provide general information on such issues as combining workforces 

from separate activities and consolidating equipment and facilities.

The Pearl Harbor Pilot has sharpened the debate over the most appropriate 

financial and organizational structures for such consolidated activities. 

Conflicting views continue to exist within the Departments of Defense and 

the Navy over whether these consolidated activities should operate under 

direct appropriations, the Navy Working Capital Fund, or a combination of 

the two financial structures, particularly in terms of maintaining visibility 

over total operational costs. Similarly, little progress has been made toward 

resolving the departments' differences over the most appropriate 

organizational structure to manage these types of activities.

This report recommends that the Navy take steps to address unresolved 

issues related to financial and organizational structures as it proceeds with 

similar consolidations.

Background In March 1994, the Chief of Naval Operations (CNO) announced a Regional 

Maintenance Program to streamline the Navy ship repair and maintenance 

processes, reduce infrastructure and costs, and maximize outputs. The 

program consisted of three phases: (1) optimizing intermediate-level 

maintenance through consolidation of intermediate activities,

(2) integrating intermediate and depot activities with management by the 
B-280524

Page 5 GAO/NSIAD-99-199 Depot Maintenance

fleet commanders, and (3) conducting fleet maintenance using a single 

process. The first phase, optimizing intermediate-level maintenance by 

minimizing redundant intermediate ship maintenance capacity and 

capability, is nearing completion. To ensure the validity of the second phase 

of the program and to provide a model for other potential consolidations, 

the Navy implemented the Pearl Harbor Pilot in 1998. Its completion date 

has not been established. The third phase, performing a single maintenance 

process by using common business and production processes, is under way 

and scheduled to be completed in fiscal year 2001.

The Pearl Harbor Pilot Prior to the consolidation, the Navy recognized that organizational and 

funding structures inhibited the ability of the Pearl Harbor Naval Shipyard 

(PHNSY) and Naval Intermediate Maintenance Facility (NIMF) to readily 

share workloads and resources. The shipyard was managed by the Naval 

Sea Systems Command (NAVSEA) and funded through the Navy Working 

Capital Fund (NWCF),1

 while the intermediate maintenance facility was 

managed by the Pacific Fleet (PACFLT) and financed through direct 

appropriations.2

 During the pilot, PACFLT assumed ownership and overall 

management and financial responsibility for the consolidated activity, 

while NAVSEA continued to be the technical and operating authority. The 

Navy named the combined activity the Pearl Harbor Naval Shipyard & 

Intermediate Maintenance Facility (NSY & IMF). To achieve a fully 

integrated organization, the Navy decided the Pearl Harbor Pilot should use 

a single financial structure and selected direct appropriations rather than 

the working capital fund as the funding approach. This latter decision was 

based on several factors, but Navy officials believed that direct 

appropriations were better suited to achieve the pilot goals and matched 

the financial structure of the largest shipyard customer−PACFLT.

1

 Under the working capital fund arrangement, activities such as a shipyard sell goods and 

services to customers based on predetermined rates designed to recoup the full cost of 

goods and services, including any military personnel costs. Customers pay for the goods and 

services with their operation and maintenance funds. The fund's accounting practices and 

related requirements for disclosure of information are intended to help managers and 

workers focus on production costs and performance. However, the Department of Defense 

(DOD) has long-standing problems in accumulating and reporting the full costs associated 

with its working capital fund operations.

2

 Direct appropriations are sometimes referred to as mission funding by DOD and Navy 

officials. They are funds appropriated directly by the Congress that authorize agencies to 

incur obligations for designated purposes. An agency may not spend more than the amount 

appropriated.
B-280524

Page 6 GAO/NSIAD-99-199 Depot Maintenance

Furthermore, in recent years there has been increasing Navy 

dissatisfaction with the costs associated with the working capital fund. 

Given the nature of the fund and its emphasis on fully recovering costs of 

the services provided, the perception sometimes exists that the fund 

included fees and charges that inflated ship maintenance costs compared 

to mission funding, which does not reflect the full cost of operations. In 

accordance with one of the basic tenets of a working capital fund, the costs 

associated with ship maintenance are more visible when financed using 

this mechanism. While the level of resources required to carry out ship 

maintenance activities is likely to be similar regardless of whether financed 

using mission funding or through a working capital fund, under the 

working capital fund the customer is more likely to be directly responsible 

for a larger portion of those costs. Use of a working capital fund better 

enables DOD components to fully account for their share of the program 

costs. In commenting on a draft of this report, the Navy said that it chose 

the Shipyard Management Information System as the cost accounting 

system for the Pearl Harbor Pilot. According to the Navy, this system 

(which is used by other naval shipyards under the NWCF) provides the 

pilot the same degree of cost visibility as a working capital fund activity.

Full accounting for the costs of federal programs is required by federal 

accounting standards. Specifically, Statement of Federal Financial 

Accounting Standards No. 4 requires federal agencies to accumulate the 

full cost of outputs through appropriate costing methodologies or cost 

finding techniques (the full cost of an output is the sum of (1) the cost of 

resources consumed that directly or indirectly contributes to the output 

and (2) the cost of identifiable supporting services provided by other 

entities).

After Expressing Concerns, 

Department of Defense 

Officials Approved the Test 

Plan in September 1998

In 1998, as Navy officials were formulating plans for the Pearl Harbor Pilot, 

DOD officials expressed concern that the Navy's test plan and baseline data 

would not adequately measure the pilot's impact on productivity and 

performance. Consequently, the Navy revised the test plan and tried to 

minimize the data limitations in the baseline. Ultimately, as shown in table 

1, the Navy selected nine test metrics to evaluate the pilot's effectiveness.
B-280524

Page 7 GAO/NSIAD-99-199 Depot Maintenance

Table 1: The Navy's Nine Test Metrics for Evaluating the Pearl Harbor Pilot's 

Effectiveness

Source: The Pearl Harbor Pilot Test Plan, as amended, Department of the Navy (June 30, 1998).

In June 1998, the DOD Inspector General had recommended the Navy delay 

the pilot and restart the data collection process, applying the same data 

collection methodologies and controls for the baseline that would be used 

to collect data during the pilot period. According to DOD Comptroller and 

Inspector General officials, the most severe of the metrics' limitations 

affected the baseline costs for the first two test metrics shown in table 1. 

For example, military personnel costs were not traditionally collected by 

intermediate maintenance facilities because these facilities were largely 

funded out of operation and maintenance appropriations while military 

personnel costs were already paid for separately out of military personnel 

appropriations. Consequently, military personnel costs were also 

accounted for separately. In addition, some labor hours appeared to be 

inaccurate. One DOD contractor reported the NIMF database contained 

zero labor hours for approximately 6 percent of the maintenance tasks 

completed by the facility. Further, some Navy officials expressed concern 

that some recorded labor hours appeared to reflect the expected work 

standards and not actual production hours.

Metric

1. Total cost of a production shop, direct labor hour of work delivered to the customer 

(indicator of the pilot's efficiency in terms of the cost per direct maintenance hour)

2. Total labor hours expended to deliver a production shop, direct labor hour to the 

customer (indicator of the pilot's productivity in terms of personnel utilization)

3. Total Current Ship Maintenance Program work items completed (indicator of the pilot's 

productivity in terms of the number of work items completed)

4. Total Current Ship Maintenance Program work items in the backlog (indicator of the 

material conditions of PACFLT ships in terms of work items not yet completed)

5. Schedule adherence of CNO maintenance projects (indicator of customer satisfaction 

in terms of completing projects on schedule)

6. Rework index for CNO maintenance projects (indicator of the pilot's quality of work in 

terms of hours required to correct work deficiencies)

7. Activity work schedule integrity index (indicator of customer satisfaction in terms of 

completing projects on schedule)

8. Casualty reports caused by activity work (indicator of the pilot's quality of work in terms 

of casualty reports—ship reports indicating equipment failure)

9. Earned value (indicator of the pilot's productivity in terms of hours to complete similar 

work items)
B-280524

Page 8 GAO/NSIAD-99-199 Depot Maintenance

In response to the DOD Inspector General recommendation, Navy officials 

said the pilot was too far along in the integration process in June 1998 to 

reestablish separate maintenance activities without incurring significant 

cost and operational turmoil. The expected cost and turmoil of such a 

reversal, according to these officials, would outweigh any potential benefit 

obtained from developing a better baseline. Consequently, the Navy revised 

the test plan to minimize the data limitations. For example, as CNO and 

NAVSEA officials requested, the Naval Audit Service (NAS) agreed to 

validate the baseline and pilot results for six of the nine test metrics.3

 In 

several instances, baseline data were routinely collected by 

preconsolidation activities, and they are available to provide a basis for 

comparison with postconsolidation performance data. In addition, as 

requested by CNO and NAVSEA officials, the NAS developed a 

methodology to estimate baseline values for the first two metrics. The 

methodology is based on the Navy's personnel wages and standards, as 

well as, on the best available data from the former NIMF. Although the 

resulting baseline estimates for the two metrics may not be exact, Navy 

officials believe they are comparable to historical trends for intermediate 

maintenance facilities and, consequently, provide a reasonable baseline on 

which to indicate the pilot's impact on productivity and performance. On 

September 14, 1998, the Principal Deputy Under Secretary of Defense 

(Comptroller) approved the Navy's test plan. Appendix II provides more 

detail concerning the evolution of the test plan and the nine metrics.

Congressional 

Requirements for the Pearl 

Harbor Pilot

The Conference Report for the DOD Appropriations Act for Fiscal Year 

19984 recognized that the Navy planned to initiate a pilot to study whether 

combining fleet intermediate maintenance facilities with Navy shipyards 

might yield economies of scale and allow maintenance managers to better 

balance workloads. The conferees concluded that it would take at least

2 years before the Navy could determine whether this new arrangement 

was effective and should be made permanent or expanded to other 

locations. Further, the report directed the Navy not to expand the pilot until 

6 months after it reported to the Committees on Appropriations on its 

findings, and that such report be made on or after April 1, 1999. The 

conferees also directed the Navy not to make any permanent changes to 

3

 Previously, the NAS had agreed to collect and validate the baseline and pilot results for the 

first two test metrics.

4

 House Report 105-265, September 23, 1997.
B-280524

Page 9 GAO/NSIAD-99-199 Depot Maintenance

the workforce in terms of total number of employees or any other 

permanent changes until the pilot had been completed and evaluated by the 

Congress. Currently, the Navy plans to issue an interim report on the pilot 

shortly after October 1999.

While the Pilot Has 

Potential to Improve 

Maintenance Activities, 

Preliminary Data Show 

Mixed Results

Preliminary results of the Pearl Harbor Pilot have been mixed, showing 

either improvements, no improvements, or insufficient data to determine 

results. For most of the measures where data are available, indications are 

that the pilot has the potential to improve maintenance activities in Hawaii. 

While the test plan calls for using nine metrics, the data has been gathered 

for only five of them. For these five metrics, preliminary results for two 

metrics indicate improvements that meet or exceed the Navy's 

expectations; two others indicate improvements that fall slightly short of 

the Navy's expectations; and one metric indicates no improvement. Other 

positive results include the fact that the Navy has integrated the 

workforces from two work centers into a common pool, increased 

management flexibility in assigning workers to maintenance projects based 

on evolving priorities, and reduced the maintenance infrastructure at Pearl 

Harbor. At the same time, other planned efficiencies are yet to be realized. 

For example, the pilot has not resulted in the number of overhead workers 

projected to move to direct maintenance positions or in the consolidation 

of all industrial plant equipment.5

 According to Navy officials, these actions 

would have increased the number of people working directly on 

maintenance projects and the utilization of existing plant equipment.

The Pilot Has the Potential 

to Improve Maintenance 

Activities in Hawaii

The nine metrics chosen for the Pearl Harbor Pilot Test Plan represent a 

variety of issues and performance indicators. Although data available for 

some of the metrics are limited, it is sufficient to provide a general, if 

imprecise, indication of the pilot's impact on productivity and 

performance. In this case, as indicated in table 2, while test results have not 

yet been fully developed, the preliminary results have been mixed, showing 

either improvements, no improvements, or insufficient data to determine 

5

 In general, overhead workers are those employees who are not working directly on 

maintenance projects and are assigned to administrative and support positions such as 

personnel and financial departments. Direct maintenance workers are those employees who 

accomplish work directly identifiable to a maintenance project such as repair, maintenance, 

and installation of equipment and components.
B-280524

Page 10 GAO/NSIAD-99-199 Depot Maintenance

results. Where data are available, overall indications are that the pilot has 

the potential to improve maintenance activities in Hawaii.

Table 2: Preliminary Results, Strengths, and Weaknesses of the Pearl Harbor Pilot Test Metrics

Metric (number) and 

anticipated goal Preliminary results Strengths (S) and weaknesses (W)

Improvement indicated

Total cost of a production shop, 

direct labor hour of work delivered 

to the customer (1)

Indicator of the pilot's efficiency in 

terms of the cost per direct 

maintenance hour

Indicates improvement because 

preliminary data show that it has 

cost slightly less to deliver one 

direct maintenance hour in fiscal 

year 1998, but falls short of the 

Navy's expectation

Variant of a traditional indicator of shipyard efficiency (S)

Shipyard baseline data are considered reliable (S)

NAS developed baseline estimates and plans to validate and 

generate the test data (S)

NAS developed baseline estimates of the NIMF military 

personnel costs because the facility did not maintain this type of 

data (W)

NIMF baseline data on production hours have limitations, but 

are considered reliable enough by Navy officials to provide a 

rough, if imprecise, estimate of the baseline (W)

Total labor hours expended to 

deliver a production shop, direct 

labor hour to the customer (2)

Indicator of the pilot's productivity 

in terms of personnel utilization

Indicates improvement because 

preliminary data show that it has 

taken fewer hours to deliver one 

direct maintenance hour in fiscal 

year 1998

Variant of a traditional indicator of shipyard productivity (S)

Shipyard baseline data are considered reliable (S)

NAS developed baseline estimates and plans to validate and 

generate the test data (S)

NIMF baseline data on production hours have limitations but are 

considered reliable enough by Navy officials to provide a rough, 

if imprecise, estimate of the baseline (W)

Total Current Ship Maintenance 

Program work items in the 

backlog (4)

Indicator of the material 

conditions of PACFLT ships in 

terms of work items not yet 

completed

Indicates improvement because 

preliminary data show that there 

are fewer backlogged work items 

since the consolidation, but falls 

short of the Navy's expectation

Traditionally considered an indicator of the material condition of 

ships (S)

Fleet and ship commanders maintain baseline and test data, 

which are considered reliable (S)

NAS plans to validate the baseline and test data (S)

Is an indicator of the NIMF and the combined facility's 

productivity but not the former shipyard (W)

Schedule adherence of CNO ship 

maintenance projects (5)

Indicator of customer satisfaction 

in terms of completing projects on 

schedule

Indicates improvement because 

preliminary data show that more 

CNO projects are being completed 

on time or ahead of schedule since 

the consolidation

Traditionally considered an indicator of the shipyard customers' 

satisfaction (S)

NAVSEA maintains baseline and test data, which are 

considered reliable (S)

NAS plans to validate the baseline and test data (S)

Is an indicator of shipyard and the combined facility's 

performance but not the former NIMF (W)

No improvement indicated

Total Current Ship Maintenance 

Program work items completed 

(3)

Indicator of the pilot's productivity 

in terms of the number of work 

items completed

Indicates no improvement because 

preliminary data show that there 

are fewer work items being 

completed since the consolidation

Fleet and ship commanders maintain baseline and test data, 

which are considered reliable (S)

NAS plans to validate the baseline and test data (S)

Is an indicator of the NIMF and the combined facility's 

productivity, but not the former shipyard (W)

Is affected by other factors not related to the pilot (W)

(continued)
B-280524

Page 11 GAO/NSIAD-99-199 Depot Maintenance

Note(s): Results as of June 22, 1999.

Appendix II provides more detail concerning the strengths and weaknesses of the nine metrics.

Source: Interviews with Navy officials and our analysis of the Pearl Harbor Pilot Test Plan dated

June 30, 1998; NAS capacity evaluation reports for fiscal years 1997 and 1998; and pilot status 

reports.

As shown in table 2, two test metrics indicate improvements that meet or 

exceed the Navy's expectations and two indicate improvements that fall 

slightly short of the Navy's expectations; only one metric indicates no 

improvement. Because of the operational turbulence experienced during 

the consolidation, we believe that it is reasonable to expect that some 

productivity and performance indicators might show diminished 

improvement during the transition year. In addition, as indicated in the 

table, the Navy has not gathered performance data for the remaining four 

metrics. The Navy is in the process of collecting test data on the rework 

index for CNO maintenance projects. However, Navy officials do not plan 

Metric (number) and 

anticipated goal Preliminary results Strengths (S) and weaknesses (W)

Not yet determined

Rework index for CNO 

maintenance projects (6)

Indicator of the pilot's quality of 

work in terms of hours required to 

correct work deficiencies

Not yet determined because the 

Navy requires additional time to 

develop the test data

Traditionally considered an indicator of the shipyard quality of 

work (S)

NAS plans to validate the baseline and test data (S)

Is an indicator of shipyard and the combined facility's 

performance, but not the former NIMF (W)

Activity work schedule integrity 

index (7)

Indicator of customer satisfaction 

in terms of completing projects on 

schedule

Not yet determined because the 

Navy does not plan to gather data 

needed to develop this metric 

unless the results of the first six 

metrics are inconclusive

Traditionally considered an indicator of the shipyard customers' 

satisfaction (S)

Shipyard maintains baseline and test data, which are 

considered reliable (S)

Is an indicator of shipyard and the combined facility's 

performance, but not the former NIMF (W)

Is similar to the fifth metric measuring schedules adherence 

and, as a result, Navy officials question the usefulness of 

developing this metric (W)

Casualty reports caused by 

activity work (8)

Indicator of the pilot's quality of 

work in terms of casualty reports

Not yet determined because the 

Navy does not plan to gather data 

needed to develop this metric 

unless the results of the first six 

metrics are inconclusive

Fleet and ship commanders maintain baseline and test data, 

which are considered reliable (S)

Is an indicator of shipyard and the combined facility's 

performance, but not the former NIMF (W)

Is similar to the sixth metric measuring rework and, as a result, 

Navy officials question the usefulness of developing this metric 

(W)

Earned value (9)

Indicator of the pilot's productivity 

in terms of hours to complete 

similar work items

Not yet determined because the 

Navy does not plan to gather data 

needed to develop this metric 

unless the results of the first six 

metrics are inconclusive

Attempts to measure the pilot's effect on maintenance outputs 

(S)

Should require significant resources to develop results (W)

Is based on a limited universe of work items (W)
B-280524

Page 12 GAO/NSIAD-99-199 Depot Maintenance

to gather data needed to develop the last three metrics unless the test 

results are inconclusive, even though the three metrics were added to the 

test plan at the request of the Principle Deputy Under Secretary of Defense 

(Comptroller).

Other Efficiencies Identified 

During the Planning Process 

Have Been Achieved

In August 1997, the Pearl Harbor Regional Maintenance Pilot Executive 

Steering Committee issued the Pilot Study Report for the integration of the 

PHNSY and NIMF.6

 The purpose of the report was to outline the approach 

for this integration. Among other improvements, the report envisioned 

increasing productivity and cost-effectiveness by integrating the workers of 

former PHNSY and NIMF into a common workforce and reducing the ship 

maintenance infrastructure in Hawaii. As planned in the Pilot Study Report, 

the Navy has integrated nearly 4,000 workers from two separate work 

centers into a common pool, increased management flexibility in assigning 

workers to maintenance projects, and reduced the ship maintenance 

infrastructure.

Prior to the pilot, it was difficult to shift work or personnel between 

maintenance activities due to multiple financial structures. More 

specifically, the Navy stated that NWCF rules prohibited assigning workers 

to a job without a funding document, which made it difficult to shift work 

between maintenance activities. Accordingly, when administrative and 

financial requirements restricted the movement of workers, shipyard 

production personnel not working on a specific maintenance project were 

sent to the excess labor shop to wait assignment. During this time, they 

performed indirect work such as facility maintenance or grounds-keeping.7

Before the pilot, the number of workers assigned daily to this shop ranged 

between 100 to 200. By integrating the workforce under a single financial 

and command structure, production shops now have more flexibility in 

assigning excess workers to other projects, including projects historically 

completed by the former intermediate maintenance facility. Consequently, 

the number of workers assigned daily to the excess labor shop has dropped 

to below 10.

6 Pilot Study Report for an Integration of Pearl Harbor Naval Shipyard and Naval 

Intermediate Maintenance Facility, Pearl Harbor, Pearl Harbor Pilot Executive Steering 

Committee (Aug. 26, 1997).

7

 Limited-duty personnel with medical injuries or health problems and personnel whose 

security clearance levels were under review were also assigned to the excess labor shop.
B-280524

Page 13 GAO/NSIAD-99-199 Depot Maintenance

In addition, as a result of the consolidation, the Navy has reduced the 

maintenance infrastructure at Pearl Harbor. For example, 11 (114,131 

square feet) of the 27 buildings previously used by the former intermediate 

maintenance facility were vacated and turned over to Pearl Harbor Naval 

Station.8

 While the consolidated facility will retain 7 of the remaining

16 NIMF buildings, it plans to vacate another 6 (24,907 square feet) and is 

reviewing the disposition of 3 buildings.9

 As of June 1999, the Pearl Harbor 

Naval Station demolished or plans to demolish eight former NIMF buildings 

and was able to demolish two of its buildings after tenants moved into 

vacated NIMF buildings. The demolition costs for all 10 buildings is 

estimated at $1.9 million with a projected annual cost avoidance of 

$307,000 or a payback period of a little over 6 years.

Some Efficiencies Have Yet 

to Be Fully Achieved

The Pilot Study Report envisioned increasing productivity and

cost-effectiveness by moving overhead workers to direct maintenance 

positions and consolidating industrial plant equipment. However, these 

efficiencies have not yet been fully achieved.

For example, the consolidation did not result in the projected number of 

overhead workers moving to direct maintenance work−an important 

element in increasing the combined facility's productivity. To increase 

productivity, one of the stated goals of the pilot was to increase the number 

of direct production workers relative to the number of supervisors and 

overhead personnel without increasing costs. Logically, increasing the 

number of production workers should result in increased production. To 

accomplish this goal, the Pilot Study Report proposed that 95 civilian 

overhead workers be moved to positions in direct maintenance work. 

However, only four workers moved. According to Pearl Harbor NSY & IMF 

officials, while they have tried, it has been impracticable for them to move 

civilians to such positions because of (1) the time required for overhead 

workers to become skilled, usually several years; (2) regulations restricting 

the process for downgrades; and (3) potentially negative action by workers

8

 In general, the Pearl Harbor Naval Station has ownership and is responsible for the 

management and maintenance of Navy buildings in Hawaii.

9

 According to Navy officials, no former shipyard buildings will be vacated as a result of the 

Pearl Harbor Pilot.
B-280524

Page 14 GAO/NSIAD-99-199 Depot Maintenance

and employee representatives.10 In addition, several department directors 

and overhead supervisors said they were unwilling to release any personnel 

because of the increased workload due to changes in administrative and 

financial systems since the consolidation. Furthermore, other Navy 

officials said that efforts to move supervisors to direct production resulted 

in too few supervisors, which led to problems in planning and coordinating 

work on maintenance projects. According to CNO and NAVSEA officials, 

more overhead workers will be moved to direct maintenance positions as 

the pilot progresses.

In addition, industrial plant equipment has not been completely 

consolidated at integrated production shops. The Pilot Study Report 

suggested the in-shop industrial equipment be efficiently consolidated and 

redistributed to improve maintenance operations. Of the 271 items of 

industrial equipment at the former intermediate maintenance facility,

114 items were relocated, 132 items were mothballed, and 25 items were 

kept operational at their original location. According to officials formerly 

associated with the Pearl Harbor NSY & IMF, the 114 related items were 

equipment that required little or no removal and installation costs. 

Although many of the 132 mothballed items are in better condition and 

newer than the same type of equipment located currently in the 

consolidated shops, the funding required to relocate the NIMF equipment 

has not been available. Most mothballed items are semipermanently 

attached heavy equipment that requires funding to remove, transport, and 

install elsewhere. According to Pearl Harbor officials, congressional 

guidance directing the Navy not to make any permanent changes until after 

the pilot period has to a lesser degree influenced the decision not to 

relocate the mothballed items. They plan to request funding to move 

equipment after the pilot period is completed.

The Pearl Harbor Pilot Will 

Serve Only as a General 

Model for Similar 

Consolidations

The Pearl Harbor Pilot is likely to serve only as a general model for future 

consolidation efforts because of the unique aspects of ship maintenance 

activities in Hawaii, such as the close proximity of facilities and the 

significant portion of fleet-funded work. Likewise, because of this 

10 Our prior work examining DOD reform initiatives has noted the importance of top 

management commitment and sustained support for reform initiatives, as well as 

overcoming culture barriers and resistance to change. See Defense Reform Initiatives: 

Organization, Status, and Challenges (GAO/NSIAD-99-87, Apr. 21, 1999).
B-280524

Page 15 GAO/NSIAD-99-199 Depot Maintenance

uniqueness, the pilot provides only a general indication that future 

consolidations will result in similar efficiencies.

While there can be clear benefits to regionalization, available statistical 

data suggest conditions may have been more favorable to such efforts in 

Pearl Harbor than may exist at other Navy locations. For example, during 

fiscal years 1996 through 1998, the Navy reportedly replaced 698 military 

personnel at the former NIMF with 504 experienced civilian workers, 

mostly from the shipyard. Although civilian workers were periodically 

transferred between intermediate facilities and shipyards at other 

locations, Pearl Harbor is the only location where a significant number of 

workers has recently been transferred. Consequently, according to Navy 

officials, this large cadre of civilian workers with recent shipyard 

experience should have expedited the integration of the workforce by 

requiring less training and fewer trade skill certifications. This condition 

does not exist to such a large extent at other potential consolidation 

locations, except at the Puget Sound area. Table 3 compares some 

additional differences between Pearl Harbor and other Navy locations.

Table 3: Comparison of Ship Maintenance Activities in the Pearl Harbor, Puget Sound, Portsmouth, and Norfolk Areas

Note: Fiscal year 1998.

Source: Our analysis of interviews and documents obtained from Navy officials.

Dollars in millions

Category Pearl Harbor Puget Sound Portsmouth Norfolk

Time required to travel between the 

intermediate maintenance facilities 

and the shipyards 

• 5 to 10 minutes • 20 to 30 minutes to

the Trident Refit Facility

• 2 to 4 hours to Everett, 

Washington, maintenance 

facility

• 2 to 3 hours

to Groton, 

Connecticut, 

maintenance facility

• 30 to 45 minutes to 

the Shore 

Intermediate 

Maintenance 

Activity

Reported shipyard revenues $379.4 $1,041.8 $394.1 $822.1

Number of military personnel

• Shipyard

• Intermediate maintenance facility 

or activity

43

641

47

1,105

53

1,134

64

2,516

Number of civilian personnel

• Shipyard

• Intermediate maintenance facility 

or activity

2,742

554

8,823

967

3,286

149

7,127

23
B-280524

Page 16 GAO/NSIAD-99-199 Depot Maintenance

As indicated in table 3, Pearl Harbor facilities are located closer to each 

other than in other Navy locations. Consequently, consolidation of 

activities separated by greater distances at other Navy locations could be 

more difficult to manage and, after the consolidation is completed, could 

possibly reduce the combined facility's flexibility in the management of the 

workforce and workload. For example, given the close proximity of 

facilities in Hawaii, workers with multiple and specialized skills from both 

the former PHNSY and NIMF can move between ship maintenance projects 

and emergent work in a few minutes compared to other Navy locations 

where transportation time could range up to 4 hours. In addition, the 

consolidated production shops in Hawaii are able within a few minutes to 

pick up and deliver replacement parts for maintenance projects compared 

to other locations, where delivery time could range up to 4 hours once the 

shops are consolidated. Consequently, the longer transportation time 

between facilities at other Navy locations could extend the time frames for 

maintenance projects compared to projects completed in Hawaii. Further, 

as indicated in table 3, Pearl Harbor facilities differ in terms of revenues 

and staffing than in some of the other areas. For example, except for the 

Portsmouth Naval Shipyard, reported revenues and staffing for the other 

two shipyards were more than double the amount of the PHNSY. The 

smaller mission at Pearl Harbor should have made the consolidation easier.

In addition during fiscal year 1997, as indicated in figure 1, more of the 

PHNSY workload was for the fleet compared with the other three naval 

shipyards that completed more work for non-fleet activities. According to 

Navy officials, a single major customer should simplify the integration and 

management of ship maintenance activities. They indicated that it is much 

easier to schedule and budget for maintenance projects on behalf of one 

primary and a few secondary customers compared to meeting the varying 

requirements of several different customers with equal workloads.
B-280524

Page 17 GAO/NSIAD-99-199 Depot Maintenance

Figure 1: Breakdown of the Naval Shipyards' Fiscal Year 1997 Workloads by Major Customer

Source: Our analysis of NAVSEA data.

In the future, Navy officials estimated that nearly 90 percent of the Pearl 

Harbor NSY & IMF workload would be for PACFLT. The remaining 10 

percent will be completed under a reimbursable basis for other activities. 

According to Navy officials, the large share of PACFLT workload in Hawaii 

should reduce the need for the combined maintenance activity to broker 

large workloads between different Navy activities. At the same time, having 

one common financial system should eliminate the need for potentially 

unwieldy financial processes and procedures to manage and budget for 

different work and funding sources. In other locations, NAVSEA and the 

0

10

20

30

40

50

60 Percentage of total

Norfolk Naval Shipyard Puget Sound 

Naval Shipyard

Portsmouth Naval

Shipyard

Pearl Harbor Naval

Shipyard

Pacific Fleet

Atlantic Fleet

Naval Sea Systems Command

Other activities
B-280524

Page 18 GAO/NSIAD-99-199 Depot Maintenance

respective fleet are estimated to share the ship maintenance workload on a 

more equitable basis.

Issues Related to 

Financial and 

Organizational 

Structures Need to Be 

Resolved

The Pearl Harbor Pilot has sharpened the debate over the most appropriate 

financial and organizational structures for such consolidated activities. 

Conflicting views continue to exist within the Departments of Defense and 

the Navy over whether these consolidated activities should operate under 

direct appropriations, the NWCF, or a combination of the two financial 

structures, particularly in terms of maintaining visibility over total 

operational costs. Similarly, little progress has been made toward resolving 

the departments' differences over the most appropriate organizational 

structure to manage these types of activities. Consequently, we believe 

these issues need to be resolved to facilitate effective and efficient 

consolidations.

Issues Remain Unresolved 

Regarding Financial 

Structure

The Navy's proposal recommending direct appropriations for the Pearl 

Harbor Pilot stated that the pilot provided an actual demonstration of using 

direct appropriations for a “major maintenance activity,” which would help 

to determine the ultimate financial structure for the Navy's regionalization 

actions. The proposal further stated that a common financial structure 

needed to be tested by means of the pilot. However, the Navy's test plan 

does not attempt to measure the impact of using a single financial structure 

or the benefit of using direct appropriations instead of the working capital 

fund. At the same time, DOD and Navy officials continue to have different 

views on the potential impact of using direct appropriations for the pilot on 

the productivity and performance of the Pearl Harbor Pilot and future 

regionalization actions. The same is true about the command structure to 

be used for such consolidated operations. Consequently, we believe these 

issues need to be resolved to facilitate effective and efficient 

consolidations.

Navy Selected Direct 

Appropriations

Prior to the consolidation, the Navy recognized that the differences 

between activity management and financial structure inhibited the 

PHNSY's and NIMF's ability to readily share workloads and resources. 

According to Navy officials, the multi-financial and command structures 

required them to use cumbersome, workaround procedures to share 

workloads and workers between the former PHNSY and NIMF. Therefore, 

the shipyard and intermediate maintenance facility did not readily share 

workloads and resources because of the time and cost required to carry-out 
B-280524

Page 19 GAO/NSIAD-99-199 Depot Maintenance

the process. To achieve a fully integrated organization, the Navy decided 

the Pearl Harbor Pilot should use a single financial structure and selected 

direct appropriations instead of the NWCF. This decision was based on 

several factors, but primarily because PACFLT was the larger customer of 

the two organizations and direct appropriations matched the fleet's 

financial structure. Consequently, Navy officials expected fewer financial 

issues under direct appropriations than NWCF because PACFLT could 

integrate the pilot into its existing financial structure without establishing 

another system.

Views on the Financial Structure 

Differ

DOD and Navy officials have different views regarding the financial impact 

of removing the PHNSY from the NWCF. There are several centrally 

managed functions in support of all shipyards for which the costs are 

recovered in the shipyard labor rates under the working capital fund 

structure, which are not normally captured under direct appropriations. 

According to DOD officials, these functions include NAVSEA personnel 

directly supporting shipyards, the Navy's Crane Center that manages all 

crane procurements and repair projects, centralized automated 

information systems, programming support, and other centrally managed 

items. In effect, removing the PHNSY from the fund without addressing the 

contribution (approximately 14 percent) it made to cover centrally funded 

costs for naval shipyards would increase their rates. However, CNO and 

NAVSEA officials believe the PHNSY's share of the overhead cost was not 

large enough to have a significant effect on the other shipyards' rates.

As shown in figure 2, the former PHNSY generated 1.7 percent of the fund's 

$21.9 billion in total revenues during fiscal year 1998 and the other 

shipyards contributed another 10.6 percent, or $2.3 billion.
B-280524

Page 20 GAO/NSIAD-99-199 Depot Maintenance

Figure 2: Comparison of Reported NWCF Revenues by Activity Group for Fiscal 

Year 1998

Note: The “other maintenance” category includes aviation and U.S. Marine Corps depot maintenance 

activities and “other” includes ordnance, information services, and transportation activities.

Percentages do not add to 100 percent due to rounding.

Source: Our analysis of the DOD Inspector General report entitled Inspector General, DOD, Oversight 

of the Naval Audit Service Audit of the Navy Working Capital Fund Financial Statements for FY 1998

(99-099, Mar. 1, 1999).

DOD officials are concerned that the Navy has not adequately addressed a 

potentially more significant issue regarding the impact of removing all 

naval shipyards from the NWCF. They believe the Navy decision to remove 

the shipyards from the fund would cause other Navy activities remaining in 

the fund to absorb a larger share of overhead costs. However, CNO and 

NAVSEA officials believe that the shipyards' share is not large enough to 

have a significant effect on the activities remaining in the fund.

Further, DOD and Navy officials have different views on other financial 

issues:

• Working capital funds are not directly subject to the annual 

appropriation cycle and can continue operations without interruption 

between fiscal years. Consequently, the NWCF provided former 

shipyard operators with significant financial flexibility because they 

were freed from reprogramming limitations and restrictions applicable 

Pearl Harbor Naval Shipyard

Other maintenance

Base support

Other

Other shipyards

Supply management

Research and development

31.7%

29.2%

10.6%

10.1%

8.7%

7.8%

1.7%
B-280524

Page 21 GAO/NSIAD-99-199 Depot Maintenance

to regular appropriations and allowed to incur costs without waiting for 

enactment of an appropriation. Because this financial flexibility is 

considered critical to shipyard operations, DOD officials are concerned 

that the Navy has not addressed the impact of eliminating this flexibility 

on the pilot operations or future consolidations.

• The Congress established working capital funds, among other purposes, 

to provide a more flexible funding mechanism that would allow DOD 

industrial, commercial, and service activities to operate on the same 

basis as the private sector, including the use of standard cost accounting 

practices and widely-accepted management techniques. According to 

DOD officials, applying these practices and techniques to naval 

shipyards has provided cost visibility and improved the efficiency of 

operations. Although we found that the Navy has retained the cost 

visibility of the former shipyard under the pilot, the cost information is 

not routinely reported to the major commands as it was for the NWCF. 

Further, the visibility is maintained by continuing to operate the former 

shipyard accounting and management system, which could be 

eliminated after the pilot is completed. For example, DOD officials are 

concerned that the visibility will not be maintained after the pilot period 

is completed. Navy officials said they plan to maintain cost visibility of 

the maintenance operations in Hawaii even after the pilot period 

because the visibility is useful in the management of those operations. 

Whichever financing method is adopted after the completion of the 

pilot, the Statement of Federal Financial Accounting Standards No. 4 

requires DOD and the Navy to continue to measure and report the full 

cost of such ship maintenance operations by responsibility segment.

• The buyer/seller relationship is a fundamental aspect of working capital 

funds. According to DOD officials, this relationship established dynamic 

interplay between the buyer and the provider that encouraged cost 

control and responsiveness to customer requirements. In addition, it 

provided the opportunity for shipyard customers to focus their attention 

on accomplishing their respective missions and shipyard operators to be 

accountable and responsive to customers' needs. Absent 

reimbursements for most maintenance transactions at Pearl Harbor 

under direct appropriations, CNO and NAVSEA officials said that a 

buyer/seller relationship has largely been eliminated. They estimated 

that only 10 percent of the combined facility's future workload, such as 

ship alterations and inactivations, will continue to be on a reimbursable 

basis after the pilot period.

• According to DOD officials, another important aspect of working capital 

funds is their capital investment program. In the case of the NWCF, the 

former shipyard depreciated its capital assets and collected this 
B-280524

Page 22 GAO/NSIAD-99-199 Depot Maintenance

expense from customers through its rates. Therefore, the fund had a 

ready reserve to finance capital investments for the shipyard. According 

to CNO and NAVSEA officials, NAVSEA will budget for future capital 

improvements for the Pearl Harbor NSY & IMF. However, DOD officials 

are concerned that future funding levels may be insufficient because of 

uncertainties in the DOD budgeting and appropriation process.

Discussions of Other Financial 

Structures Continue

Although other similar regionalization efforts are likely, according to Navy 

officials, they may not look like the Pearl Harbor Pilot with direct 

appropriations as the single financial structure. For example, Navy officials 

are currently discussing using either direct appropriations or the working 

capital fund to finance the combined activity of future consolidations, or a 

combination of the two financial structures.11 One combination being 

discussed involves using direct appropriations to finance the overhead and 

military personnel of the combined facility while using the NWCF to 

finance direct maintenance work and materials. Because they believe that 

the working capital fund is more costly and less flexible than direct 

appropriations, several Navy officials are reluctant to use the fund to 

finance the cost of intermediate maintenance activities, which are 

currently financed with direct appropriations. 

Differing Views Remain 

Regarding Organizational 

Structure

The Navy has made little progress toward resolving questions over the 

appropriate organizational structure for such consolidations. While the 

proposal recommending the pilot's financial structure stated that common 

ownership was believed to be beneficial to the Navy's regional 

maintenance initiative, it needed to be tested by the pilot. However, the 

Pearl Harbor Pilot Test Plan does not attempt to measure the impact of 

providing PACFLT ownership and management responsibilities for the pilot 

and Navy officials are uncertain which command structure should be used 

for similar consolidations at other locations.

Although the pilot was expected to streamline the maintenance process 

and eliminate stovepipe organizations, DOD and Navy officials do not 

know the extent to which the pilot's current structure is meeting these 

objectives or how they might have been met under other command 

structures. Prior to the pilot, the NIMF operated as a PACFLT activity while 

the former shipyard operated as a NAVSEA activity. Under the pilot, 

11 In its comments on a draft of this report, the Navy said that NAVSEA and the Fleet 

Commanders had agreed now to operate future consolidations under direct appropriations.
B-280524

Page 23 GAO/NSIAD-99-199 Depot Maintenance

PACFLT assumed ownership and overall management and financial 

responsibility for the consolidated activity while NAVSEA continued to be 

the technical and operating authority. Although CNO and NAVSEA officials 

viewed the duality of fleet ownership and NAVSEA operation as not adding 

another management layer, according to several DOD and other Navy 

officials, this appeared to add a level of responsibility and oversight in lieu 

of streamlining the former maintenance structure and eliminating 

organizations. For example, several Pearl Harbor NSY & IMF officials said 

that the current ownership and command structure has resulted in the 

practice of reporting and coordinating simultaneously with both 

commands. In addition, there has been some confusion on which command 

was ultimately responsible for collecting test data and analyzing the results 

of the pilot. Navy officials are still undecided about the appropriate 

organizational structure for these types of activities and are considering 

different structures for other regionalization efforts.

Conclusions Although the Pearl Harbor Pilot is not complete, preliminary results have 

been mixed, showing either performance improvements, no improvements, 

or insufficient data to determine results. For most of the measures where 

data are available, indications are that the pilot has the potential to improve 

maintenance activities in Hawaii. However, because of unique aspects of 

ship maintenance activities in Hawaii, the pilot is likely to serve only as a 

general model for future consolidation efforts and provides only a general 

indication that future consolidations will result in similar efficiencies. At 

the same time, DOD and Navy officials continue to have different views on 

the potential impact of using direct appropriations to finance the pilot on 

(1) the Navy shipyards and activities remaining in the working capital fund, 

(2) ship maintenance activities during periods without appropriations,

(3) cost visibility of ship maintenance activities, (4) incentives inherent 

under NWCF's buyer/seller relationship for improving productivity and 

performance, and (5) the capital investment program for ship maintenance 

activities. Other issues that are still unresolved include determining 

whether the pilot's organization structure under PACFLT ownership has 

helped streamline the ship maintenance process and improve operations in 

Hawaii.

Recommendations As the Navy proceeds with other similar consolidations, we recommend 

that the Secretary of Defense require the Secretary of the Navy to resolve 

issues related to the appropriate mechanism to finance and manage these 
B-280524

Page 24 GAO/NSIAD-99-199 Depot Maintenance

types of activities. Specific financial questions that need to be resolved 

include the impact of using the direct appropriations to finance the pilot 

and other potential regionalization actions with regard to (1) the Navy 

shipyards and activities remaining in the working capital fund, (2) ship 

maintenance activities during periods without appropriations, (3) cost 

visibility of ship maintenance activities, (4) incentives inherent under 

NWCF's buyer/seller relationship for improving productivity and 

performance, and (5) the capital investment program for ship maintenance 

activities. In addition, other questions that need to be resolved include 

determining whether the pilot's command structure under the Fleet's 

ownership has helped streamline the ship maintenance process and 

improve operations in Hawaii.

Agency Comments and 

Our Evaluation

On August 5, 1999, we requested comments on a draft of this report from 

the Secretary of Defense. On August 31, 1999, officials of the Office of the 

Assistant Deputy Under Secretary of Defense for Maintenance Policy, 

Programs, and Resources said that they concurred with the intent of our 

recommendations. However, DOD Comptroller officials expressed concern 

that the draft of this report did not adequately address the Navy's 

underlying assertions and assumptions for the pilot or whether the stated 

objectives of the pilot were met. By recommending that the Navy take steps 

to address unresolved issues related to financial and organizational 

structures as it proceeds with similar consolidations, we believe our report 

addresses the most significant issues for such consolidations. Because the 

pilot is not yet complete, we did not attempt to determine whether the 

stated objectives were fully met but, instead, reported the preliminary 

results of the pilot on improving performance and maintenance activities in 

Hawaii. We believe that it could take several years for the full effects of the 

pilot to develop and, as requested, we are reporting the preliminary results.

In addition, Navy officials said that they generally concurred with the draft 

of this report except for our use of fiscal year 1998 data to indicate the 

preliminary results of the Pearl Harbor Pilot, and they advised that fiscal 

year 1999 data would provide the most valid assessment of the pilot. They 

considered fiscal year 1998 as a transition year when the maintenance 

activities, financial systems, and workforce were not fully integrated and, 

consequently, did not indicate the pilot's full performance. As discussed in 

this report, we also considered fiscal year 1998 as a transition year for the 

pilot and reported that the pilot was unlikely to result in significant 

improvements during the transition year because of the operational 

turbulence experienced during the consolidation. However, we used 
B-280524

Page 25 GAO/NSIAD-99-199 Depot Maintenance

available data from both the transition year and fiscal year 1999 to provide 

the preliminary results of the pilot. We continue to believe that, in either 

year, overall indications are that the pilot has the potential to improve 

maintenance activities in Hawaii.

DOD and the Navy also provided technical corrections and clarifications 

and, where appropriate, we incorporated them in the report.

We are sending copies of this report to Senator Pete V. Domenici, Senator 

Daniel K. Inouye, Senator Frank R. Lautenberg, Senator Carl Levin, Senator 

Joseph I. Lieberman, Senator Fred Thompson, Senator Ted Stevens, 

Senator Charles S. Robb, and Senator John W. Warner, and to 

Representative John R. Kasich, Representative Jerry Lewis, Representative 

John P. Murtha, Representative Ike Skelton, Representative Floyd Spence, 

and Representative John M. Spratt, Jr., in their capacities as Chair or 

Ranking Minority Member of cognizant Senate and House Committees and 

Subcommittees. We are also sending copies of this report to the Honorable 

William S. Cohen, Secretary of Defense; the Honorable William J. Lynn, 

Under Secretary of Defense (Comptroller); the Honorable Richard Danzig, 

Secretary of the Navy; and the Honorable Jacob Lew, Director, Office of 

Management and Budget. Copies will also be made available to others upon 

request.

GAO contacts and key contributors to this report are listed in appendix III.

Sincerely yours,

David R. Warren, Director

Defense Management Issues
Page 26 GAO/NSIAD-99-199 Depot Maintenance

Appendix I

Scope and Methodology ApendxIi

During our review, we obtained data and interviewed Departments of 

Defense (DOD) and Navy officials, including those from the offices of the 

Under Secretary of Defense (Comptroller), Deputy Under Secretary of 

Defense for Logistics, DOD Inspector General, Assistant Secretary of the 

Navy for Financial Management and Comptroller, Chief of Naval 

Operations (CNO), Naval Sea Systems Command (NAVSEA), the Pacific 

Fleet (PACFLT), Pearl Harbor Naval Shipyard & Intermediate Maintenance 

Facility (NSY & IMF), San Diego Supervisor of Ship Building Conversion 

and Repair Pearl Harbor Detachment, and the Naval Audit Service (NAS). 

In addition, we interviewed officials of the Hawaii Federal Employees 

Metal Trades Council, the Ship Repair Association of Hawaii, and 

contractors involved in the management and assessment of the Pearl 

Harbor Pilot.

To identify the preliminary results of the Pearl Harbor Pilot, we obtained 

and reviewed status reports and briefings, financial and human resources 

documents, infrastructure data, budget documents, NAS capacity 

evaluation reports and documents, the Pearl Harbor Pilot Test Plan and 

related guidance, and PACFLT regional maintenance and business plans. In 

addition, we interviewed DOD and Navy officials to discuss the preliminary 

results of the pilot and challenges that must be resolved to obtain the full 

benefits of the consolidation. We observed the operations of the Pearl 

Harbor NSY & IMF and interviewed superintendents, first-line supervisors, 

and workers to identify operational conditions and determine the pilot's 

impacts on their maintenance shops and activities. We reviewed the Pearl 

Harbor Pilot Study Report and other planning documents to identify 

maintenance procedures and processes expected to improve under the 

pilot and compared these expectations with actual operational results for 

fiscal year 1998 and the beginning of fiscal year 1999. When these results 

did not meet expectations, we discussed the differences with Navy officials 

to determine the causes and identify expectations for future operations. In 

addition, we interviewed officials of various PACFLT commands 

(customers of the Pearl Harbor NSY & IMF) to determine whether the 

combined facility's maintenance activities and services were comparable to 

those provided prior to the consolidation. Additionally, we interviewed 

Navy officials and collected data on the apprentice program, safety and 

quality assurance programs, and issues related to the morale of Pearl 

Harbor NSY & IMF.

To assess the adequacy of the Pearl Harbor Pilot Test Plan, we reviewed the 

DOD Inspector General's preliminary assessment of the test plan and 

baseline data. We also interviewed DOD, Navy, and contractor officials 
Appendix I

Scope and Methodology

Page 27 GAO/NSIAD-99-199 Depot Maintenance

about the methodology for the test plan, reliability of the data, and 

strengths and weaknesses of each test metric. To obtain an understanding 

of the methodology and data reliability, we reviewed the process and data 

used to develop the fiscal year 1997 baseline, the fiscal year 1998 transition 

results, and the fiscal year 1999 preliminary results and, on a selected basis, 

we traced the data to their original source documents. To identify potential 

areas for improving the test plan, we discussed the strengths and 

weaknesses of the test plan with DOD, Navy, and contractor officials. We 

also discussed several key elements of the pilot that were omitted from the 

test plan, including the reasons for omission or the potential need for 

inclusion.

To identify issues that remain for future consolidations at other Navy 

locations, we obtained and reviewed status reports and briefings, financial 

and human resources documents, the Pearl Harbor Pilot Test Plan and 

related guidance, and PACFLT regional maintenance and business plans. In 

addition, we interviewed DOD and Navy officials to discuss issues that 

must be resolved to obtain the full benefits of the Pearl Harbor 

consolidation and other similar consolidations at other Navy locations. In 

addition, during meetings with DOD and Navy officials, we discussed the 

advantages and disadvantages of whether future consolidations should use 

direct appropriations, the Naval Working Capital Fund (NWCF), or a 

combination of the two financial systems. To help assess the potential 

impact of removing the former Pearl Harbor Naval Shipyard (PHNSY) and 

other naval shipyards from the NWCF, we identified and analyzed the 

percentage of revenues the former PHNSY and the other shipyards 

contributed to the fund during fiscal years 1997 and 1998. We also 

discussed the potential impact of their removal from the fund with DOD 

and Navy officials. Additionally, during meetings with DOD and Navy 

officials, we discussed the advantages and disadvantages of whether 

NAVSEA or the fleets should be responsible for the ownership and 

management of future consolidations. To determine whether the pilot 

could serve as a model for similar consolidations at other locations, we 

identified and analyzed reasons why the Navy selected Pearl Harbor 

maintenance activities for the pilot. We compared these conditions with 

those found at the Puget Sound, Portsmouth, and Norfolk areas and 

discussed the potential impact of these differences on other consolidations 

with DOD and Navy officials.

Furthermore, we documented the reasons the Navy selected direct 

appropriations, and the pilot's experience of operating under direct 

appropriations. We also interviewed DOD and Navy officials to discuss the 
Appendix I

Scope and Methodology

Page 28 GAO/NSIAD-99-199 Depot Maintenance

actual level of cost visibility retained under the pilot and the various levels 

considered adequate for ship maintenance activities. In addition, we 

determined the level of cost visibility retained for four ongoing 

maintenance projects for the U.S.S. Charlotte, U.S.S. Russell, U.S.S. 

Tucson, and U.S.S. Crommelin. Specifically, we reviewed the start date, 

estimated date of completion, estimated cost, direct labor hours of military 

and civilian personnel, overtime hours, material and supply cost, and 

contract services for the projects. In addition, to determine whether the 

same level of detail was being maintained by the Pearl Harbor NSY & IMF 

under direct appropriations, we reviewed material expenditures and laborhours for military and civilian personnel by job order number for the 

projects.

In performing this review, we used the same budget and accounting 

systems, reports, and statistics DOD and the Navy use to manage and 

monitor their ship maintenance program. We did not independently 

determine the reliability of the reported financial information. However, 

our recent audit of the federal government’s financial statements, including 

DOD’s and the Navy’s statements, questioned the reliability of reported 

financial information because not all obligations and expenditures are 

recorded to specific budgetary accounts.

We conducted our review from May 1998 to July 1999 in accordance with 

generally accepted government auditing standards.
Page 29 GAO/NSIAD-99-199 Depot Maintenance

Appendix II

Description of the Pearl Harbor Pilot Test Plan 

and Its Strengths and Weaknesses ApendxIi

The Navy's test plan for assessing the effectiveness of the Pearl Harbor 

Pilot evolved between December 1997, when the Deputy Secretary of 

Defense first required a test plan, and September 1998, when the last three 

test metrics were added to the plan. During this evolution, DOD, Navy, and 

audit agency officials expressed various concerns with the scope and 

validity of the test plan. Ultimately, the Navy selected nine test metrics to 

evaluate the productivity and performance of the pilot.

Metrics to Be Used to 

Evaluate the Pilot

After several revisions and additions, the current test plan contains nine 

metrics that capture unit cost per output, production efficiency, material 

readiness of ships home ported at Pearl Harbor, and customer satisfaction. 

During the initial planning period in late 1997, Navy officials discussed 

several metrics to measure the impact of the pilot and initially agreed upon 

the two shown in table 4.

Table 4: Navy's Initial Two Metrics to Test Whether the Consolidated Pearl Harbor NSY & IMF Increased Manpower Utilization 

and Lowered Unit Cost of Production

Source: Interviews with Navy officials and our analysis of the Pearl Harbor Pilot Test Plan dated

June 30, 1998.

In December 1997, recognizing that the Navy had not developed a plan or 

established a baseline for measuring success or failure for the pilot, the 

Deputy Secretary of Defense required the Navy to develop a test plan to 

measure the benefits of the pilot.1 In addition, the Deputy Secretary of 

Defense suggested that “productive man-days delivered per dollar outlay” 

and “number of civilian end strengths and military on-board it took to 

Metric Data elements Calculation

1. Total cost of a production shop, direct 

labor hour of work delivered to the 

customer

• Total costs of the ship maintenance activity

• Total production shop, direct labor hours delivered

• Total costs of the ship maintenance 

activity or activities divided by the 

total production shop, direct labor 

hours delivered

2. Total available labor hours expended 

to deliver a production shop, direct labor 

hour to the customer

• Total available labor hours, including direct and 

indirect hours, expended by the ship maintenance 

activity

• Total production shop, direct labor hours delivered

• Total available labor hours divided by 

the total production shop, direct labor 

hours delivered

1

 Program Budget Decision Number 404, Deputy Secretary of Defense (Dec. 11, 1997).
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 30 GAO/NSIAD-99-199 Depot Maintenance

produce the productive man-days” should be used as key assessment 

metrics. Subsequently, the Chief of Naval Operations (CNO) and the Naval 

Sea Systems Command (NAVSEA) requested the Naval Audit Service 

(NAS) determine the combined cost of operations for the former Pearl 

Harbor Naval Shipyard (PHNSY) and the Naval Intermediate Maintenance 

Facility (NIMF) for fiscal year 1997.2

 In an April 1998 report, the NAS 

provided a basis to compare the cost effectiveness of combining the two 

maintenance activities and metric calculations for the two assessment 

metrics defined at that time, which are the first two metrics of the current 

test plan.3

 In the report, the NAS recognized that the consolidation was an 

evolving process and revisions to its report may be necessary to reflect 

changes in consolidation decisions. In May 1999, after making changes to 

the baseline data and fine-tuning its methodology, the NAS revised its 

baseline cost and productivity estimates.4

 The revised estimates did not 

change significantly from the initial estimates.

In April 1998, the Navy issued a draft of the Pearl Harbor Pilot Test Plan 

that included five metrics to capture cost per unit output, production 

efficiency and resource utilization, material readiness of PACFLT ships, 

customer satisfaction, and quality. However, in June 1998, the DOD 

Assistant Inspector General for Auditing stated that the draft test plan 

would not provide sufficient or supportable data to draw conclusions about 

the effectiveness of consolidating the operations of the former shipyard 

and intermediate maintenance facility. Consequently in June 1998, a joint 

DOD and Navy team issued a revised Pearl Harbor Pilot Test Plan that 

changed the basis for the quality metric from formal customer surveys to 

an analysis of rework items and added a metric measuring the number of

2

 The Navy selected fiscal year 1997 as the baseline year because this was the last full year 

the former PHNSY and NIMF operated as independent activities. Fiscal year 1998 was 

eliminated because of the operational turbulence expected by the consolidation of activities 

during the year. Fiscal year 1996 was eliminated because there were separate surface ship 

and submarine intermediate maintenance facilities operating during the fiscal year.

3 Capacity Evaluation Report: Baseline Costs of Operating Naval Intermediate Maintenance 

Facility, Pearl Harbor and Pearl Harbor Naval Shipyard, Auditor General of the Navy (98-

0400, Apr. 1998).

4 Capacity Evaluation Report: Revised Baseline Costs of Operations and Metrics for Naval 

Intermediate Maintenance Facility, Pearl Harbor and Pearl Harbor Naval Shipyard 

Consolidation, Auditor General of the Navy (99-0408, May 1999).
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 31 GAO/NSIAD-99-199 Depot Maintenance

work-items completed.5 Consequently, the Navy included four metrics in 

addition to those two metrics listed in the previous table to its final test 

plan. (See table 5.)

Table 5: Metrics Three Through Six to Test Whether the Consolidated Pearl Harbor NSY & IMF Increased Manpower Utilization 

and Lowered Unit Cost of Production

Source: Interviews with Navy officials and our analysis of the Pearl Harbor Pilot Test Plan dated

June 30, 1998.

In September 1998, the Principal Deputy Under Secretary of Defense 

(Comptroller) suggested the Navy incorporate three additional test metrics 

developed under contract by the Anteon Corporation into the revised June 

1998 test plan.6

 (See table 6.) In response, the Acting Deputy Chief of Naval 

5

 The Pearl Harbor NSY & IMF has developed three additional internal metrics:

production-shop hours versus total available hours, direct production worker versus nonproduction worker, and civilian labor cost per labor hour. While they are not formally part of 

the test plan, Pearl Harbor NSY & IMF officials believe the metrics will assist them in 

assessing the effectiveness of the pilot.

Metric Data elements Calculation

3. Total Current Ship Maintenance 

Program (CSMP) work items completed

• Total number of CSMP work items completed on 

ships home ported at Pearl Harbor during the fiscal 

year

• None required, absolute number of 

completed CSMP work items used

4. Total Current Ship Maintenance 

Program work items in the backlog

• Total number of CSMP work items in the backlog for 

ships home ported at Pearl Harbor 

• None required, absolute number of 

CSMP work items in the backlog used

5. Schedule adherence of CNO 

maintenance projects

• Sum of the differences in the actual and scheduled 

completion dates for each CNO ship maintenance 

project completed during the fiscal year

• Sum of the scheduled duration (number of days) for 

each CNO maintenance project completed during 

the fiscal year

• Sum of the differences in the actual 

and scheduled completion dates 

divided by the total scheduled 

duration (number of days) for each 

CNO scheduled ship maintenance 

project completed during the fiscal 

year

6. Rework index for CNO maintenance 

projects

• Sum of the labor hours expended to correct work 

deficiencies for each CNO scheduled ship 

maintenance project completed during the fiscal 

year

• Total production shop, direct labor hours delivered 

for each CNO scheduled ship maintenance project 

completed during the fiscal year

• Sum of the labor hours expended to 

correct work deficiencies divided by 

the total number of direct labor hours 

delivered for each CNO scheduled 

ship maintenance project completed 

during the fiscal year

6 The Pearl Harbor Pilot Study, Anteon Corporation (Sept. 10, 1998).
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 32 GAO/NSIAD-99-199 Depot Maintenance

Operations (Logistics) attached them in an appendix E to the plan. At this 

time, Principal Deputy Under Secretary of Defense approved the test plan.

Table 6: Metrics Seven Through Nine to Test Whether the Consolidated Pearl Harbor NSY & IMF Increased Manpower Utilization 

and Lowered Unit Cost of Production

a

Reports filed by the ship of equipment failures that may indicate faulty work performed during more 

recent maintenance projects.

Source: Interviews with Navy and Anteon Corporation officials and our analysis of the Pearl Harbor 

Pilot Test Plan dated June 30, 1998.

In November 1998, the Director, Industrial Capability, Maintenance Policy 

and Acquisition Logistics Division, CNO, requested the NAS to assume the 

overall lead and control in the test plan assessment process. The Director 

requested that NAS involvement be expanded to include data collection for 

all of the test plan metrics, metric calculation and assessment, and pilot 

metric assessment reporting. In response, the auditors agreed to continue 

to develop and report data for the first two metrics in the test plan and 

verify the data for metrics number 3 through 6. Additionally, NAS 

considered the responsibility for data collection and reporting the results 

of the pilot to be a management function. It did not assume responsibility 

for collecting and validating the data for the contractor-generated metrics.

Baseline, Performance 

Expectations, and 

Preliminary Values for 

the Pearl Harbor Pilot

Given the time and data constraints in developing the Pearl Harbor Pilot 

Test Plan, the test plan and assessment metrics provide the Navy a 

mechanism for evaluating the pilot's productivity and performance. As 

shown in table 7, the Navy has developed the values for the fiscal year 1997 

baseline and performance expectations for six of the nine metrics and 

Metric Data elements Calculation

7. Activity work schedule 

integrity index

• Budgeted quantity of work scheduled (labor days)

• Actual quantity of work performed (labor days)

• Budgeted allowance of work scheduled (labor 

days) divided by the actual amount of work 

performed (labor days)

8. Casualty reportsa

 caused by 

activity work

• Casualty reports • Analysis of reports occurring within 6 months 

following the completion of maintenance 

9. Earned value • Actual quantity of work performed (labor days) for 

selected Ship Work Line Item Numbers for the fiscal 

year

• Statistical analysis of actual quantity of work 

performed (labor days) for selected Ship Work 

Line Item Numbers for fiscal years 1997 and 

1999
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 33 GAO/NSIAD-99-199 Depot Maintenance

preliminary values for five metrics. However, the details for collecting and 

analyzing the contractor-generated metrics have not yet been determined.

Table 7: Baseline, Performance Expectations, and Preliminary Results of the Test Plan

a

Reported by the NAS.

b

Adjusted for increases in military and civilian pay rates due to inflation.

c

Reported by the Pearl Harbor NSY & IMF, but not yet verified by the NAS.

Source: Interviews with Navy and Anteon Corporation officials and our analysis of the Pearl Harbor 

Pilot Test Plan dated June 30, 1998; NAS capacity evaluation reports for fiscal years 1997 and 1998; 

and pilot status reports.

Although the Navy has established nine metrics that provide indicators of 

productivity and performance, the metrics have data limitations and 

weaknesses that preclude their use for precise measurements. 

Nevertheless, they provide a basis for an order of magnitude comparison. A 

discussion of the nine metrics follows.

Metric Fiscal year 1997 baseline Expectation Preliminary result

1. Total cost of a production shop, 

direct labor hour of work delivered 

to the customera

• $135.36 in current year 

dollars

• $138.13 in adjusted dollarsb

• 3-5 percent reduction • $135.59 per hour in fiscal year 1998

2. Total available labor hours 

expended to deliver a production 

shop, direct labor hour to the 

customera

• 3.15 hours • 3-5 percent reduction • 3.04 hours in fiscal year 1998

3. Total Current Ship Maintenance 

Program work items completedc

• 19,777 • Maintain at least the 

same completion rate

• 14,568 items in fiscal year 1998

• 7,786 items as of May 1999 

4. Total Current Ship Maintenance 

Program work items in the backlogc

• 17,733 • 10 percent reduction • 16,462 items as of September 30, 1998

• 16,538 items as of May 1999

5. Schedule adherence of CNO 

maintenance projectsc

• 11.4 percent late • To improve • 18.6 percent late in fiscal year 1998

• 3.7 percent early in fiscal year 1999 

through May 1999

6. Rework index for CNO 

maintenance projectsc

• 0.086 • No degradation of the 

index

• Not yet determined

7. Activity work schedule integrity 

index

• Not yet determined • Unclear • Not yet determined

8. Casualty reports caused by 

activity work

• Not yet determined • Unclear • Not yet determined

9. Earned value • Varies by work item • Unclear • Not yet determined
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 34 GAO/NSIAD-99-199 Depot Maintenance

Cost of a Production Shop, 

Direct Labor Hour of Work 

Delivered

Measuring the pilot's efficiency in terms of the cost per direct maintenance 

hour, the Navy's first metric is determined by dividing the total activity cost 

by the number of production shop, direct labor hours delivered to the 

customer.7 The NAS estimated that it cost $135.36 to deliver one production 

shop, direct labor hour in fiscal year 1997—the baseline year for measuring 

the pilot's success. For the pilot to show improvement, this index should 

decrease, indicating that since the consolidation, it cost less to deliver a 

direct maintenance hour. The NAS estimated that the index was $135.59 

during the fiscal year 1998 transition, indicating some improvement when 

compared with the adjusted baseline of $138.13, but falling short of Navy's 

performance expectation of 3- to 5-percent reduction in the index.8

 The 

NAS adjusted the baseline to account for differences in military and civilian 

wages and inflation between fiscal years 1997 and 1998. Because of the 

operational turbulence experienced during the consolidation, we believe 

this indicator was unlikely to show a significant improvement during the 

transition year.

Strengths and Weaknesses In analyzing this metric, we noted strengths and weaknesses. The strengths 

of this metric are that the baseline is partly based on shipyard data, which 

are considered reliable, and the NAS developed the baseline estimates and 

plans to validate and generate the test data. However, DOD and Navy 

officials have expressed concern with the accuracy of the intermediate 

maintenance data collected during the baseline year, which is considered a 

weakness. For example, military personnel costs were not collected 

because intermediate maintenance facilities traditionally did not collect 

these costs. In addition, some production hours appeared to be inaccurate. 

For example, a DOD contractor reported the former NIMF database 

contained zero labor hours for approximately 6 percent of the maintenance 

tasks completed by the facility. Consequently, to strengthen the metric, the 

NAS developed a methodology that used Navy personnel wages and 

standards and the best available NIMF data to estimate a baseline for this 

metric. Although the NIMF baseline data on military personnel costs and 

7

 The production shops consist of the crane, ship fitter, sheet metal, welding, inside 

machine, outside machine, boilermaker, electrical, pipe fitter, wood/fabric, cryptographic 

shore electronics, electronics, shipwright/rigging, and temporary services shops and the 

Navy calibration and nuclear regional maintenance departments.

8 Fiscal Year 1998 Interim Costs for the Consolidated Pearl Harbor Naval Shipyard and 

Intermediate Maintenance Facility, Pearl Harbor, Auditor General of the Navy (99-0400,

June 1, 1999).
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 35 GAO/NSIAD-99-199 Depot Maintenance

production hours have limitations, they are considered reliable enough by 

Navy officials to provide a rough, if imprecise, estimate of the baseline.

Labor Hours Expended to 

Deliver a Production Shop, 

Direct Labor Hour

Measuring the pilot's productivity in terms of personnel utilization, the 

second metric is determined by dividing the total number of available labor 

hours by the number of production shop, direct labor hours delivered to 

the customers. The NAS estimated that it took 3.15 activity labor hours 

(overhead and direct maintenance hours) to deliver one production shop, 

direct labor hour in fiscal year 1997. For the pilot to show improvement, 

this index should decrease indicating that it has taken fewer overhead 

hours to generate 1 direct maintenance hour since the consolidation. The 

NAS reported that the index decreased to 3.04 hours in fiscal year 1998, 

indicating a 3.5-percent improvement during the transition year. This met 

the low end of the Navy's performance expectation of 3- to 5-percent 

reduction in value.

Strengths and Weaknesses In our analysis, we noted the strengths and weaknesses of this metric. The 

strengths of this metric are that the baseline is partly based on shipyard 

data, which are considered reliable, and the NAS developed the baseline 

estimates and plans to validate and generate the test data. However, as 

discussed previously, DOD and Navy officials have expressed concern with 

the intermediate maintenance data collected during the baseline year. 

Consequently, to strengthen the metric, the NAS developed a methodology 

that used best available NIMF data to estimate a baseline for this metric. 

Although the NIMF baseline data on production hours have weaknesses, 

they are considered reliable enough by Navy officials to provide a rough, if 

imprecise, estimate of the baseline.

Current Ship Maintenance 

Program Work Items 

Completed

Measuring the pilot's productivity in terms of output, the third metric is the 

number of Current Ship Maintenance Program (CSMP) work items 

completed by the Pearl Harbor NSY & IMF. Although this metric is not an 

indicator of the former shipyard's productivity, Navy officials have 

considered the number of CSMP work items completed by their 

intermediate maintenance facilities as a measurement of their 

productivity.9

 The CSMP is the central database for the maintenance 

history of each ship and contains corrective, preventative, and alteration 

9

 Traditionally, naval shipyards completed a minimal number of CSMP items.
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 36 GAO/NSIAD-99-199 Depot Maintenance

maintenance items requiring work. The Pearl Harbor NSY & IMF estimated 

that the former intermediate maintenance facility completed 19,777 CSMP 

work items in fiscal year 1997. However, it further estimated that the 

consolidated activity completed 14,568 CSMP work items during the fiscal 

year 1998 transition and 7,786 work items as of May in fiscal year 1999, 

indicating no improvement in performance since the consolidation. 

Although the combined facility was expected to maintain the same 

completion rate as the former NIMF, several Navy officials believe that this 

expectation was unreasonable because the number of military enlisted 

personnel decreased from 1,275 in October 1996 to 616 in April 1999. 

Traditionally, enlisted personnel completed the majority of CSMP work 

items. Consequently, Navy officials have discussed lowering the 

performance expectation for this metric.

Strengths and Weaknesses During our analysis, we found that this metric had both strengths and 

weaknesses. The strengths of this metric are that the fleet and ship 

commanders maintain the baseline and test data, which are considered 

reliable, and the NAS plans to validate the baseline and pilot results. 

However, other factors not related to the pilot affect the metric that 

weaken its usefulness. For example, Navy officials have expressed concern 

with using CSMP work items to measure the benefits of the Pearl Harbor 

Pilot because one work item does not equal another item in terms of the 

labor hours, work skills, time frame, and material cost required. CSMP 

items range from replacing a light bulb in a control panel, which may take a 

worker less than an hour to replace, to overhauling a pump, which may 

take several workers days to complete. In addition, the requirements 

needed to overhaul one pump differ from other overhauls depending on the 

problem and the type of pump. Although this metric has weaknesses that 

preclude its use for precise measurements, Navy officials believe that it 

provides a basis for an order-of-magnitude comparison.

Current Ship Maintenance 

Program Work Items in the 

Backlog

Historically, Navy officials have measured the material condition of their 

ships based on the number of backlogged CSMP items. Fewer backlogged 

items imply that the ships are in better condition. The Navy's performance 

expectation for the combined facility was to reduce the estimated

17,733-baseline backlog by 10 percent to 15,960 work items. Recently, the 

Pearl Harbor NSY & IMF estimated that the backlog decreased to 16,538 

items as of May 1999. Although this decrease indicates improvement, it falls 

short of the Navy's expectation for the pilot. However, it was unlikely that 

the CSMP backlog would decrease significantly during this period because 
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 37 GAO/NSIAD-99-199 Depot Maintenance

of the reduction in the number of military enlisted personnel since October 

1996.

Strengths and Weaknesses As discussed previously, using CSMPs as a test metric had both strengths 

and weaknesses. The strengths of this metric are that it traditionally has 

been considered an indicator of the material condition of fleet ships and 

that the fleet and ship commanders maintain the baseline and test data, 

which are considered reliable. In addition, the NAS plans to validate the 

baseline and pilot results. For many of the same reasons that CSMP work 

items are not a precise measurement of output, the backlog is not a precise 

measurement of the ship's material condition, which makes the backlog a 

weakness in regard to measurement. For example, one work item related 

to the safety of a nuclear submarine could potentially restrict the 

submarine operations, while a similar item on a surface ship could have 

little or no effect on its operations. Additionally, other factors affecting the 

backlog include non-pilot related items such as

• decommissions of PACFLT ships homeported at Pearl Harbor, which 

decrease the backlog by the number of CSMP items recorded for the 

decommissioned ship;

• visits by the U.S.S. McKee (a maintenance tender), which decrease the 

backlog by the number of CSMP items completed by the tender;

• maintenance inspections, which increase the backlog by the number of 

unrecorded CSMP items identified by the inspection team; and

• procedural changes in identifying and recording CSMP items, which 

may increase or decrease the backlog depending on whether the 

changes weaken or strengthen the process.

Although this metric has weaknesses that preclude its use for precise 

measurements, Navy officials believe that it provides a basis for an orderof-magnitude comparison.

Schedule Adherence of CNO 

Maintenance Projects

According to Navy officials, adherence to completing CNO maintenance 

projects on schedule should provide an indication of customer satisfaction 

because, from the fleet and ship commanders' perspective, these 

maintenance projects should be completed on schedule.10 The performance 

10 CNO maintenance projects include depot-level maintenance that require skills or facilities 

beyond those of the ship and intermediate maintenance activities and are completed by 

public and private shipyards.
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 38 GAO/NSIAD-99-199 Depot Maintenance

expectation during the pilot test was to reduce the 11.4 percent schedule 

adherence index achieved during fiscal year 1997, since a lower percentile 

would indicate that projects were being completed closer to their 

scheduled completion dates.11 According to the Pearl Harbor NSY & IMF, 

the schedule adherence index increased to 18.6 percent during fiscal year 

1998—indicating a worsening performance during the transition year. 

However, Pearl Harbor NSY & IMF data showed that the schedule 

adherence index for the CNO maintenance projects completed in fiscal 

year 1999 has exceeded the Navy's performance expectations for the pilot. 

For example, although the first CNO project completed in fiscal year 1999 

was 20 percent late, the four subsequent CNO projects have been either on 

time or early, decreasing the cumulative index to 3.7 percent early.

Strengths and Weaknesses The strengths of this metric are that it traditionally has been considered an 

indicator of the shipyard customers' satisfaction and that NAVSEA 

maintains the baseline and test data, which are considered reliable. In 

addition, the NAS plans to validate the baseline and pilot results. The 

weaknesses include that the metric is based solely on the former shipyard's 

performance in meeting the completion dates for CNO maintenance 

projects and, consequently, does not include all types of maintenance 

projects. Pearl Harbor NSY & IMF officials believe this metric should 

include fleet maintenance projects because there are more fleet 

maintenance projects than CNO projects, thus affecting more ship 

commanders. However, according to CNO and NAVSEA officials, they 

included only CNO projects for this metric because the schedule adherence 

data for fleet maintenance projects were considered of minimal value. For 

example, the former intermediate maintenance process was to defer work 

rather than to extend the duration of fleet maintenance projects and, 

consequently, these projects were generally completed on time. Because 

the pilot has not changed this process for fleet maintenance projects, CNO 

and NAVSEA officials believe an analysis of schedule adherence data for 

these projects would be of minimal value in assessing the pilot's 

effectiveness.

11 The schedule adherence index is the sum of the differences in the actual and scheduled 

completion dates divided by the total scheduled duration (number of days) for each CNO 

ship maintenance project completed during the fiscal year.
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 39 GAO/NSIAD-99-199 Depot Maintenance

Rework Index for CNO 

Maintenance Projects

In analyzing this metric, we found strengths and weaknesses. Measuring 

the quality of work completed by the Pearl Harbor NSY & IMF, the sixth 

metric is determined by dividing the total number of direct rework hours 

required to correct work deficiencies by the number of direct labor hours 

delivered for CNO maintenance projects completed during the fiscal year. 

Rework hours are the hours spent to correct work deficiencies after the 

maintenance project is completed.

Strengths and Weaknesses The strengths of the metric are that it traditionally has been considered an 

indicator of the shipyard quality of work and the NAS plans to validate the 

baseline and test data. However, its weaknesses include that the baseline 

for this metric is limited to the shipyard's CNO projects because the former 

intermediate maintenance facility did not collect rework data for its 

projects. In addition, the metric has not been computed for fiscal years 

1998 or 1999 because no one has assumed responsibility for tracking this 

information since the inception of the pilot.

Activity Work Schedule 

Integrity Index

Measuring the pilot's degree of schedule integrity, the seventh metric is 

calculated by dividing the budget quantity of work (direct labor hours) 

scheduled by the actual amount of work (direct labor hours) delivered to 

complete the maintenance project. According to the test plan, actual work 

exceeding budgeted work indicates inefficiencies in the facility's planning 

or production processes. A metric value of more than one indicates the 

maintenance project was completed within budget.

Strengths and Weaknesses The strengths of this metric are that it traditionally has been considered an 

indicator of the shipyard customers' satisfaction and the shipyard 

maintains the baseline and test data, which are considered reliable. 

However, one weakness of metric is that it does not include data from the 

former intermediate maintenance facility. In addition, Navy officials believe 

this metric is similar to the fifth metric measuring schedule adherence and, 

therefore, is not significantly useful in determining the benefits of the pilot. 

Consequently, they have not expended resources to compute the metric 

value for the baseline year or established performance expectations for this 

metric. Furthermore, the details of collecting and analyzing the data have 

not yet been determined. According to CNO and NAVSEA officials, the 

Navy does not plan to gather the data needed to develop and analyze this 

metric unless the results of the first six metrics are inconclusive. For 

example, if the pilot results for those six metrics are mixed, then it may be 

necessary to measure other pilot results, such as this metric.
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 40 GAO/NSIAD-99-199 Depot Maintenance

Casualty Reports Caused by 

Activity Work

Measuring the quality of work completed by the Pearl Harbor NSY & IMF, 

the eighth metric is based on analysis of casualty reports. These reports are 

filed by the ship after any equipment failure and may help identify work 

improperly performed by the Pearl Harbor NSY & IMF. Those reports 

indicating improperly performed work may offer insight to potential 

corrective actions to the facility's methods and procedures.

Strengths and Weaknesses We noted strengths and weaknesses in our analysis of this metric. The 

primary strength of the metric is that the fleet and ship commanders 

maintain the baseline and test data, which are considered reliable. 

However, considered a weakness, the metric does not include data from 

the former intermediate maintenance facility. In addition, Navy officials 

believe this metric is similar to the sixth metric measuring rework items 

and, therefore, is not significantly useful in identifying the benefits of the 

pilot. Consequently, they have not expended resources to determine the 

details of collecting and analyzing the data for this metric. Similar to the 

previous metric, the Navy does not plan to gather the data needed to 

develop and analyze this metric unless the results of the first six metrics 

are inconclusive.

Earned Value For the earned value metric, the process determines the value of a 

completed task by comparing actual labor hours it took to complete a unit 

of ship maintenance work in fiscal year 1997 with actual labor hours it took 

to complete the same unit of work in fiscal year 1999 and subsequent test 

years. This metric attempts to measure the pilot's effect on maintenance 

outputs. Based on an analysis of the former shipyard and NIMF production 

and labor data for fiscal year 1997, the Anteon Corporation identified

14 surface ship and submarine systems that were considered cost drivers, 

consuming the most labor hours during the fiscal year.12 On the basis of the 

distribution of labor hours for each of these cost drivers, the Anteon 

Corporation determined that, while the distributions are nearly identical in 

form, they were not distributed normally at the 99.9 percent confidence 

level. The most common statistics used to describe data normally 

distributed are the mean and standard deviation. In this case, however, the 

mean and standard deviations do not adequately represent the data 

distribution. Consequently, non-parametric statistics−which are generally 

12 The Anteon Corporation defined a cost driver as a quantity of ship maintenance work that 

represented a major or controlling portion of work on a particular ship maintenance project.
Appendix II

Description of the Pearl Harbor Pilot Test 

Plan and Its Strengths and Weaknesses

Page 41 GAO/NSIAD-99-199 Depot Maintenance

based on the median (as opposed to the mean)−and percentile ranges (as 

opposed to the standard deviation) of the distribution of labor hours will be 

used to describe the labor hour distribution for each of the cost drivers.

If the distribution of labor hours for the 14 cost drivers in fiscal year 1999 is 

less than the fiscal year 1997 baseline, the earned value metric indicates 

that the productivity of the work performed at the combined facility has 

improved compared to when the facilities operated separately. Conversely, 

if distribution of labor hours in fiscal year 1999 is greater than the fiscal 

year 1997 baseline, the productivity of the combined facility has decreased 

compared to when the facilities operated separately. Although it is 

expected that the cost drivers from fiscal year 1997 will be similar to those 

in fiscal year 1999, a statistical method will be used to verify that the data 

for the different fiscal years are similar. If the data are not statistically 

similar, additional analysis of data will be conducted to develop the earned 

value metric.

Strengths and Weaknesses A strength of this metric is that it attempts to measure the pilot's effect on 

maintenance outputs. However, the primary weakness of the metric, 

according to Navy officials, is that it affects such a small population that 

the metric is not significantly useful in measuring the efficiency of the pilot. 

Another stated weakness is that the metric requires significant resources to 

develop the results. Consequently, Navy officials have not expended 

resources to determine the details of collecting and analyzing the data for 

the metric. Similar to the previous two metrics, the Navy does not plan to 

gather the data needed to develop and analyze this metric unless the results 

of the first six metrics are inconclusive.
Page 42 GAO/NSIAD-99-199 Depot Maintenance

Appendix III

GAO Contacts and Staff Acknowledgments ApendxIi

GAO Contacts David Warren (202) 512-8412

Julia Denman (202) 512-4290

Acknowledgments In addition to those named above, Mark Little, 

Dennis De Hart, Jean Orland, Barbara Wooten, and 

John Brosnan made key contributions to this report.
Page 43 GAO/NSIAD-99-199 Depot Maintenance

Related GAO Products

Navy Ship Maintenance: Allocation of Ship Maintenance Work in the 

Norfolk, Virginia, Area (GAO/NSIAD-99-54, Feb. 24, 1999).

Defense Depot Maintenance: Public and Private Sector Workload 

Distribution Reporting Can Be Further Improved (GAO/NSIAD-98-175,

July 23, 1998).

Defense Depot Maintenance: Contracting Approaches Should Address 

Workload Characteristics (GAO/NSIAD-98-130, June 15, 1998).

Defense Depot Maintenance: Use of Public-Private Partnering 

Arrangements (GAO/NSIAD-98-91, May 7, 1998).

Navy Ship Maintenance: Temporary Duty Assignments of Temporarily 

Excess Shipyard Personnel Are Reasonable (GAO/NSIAD-98-93, Apr. 21, 

1998).

Public-Private Competitions: DOD's Additional Support for Combining 

Depot Workloads Contains Weaknesses (GAO/NSIAD-98-143, Apr. 17, 

1998).

Defense Depot Maintenance: DOD Shifting More Workload for New 

Weapon Systems to the Private Sector (GAO/NSIAD-98-8, Mar. 31, 1998).

Public-Private Competitions: DOD's Determination to Combine Depot 

Workloads Is Not Adequately Supported (GAO/NSIAD-98-76, Jan. 20, 1998).

Defense Depot Maintenance: Information on Public and Private Sector 

Workload Allocations (GAO/NSIAD-98-41, Jan. 20, 1998).

Navy Regional Maintenance: Substantial Opportunities Exist to Build on 

Infrastructure Streamlining Progress (GAO/NSIAD-98-4, Nov. 13, 1997).

Navy Depot Maintenance: Privatizing Louisville Operations in Place Is Not 

Cost-Effective (GAO/NSIAD-97-52, July 31, 1997).

Defense Depot Maintenance: Challenges Facing DOD in Managing Working 

Capital Funds (GAO/T-NSIAD/AIMD-97-152, May 7, 1997).

Defense Depot Maintenance: Uncertainties and Challenges DOD Faces in 

Restructuring Its Depot Maintenance Program (GAO/T-NSIAD-97-112,

May 1,1997) and (GAO/T-NSIAD-97-111, Mar. 18, 1997). 
Related GAO Products

Page 44 GAO/NSIAD-99-199 Depot Maintenance

Defense Outsourcing: Challenges Facing DOD as It Attempts to Save 

Billions in Infrastructure Costs (GAO/T-NSIAD-97-110, Mar. 12, 1997).

High-Risk Series: Defense Infrastructure (GAO/HR-97-7, Feb. 1997).

(709349) Leter
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the 

following address, accompanied by a check or money order made 

out to the Superintendent of Documents, when necessary, VISA and 

MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are 

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and 

testimony. To receive facsimile copies of the daily list or any list 

from the past 30 days, please call (202) 512-6000 using a touchtone 

phone. A recorded menu will provide information on how to obtain 

these lists.

For information on how to access GAO reports on the INTERNET, 

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at: 

http://www.gao.gov
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00