a

GAO

United States Government Accountability Office

Report to Congressional Requesters

September 2005 DOD BUSINESS 

SYSTEMS 

MODERNIZATION

Navy ERP Adherence 

to Best Business 

Practices Critical to 

Avoid Past Failures

GAO-05-858
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

www.gao.gov/cgi-bin/getrpt?GAO-05-858.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Gregory Kutz at 

(202) 512-9505 or Keith Rhodes at (202) 512-

6412. 

Highlights of GAO-05-858, a report to 

congressional requesters 

September 2005

DOD BUSINESS SYSTEMS 

MODERNIZATION 

Navy ERP Adherence to Best Business 

Practices Critical to Avoid Past Failures 

The Navy invested approximately $1 billion in four ERP pilots without 

marked improvement in its day-to-day operations. The planning for the 

pilots started in 1998, with implementation beginning in fiscal year 2000. 

The four pilots were limited in scope and were not intended to be corporate 

solutions for any of the Navy’s long-standing financial and business 

management problems. Furthermore, because of the various inconsistencies 

in the design and implementation of the pilots, they were not interoperable, 

even though they performed many of the same business functions. In short, 

the efforts were failures and $1 billion was largely wasted. 

Because the pilots would not meet its overall requirements, the Navy 

decided to start over and develop a new ERP system, under the leadership of 

a central program office. Using the lessons learned from the pilots, the 

current Navy ERP program office has so far been committed to the 

disciplined processes necessary to manage this effort. GAO found that, 

unlike other systems projects it has reviewed at DOD and other agencies, 

Navy ERP management is following an effective process for identifying and 

documenting requirements. The strong emphasis on requirements 

management, which was lacking in the previous efforts, is critical since 

requirements represent the essential blueprint that system developers and 

program managers use to design, develop, test, and implement a system and 

are key factors in projects that are considered successful. 

While the Navy ERP has the potential to address some of the Navy’s financial 

management weaknesses, as currently planned, it will not provide an allinclusive end-to-end corporate solution for the Navy. For example, the 

current scope of the ERP does not include the activities of the aviation and 

shipyard depots. Further, there are still significant challenges and risks 

ahead as the project moves forward, such as developing and implementing 

44 system interfaces with other Navy and DOD systems and converting data 

from legacy systems into the ERP system. The project is in its early phases, 

with a current estimated completion date of 2011 at an estimated cost of 

$800 million. These estimates are subject to, and very likely will, change. 

Broader challenges, such as alignment with DOD’s business enterprise 

architecture, which is not fully defined, also present a significant risk. Given 

DOD’s past inability to implement business systems that provide the 

promised capability, continued close management oversight—by the Navy 

and DOD—will be critical. In this regard, the Navy does not have in place 

the structure to capture quantitative data that can be used to assess the 

effectiveness of the overall effort. Also, the Navy has not established an 

IV&V function. Rather, the Navy is using in-house subject matter experts 

and others within the project. Industry best practices indicate that the IV&V 

activity should be independent of the project and report directly to agency 

management in order to provide added assurance that reported results on 

the project’s status are unbiased. 

The Department of Defense’s 

(DOD) difficulty in implementing 

business systems that are efficient 

and effective continues despite the 

billions of dollars that it invests 

each year. For a decade now—

since 1995—we have designated 

DOD’s business systems 

modernization as “high-risk.” GAO 

was asked to (1) provide a 

historical perspective on the 

planning and costs of the Navy’s 

four Enterprise Resource Planning 

(ERP) pilot projects, and the 

decision to merge them into one 

program; (2) determine if the Navy 

has identified lessons from the 

pilots, how the lessons are being 

used, and challenges that remain; 

and (3) determine if there are 

additional best business practices 

that could be used to improve 

management oversight of the Navy 

ERP. 

What GAO Recommends

GAO makes three 

recommendations to DOD: (1) 

develop and implement the 

quantitative metrics needed to 

evaluate project performance and 

risks and use the metrics to assess 

progress and compliance with 

disciplined processes, (2) establish 

an independent verification and 

validation (IV&V) function and 

direct that all IV&V reports be 

provided to Navy management and 

the appropriate DOD investment 

review board, and (3) institute 

semiannual reviews of the 

program. DOD generally agreed 

with our recommendations. 
Page i GAO-05-858 Navy Enterprise Resource Planning

Contents

Letter 1

Results in Brief 3

Background 7

Navy’s Pilot Projects Lacked Coordinated Management Oversight 12

Requirements Management Process Effective, but Implementation 

Challenges and Risks Remain 23

Additional Actions Can be Taken to Improve Management Oversight 

of the Navy ERP Effort 41

Conclusions 47

Recommendations for Executive Action 48

Agency Comments and Our Evaluation 48

Appendixes

Appendix I: Scope and Methodology 54

Appendix II: Comments from the Department of Defense 56

Appendix III: Identification of the Navy and Defense Systems That Must 

Interface with the ERP 60

Appendix IV: GAO Contacts and Acknowledgments 62

Tables Table 1: Navy ERP Pilot Projects 13

Table 2: Functions Performed by the Pilot Projects 15

Table 3: Documentation for Detailed Requirements 30

Figures Figure 1: Percent of Effort Associated with Undisciplined 

Projects 22

Figure 2: Relationship between Requirements Development and 

Testing 27

Figure 3: Navy ERP Required System Interfaces 35

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately.
Page 1 GAO-05-858 Navy Enterprise Resource Planning

United States Government Accountability Office

Washington, D.C. 20548

A

September 29, 2005 Letr

Congressional Requesters

The Department of Defense (DOD) has historically been unable to develop 

and implement business systems on time, within budget, and with the 

promised capability. As noted in our recent report,1

 this difficulty 

continues despite the billions of dollars that DOD spends each year to 

operate, maintain, and modernize its currently reported 4,150 business 

systems. For fiscal year 2005, the department requested $13 billion for its 

existing business systems environment. For a decade now—since 1995—

we have designated DOD’s financial management and business systems 

modernization as “high-risk.” In fact, of the 25 areas on GAO’s 

governmentwide “high-risk” list, 8 are DOD specific program areas, and the 

department shares responsibility for 6 other high-risk areas that are 

governmentwide in scope.2

 

DOD has recognized the importance of transforming its business 

operations and systems to make them more efficient and effective in 

support of the department’s defense mission. A critical aspect of the 

department’s transformation effort will be its ability to effectively 

implement business systems on time, within budget, and with the promised 

capability.

This report responds to your request for information on DOD’s 

management of selected facets of its business modernization efforts that 

are intended to enhance the department’s reporting on its results of 

operation. As agreed with your offices, we selected the Department of the 

1

GAO, DOD Business Systems Modernization: Billions Being Invested without Adequate 

Oversight, GAO-05-381 (Washington, D.C.: Apr. 29, 2005). 

2

GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.: January 2005). The 

eight specific DOD high-risk areas are (1) approach to business transformation, (2) business 

systems modernization, (3) contract management, (4) financial management, (5) personnel 

security clearance program, (6) supply chain management, (7) support infrastructure 

management, and (8) weapon systems acquisition. The six governmentwide high-risk areas 

that include DOD are: (1) disability programs, (2) interagency contracting, (3) information 

systems and critical infrastructure, (4) information sharing for homeland security, (5) 

human capital, and (6) real property.
Page 2 GAO-05-858 Navy Enterprise Resource Planning

Navy’s (Navy) Enterprise Resource Planning (ERP) program,3 initially 

created as four pilot projects and now being pursued as a consolidated 

effort, to review and determine if it will help to resolve some of the Navy’s 

long-standing financial and business management problems. The Navy 

expects the ERP to be fully operational by fiscal year 2011. It is currently 

estimated that for fiscal years 2004-2011, the program will cost 

approximately $800 million. When fully operational, the Navy reports that 

the ERP will manage an estimated 80 percent of its appropriated funds.4

 

Our objectives were to (1) provide a historical perspective on the planning 

and costs of the Navy’s four ERP pilot projects, and the decision to merge 

them into one program; (2) determine if the Navy has identified lessons 

from the pilots, how the lessons are being used, and the challenges that 

remain; and (3) determine if there are additional best business practices 

that could be used to improve management oversight of the Navy ERP.

To obtain a historical perspective on the planning and costs of the Navy’s 

four ERP pilot projects, and the decision to merge them into one program, 

we reviewed DOD’s budget justification materials and other background 

information on the four pilot projects and met with program management 

and DOD Chief Information Officer (CIO) officials. Additionally, we 

reviewed project documentation provided by DOD and held discussions 

with program management officials related to two key processes—

requirements management and testing—to determine if these key aspects 

of program management were being performed and if the system is being 

designed to help address some of Navy’s long-standing management 

problems. Further, we reviewed relevant industry standards and best 

practices, and interviewed and requested documentation from the Navy 

ERP to determine whether there are additional best business practices that 

could appropriately be used to improve management oversight of the ERP. 

Given that this effort is still in the early stages of development, we did not 

evaluate all best practices. Rather, we concentrated on those that could 

have an immediate impact in improving management’s oversight.

3

An ERP solution is an automated system using commercial off-the-shelf (COTS) software 

consisting of multiple, integrated functional modules that perform a variety of businessrelated tasks such as payroll, general ledger accounting, and supply chain management. 

4

The 80 percent is calculated after excluding estimated appropriated funding for the Marine 

Corps and military personnel and pay. Of the 80 percent, about 2 percent of the 

appropriated funds will be executed and maintained in detail by the financial management 

systems at the aviation and shipyard depots. 
Page 3 GAO-05-858 Navy Enterprise Resource Planning

Our work was performed from August 2004 through June 2005 in 

accordance with U.S. generally accepted government auditing standards. 

Details on our scope and methodology are included in appendix I. We 

requested comments on a draft of this report from the Secretary of Defense 

or his designee. Written comments from the Deputy Under Secretary of 

Defense (Financial Management) and the Deputy Under Secretary of 

Defense (Business Transformation) are reprinted in appendix II.

Results in Brief The Navy has invested approximately $1 billion in its four pilot ERP efforts, 

without marked improvement in its day-to-day operations. The four pilots 

were limited in scope and were not intended to be a corporate solution for 

resolving any of the Navy’s long-standing financial and business 

management problems. Because of the various inconsistencies in the 

design and implementation, the pilots were not interoperable, even though 

they performed many of the same business functions. The lack of a 

coordinated effort among the pilots led to a duplication of efforts in 

implementing many business functions and resulted in ERP solutions that 

carry out similar functions in different ways from one another. In essence, 

the pilots resulted in four more DOD stovepiped systems that did not 

enhance DOD’s overall efficiency and resulted in $1 billion being largely 

wasted.

Because the pilots were stovepiped, limited within the scope of their 

respective commands, and not interoperable, they did not transform the 

Navy’s business operations. As a result, under the leadership of a central 

program office—something that was lacking in the pilots—the Navy 

decided to start over and undertake the development and implementation 

of a single ERP system. Using the lessons learned from the pilots, the 

current Navy ERP program office has so far demonstrated a commitment 

to the disciplined processes necessary to manage this effort and reduce the 

risks associated with system implementation. Specifically, we found that, 

unlike other reviews we have performed at DOD and other agencies, the 

Navy ERP program office is following an effective process for identifying 
Page 4 GAO-05-858 Navy Enterprise Resource Planning

and documenting requirements.5 Requirements represent the essential 

blueprint that system developers and program managers use to design, 

develop, test, and implement a system.

While the Navy ERP has the potential to address some of the Navy’s 

financial management weaknesses, its current planned functionality will 

not provide an all-inclusive end-to-end corporate solution for the Navy. For 

example, the current scope of the ERP does not provide for real-time asset 

visibility of shipboard inventory. Asset visibility has been and continues to 

be a long-standing problem within the department. The scope of the 

current effort is much larger than the pilots. The system is intended to 

manage an estimated 80 percent of the Navy’s appropriated funds, 

according to the Navy ERP program office.

The project has a long way to go, with a current estimated completion date 

of 2011, at an estimated cost of $800 million. These estimates are very 

likely to change, given (1) the Navy ERP’s relatively early phase of 

development and (2) DOD’s past history of not implementing systems on 

time and within budget. The project faces numerous significant challenges 

and risks that must be dealt with as the project moves forward. For 

example, 44 system interfaces6

 with other Navy and DOD systems must be 

developed and implemented. Long-standing problems regarding the lack of 

integrated systems and use of nonstandard data within DOD pose 

significant challenges and risks to a successful Navy ERP interface with 

these systems. Testing the system interfaces in an end-to-end manner is 

necessary in order for the Navy to have reasonable assurance that the ERP 

will be capable of providing the intended functionality. Another challenge 

and risk is the Navy’s ability to convert the necessary data from its legacy 

systems into the ERP system. Data conversion is one of the critical tasks 

necessary to successfully implement a new financial system. If data 

conversion is done right, the new system has a much greater opportunity 

for success. On the other hand, converting data incorrectly or entering 

unreliable data from a legacy system has lengthy and long-term 

5

According to the Software Engineering Institute, requirements management is a process 

that establishes a common understanding between the customer and the software project 

manager regarding the customer’s business needs that will be addressed by a project. A 

critical part of this process is to ensure that the requirements development portion of the 

effort documents, at a sufficient level of detail, the problems that need to be solved and the 

objectives that need to be achieved.

6

An interface is a connection between two devices, applications, or networks or a boundary 

across which two systems communicate. 
Page 5 GAO-05-858 Navy Enterprise Resource Planning

repercussions. The adage “garbage in, garbage out” best describes the 

adverse impact of inadequate data conversion efforts. 

A broader challenge and risk that is out of the Navy ERP project’s control, 

but could significantly affect it, is DOD’s development of a business 

enterprise architecture (BEA).7

 An enterprise architecture consists of 

snapshots of the enterprise’s current environment and its target 

environment, as well as a capital investment road map for transitioning 

from the current to the target environment. The real value of an enterprise 

architecture is that it provides the necessary content for guiding and 

constraining system investments in a way that promotes interoperability 

and minimizes overlap and duplication. As we have recently reported,8

DOD’s BEA still lacks many of the key elements of a well-defined 

architecture, and no basis exists for evaluating whether the Navy ERP will 

be aligned with the BEA and whether it would be a corporate solution for 

DOD in its “To Be” or target environment. At this time, it is unknown what 

the target environment will be. Therefore, it is unknown what business 

processes, data standards, and technological standards the Navy ERP must 

align to, as well as what legacy systems will be transitioned into the target 

environment. Developing a large-scale systems modernization program, 

such as the Navy ERP, without the context of an enterprise architecture 

poses risks of rework to comply with the BEA once it is fully defined.

While we are encouraged by the Navy’s management of the requirements 

process, other actions are needed to enhance management’s oversight, 

both Navy and DOD, of the ERP effort. The Navy does not have in place 

the structure to capture quantitative data that can be used to assess the 

effectiveness of the overall effort. This information is necessary to 

understand the risk being assumed and whether the project will provide the 

desired functionality. Additionally, the Navy has not established an 

independent verification and validation (IV&V) function to provide an 

assessment of the Navy ERP to DOD and Navy management. Rather, the 

Navy is using in-house subject matter experts and others who report to the 

7

In July 2001, the Secretary of Defense established the Business Management Modernization 

Program to improve the efficiency and effectiveness of DOD business operations and to 

provide the department’s leaders with accurate and timely information through the 

development and implementation of a business enterprise architecture. 

8

GAO, DOD Business Systems Modernization: Long-standing Weaknesses in Enterprise 

Architecture Development Need to Be Addressed, GAO-05-702 (Washington, D.C.: July 22, 

2005).
Page 6 GAO-05-858 Navy Enterprise Resource Planning

Quality Assurance team leader. Industry best practices indicate that IV&V 

activities should be conducted by individuals independent of the project 

and who report directly to agency management in order to provide added 

assurance that reported results on the project’s status are unbiased. Since 

effective implementation of the disciplined processes has been shown to be 

a key indicator of whether a project has reduced its risk to acceptable 

levels, these management actions would provide increased confidence that 

the Navy ERP project is “on track.” 

Considering that the project is still in its early stages of development and 

that there are significant challenges and high risks ahead, it is critical that 

mechanisms be in place to critically review the project at all stages to 

ensure it will result in improvements to the Navy’s operations and alert the 

Navy and DOD if the project does not remain on schedule and within 

budget. Given the department’s past history of not being able to 

successfully implement business systems on time and within budget, 

accountability is imperative. Thus, we are making three recommendations 

to the Secretary of Defense directed towards improving oversight over the 

Navy ERP. In its written comments on a draft of this report, DOD generally 

agreed with our recommendations. DOD agreed to develop quantitative 

metrics that can be used to evaluate the program and stated that the 

metrics would be developed by December 2005. While the department 

agreed with the recommendation to establish an IV&V, it noted that the 

IV&V would be within the project team and will report directly to the Navy 

ERP program manager. Additionally, while the department agreed with the 

intent of our recommendation that the Defense Business System 

Management Committee review the status of the Navy ERP on a 

semiannual basis, DOD noted that its tiered accountability structure, 

recently put in place to improve the control and accountability of business 

system investments, would provide the necessary oversight. In regard to 

the last two points, we continue to believe that providing the IV&V reports 

to the appropriate investment review board and instituting semiannual 

reviews by the Defense Business System Management Committee are the 

appropriate courses of action. Given (1) the Navy’s initial estimate that the 

new ERP will cost at least $800 million and (2) the department’s past 

difficulties in effectively developing and implementing business systems, 

substantive reviews that are focused just on the Navy ERP by the highest 

levels of management within the department are warranted so that 

management can act quicker rather than later if problems emerge.

In its comments, the department took exception to our characterization of 

the pilots as failures and identified what it believes were achievements of 
Page 7 GAO-05-858 Navy Enterprise Resource Planning

the pilot programs. However, as discussed in the report, the pilots were 

limited in scope. Although they used the same software, inconsistencies in 

the design and implementation resulted in them not being interoperable. In 

essence, the department had four more stovepiped systems. The 

department’s comments did not disagree with any of the above points. We 

characterized the pilots as failures because the department spent $1 billion 

on systems that did not result in marked improvement in the Navy’s day-today operations. Furthermore, if there had been effective management 

oversight of the pilot programs at the outset, there would not have been a 

need to, in essence, start over with the Navy ERP. See the Agency 

Comments and Our Evaluation section of this report for a more detailed 

discussion of the agency comments. We have reprinted DOD’s written 

comments in appendix II.

Background The Navy, with reported assets totaling $321 billion in fiscal year 2004,9

 

would be ranked among the largest corporations in the world if it were a 

private sector entity. According to the Navy, based upon the reported value 

of its assets, it would be ranked among the 15 largest corporations on the 

Fortune 500 list. Additionally, in fiscal year 2004 the Navy reported that its 

inventory was valued at almost $73 billion and that it held property, plant, 

and equipment with a reported value of almost $156 billion. Furthermore, 

the Navy reported for fiscal year 2004 that its operations involved total 

liabilities of $38 billion, that its operations had a net cost of $130 billion, 

and that it employed approximately 870,000 military and civilian 

personnel—including reserve components.10 

The primary mission of the Navy is to control and maintain freedom of the 

seas, performing an assortment of interrelated and interdependent 

business functions to support its military mission with service members 

and civilian personnel in geographically dispersed locations throughout the 

world. To support its military mission and perform its business functions, 

the Navy requested for fiscal year 2005 almost $3.5 billion for the operation, 

maintenance, and modernization of its business systems and related 

infrastructure—the most of all the DOD components—or about 27 percent 

9

Department of the Navy, Fiscal Year 2004 Annual Financial Report (November 2004). 

Numbers are combined from the fiscal year 2004 General Fund and Working Capital Fund 

financial reports.

10This includes the Marine Corps.
Page 8 GAO-05-858 Navy Enterprise Resource Planning

of the total $13 billion DOD fiscal year 2005 business systems budget 

request. Of the 4,150 reported DOD business systems, the Navy holds the 

largest inventory of business systems—with 2,353 reported systems or 57 

percent of DOD’s reported inventory of business systems. 

The Secretary of Defense recognized that the department’s business 

operations and systems have not effectively worked together to provide 

reliable information to make the most effective business decisions. He 

challenged each military service to transform its business operations to 

support DOD’s warfighting capabilities and initiated the Business 

Management Modernization Program (BMMP) in July 2001. Further, the 

Assistant Secretary of the Navy for Financial Management and Comptroller 

(Navy Comptroller) testified that transforming the Navy’s business 

processes, while concurrently supporting the Global War on Terrorism, is a 

formidable but essential task.11 He stated that the goal of the 

transformation is to “establish a culture and sound business processes that 

produce high-quality financial information for decision making.” One of 

the primary elements of the Navy’s business transformation strategy is the 

Navy ERP. 

Recently Reported Business 

and Financial Weaknesses 

at the Navy

The need for business processes and systems transformation to provide 

management with timely information to make important business decisions 

is clear. However, none of the military services, including the Navy, have 

passed the scrutiny of an independent financial audit. Obtaining a clean 

(unqualified) financial audit opinion is a basic prescription for any wellmanaged organization, as recognized by the President’s Management 

Agenda. For fiscal year 2004, the DOD Inspector General issued a 

disclaimer on the Navy’s financial statements—Navy’s General Fund and 

Working Capital Fund—citing eight material weaknesses12 and six material 

11Status of Financial Management Reform Within the Department of Defense and the 

Individual Services: Hearing Before the Subcommittee on Readiness and Management 

Support, Senate Armed Services Committee, 108th Cong. 2 (Nov. 18, 2004) (statement by the 

Assistant Secretary of the Navy (Financial Management and Comptroller), Richard Greco, 

Jr.). 

12The eight material weaknesses for the Navy General Fund are related to: (1) accounting 

and financial management systems; (2) fund balance with Treasury; (3) accounts receivable; 

(4) inventory and related property; (5) general property, plant, and equipment; (6) accounts 

payable; (7) environmental liabilities; and (8) problem disbursements. 
Page 9 GAO-05-858 Navy Enterprise Resource Planning

weaknesses13 respectively, in internal control and noncompliance with the 

Federal Financial Management Integrity Act of 1996 (FFMIA).14 The 

inability to obtain a clean financial audit opinion is the result of 

weaknesses in the Navy’s financial management and related business 

processes and systems. Most importantly, the Navy’s pervasive 

weaknesses have (1) resulted in a lack of reliable information to make 

sound decisions and report on the status of activities, including 

accountability of assets, through financial and other reports to the Navy 

and DOD management and the Congress; (2) hindered its operational 

efficiency; (3) adversely affected mission performance; and (4) left the 

Navy and DOD vulnerable to fraud, waste, and abuse, as the following 

examples illustrate.

• The Navy’s lack of detailed cost information hinders its ability to 

monitor programs and analyze the cost of its activities. We reported15

that the Navy lacked the detailed cost and inventory data needed to 

assess its needs, evaluate spending patterns, and leverage its 

telecommunications buying power. As a result, at the sites we 

reviewed, the Navy paid for telecommunications services it no longer 

required, paid too much for services it used, and paid for potentially 

fraudulent or abusive long-distance charges. In one instance, we found 

that DOD paid over $5,000 in charges for one card that was used to place 

189 calls in one 24-hour period from 12 different cities to 12 different 

countries.

13The six material weaknesses for the Navy Working Capital Fund are related to: (1) 

accounting and financial management systems; (2) fund balance with Treasury; (3) accounts 

receivable; (4) inventory and related property; (5) general property, plant, and equipment; 

and (6) accounts payable.

14FFMIA, Pub. L. No. 104-208, div. A., §101(f), title VIII, 110 Stat. 3009, 3009-389 (Sept. 30, 

1996), requires the 24 major departments and agencies covered by the Chief Financial 

Officers Act of 1990, Pub. L. No. 101-576, 104 Stat. 2838 (Nov. 15, 1990) (31 U.S.C. § 901(b), 

as amended), to implement and maintain financial management systems that comply 

substantially with (1) federal financial management systems requirements, (2) applicable 

federal accounting standards, and (3) the U.S. Standard General Ledger at the transaction 

level.

15GAO, Vendor Payments: Inadequate Management Oversight Hampers the Navy’s Ability 

to Effectively Manage Its Telecommunication Program, GAO-04-671 (Washington, D.C.: 

June 14, 2004).
Page 10 GAO-05-858 Navy Enterprise Resource Planning

• Ineffective controls over Navy foreign military sales using blanket 

purchase orders placed classified and controlled spare parts at risk of 

being shipped to foreign countries that may not be eligible to receive 

them. For example, we identified instances in which Navy country 

managers (1) overrode the system to release classified parts under 

blanket purchase orders without filing required documentation 

justifying the release; and (2) substituted classified parts for parts 

ordered under blanket purchase orders, bypassing the control-edit 

function of the system designed to check a country’s eligibility to 

receive the parts.16

• The Naval Inventory Control Point and its repair contractors have not 

followed DOD and Navy procedures intended to provide the 

accountability for and visibility of inventory shipped to Navy repair 

contractors. Specifically, Navy repair contractors are not routinely 

acknowledging receipt of government-furnished material received from 

the Navy. A DOD procedure requires repair contractors to acknowledge 

receipt of government-furnished material that has been shipped to them 

from the Navy’s supply system. However, Naval Inventory Control Point 

officials are not requiring repair contractors to acknowledge receipt of 

these materials. By not requiring repair contractors to acknowledge 

receipt of government-furnished material, the Naval Inventory Control 

Point has also departed from the procedure to follow up with the 

contractor within 45 days when the contractor fails to confirm receipt 

for an item. Without material receipt notification, the Naval Inventory 

Control Point cannot be assured that its repair contractors have 

received the shipped material. This failure to acknowledge receipt of 

material shipped to repair contractors can potentially impair the Navy’s 

ability to account for shipments leading to possible fraud, waste, or 

abuse.17

16GAO, Foreign Military Sales: Improved Navy Controls Could Prevent Unauthorized 

Shipment of Classified and Controlled Spare Parts to Foreign Countries, GAO-04-507

(Washington, D.C.: June 25, 2004).

17GAO, Defense Inventory: Navy Needs to Improve the Management Over GovernmentFurnished Material Shipped to Its Repair Contractors, GAO-04-779 (Washington, D.C.: July 

23, 2004).
Page 11 GAO-05-858 Navy Enterprise Resource Planning

• A limited Naval Audit Service audit revealed that 53 of 118 erroneous 

payment transactions, valued at more than $990,000, occurred because 

Navy certifying officials did not ensure accurate information was 

submitted to the Defense Finance and Accounting Service (DFAS) prior 

to authorizing payment. In addition, certifying officials submitted 

invoices to DFAS authorizing payment more than once for the same 

transaction.18

Brief Overview of Navy ERP To address the need for business operations reform, in fiscal year 1998 the 

Navy established an executive committee responsible for creating a 

“Revolution in Business Affairs” to begin looking at transforming business 

affairs and identifying areas of opportunity for change. This committee, in 

turn, set up a number of working groups, including one called the 

Commercial Business Practices (CBP) Working Group,19 which consisted 

of representatives from financial management organizations across the 

Navy. This working group recommended that the Navy should use ERP as 

a foundation for change and identified various ERP initiatives that were 

already being developed or under consideration within the Navy. 

Ultimately, the Navy approved the continuation of four of these initiatives, 

using funds from existing resources from each of the sponsors (i.e., 

commands) to test the feasibility of ERP solutions within the Navy. From 

1998 to 2003, four different Navy commands began planning, developing, 

and implementing four separate ERP pilot programs to address specific 

business areas. A CBP Executive Steering Group was created in December 

1998 to monitor the pilot activities.

18Naval Audit Service, Erroneous Payments Made to Navy Vendors, N2005-0011 

(Washington, D.C.: Dec. 2, 2004).

19Initially the focus of the committee was on financial practices, and it was named the 

Commercial Financial Practices Working Group. The committee changed its name to reflect 

a broader focus from financial practices to business practices.
Page 12 GAO-05-858 Navy Enterprise Resource Planning

As the pilots progressed in their development and implementation, the 

Navy identified issues that had to be addressed at a higher level than the 

individual pilots, such as the integration between the pilots as well as with 

other DOD systems, and decided that one program would provide a more 

enterprisewide solution for the Navy. In August 2002, the Assistant 

Secretary of the Navy for Research, Development, and Acquisition 

established a Navy-wide ERP program to “converge” the four ongoing 

pilots into a single program. This Navy-wide program is expected to 

replace all four pilots by fiscal year 2008 and to be “fully operational” by 

fiscal year 2011. The Navy estimates that the ERP will manage about 80 

percent of the Navy’s estimated appropriated funds—after excluding 

appropriated funds for the Marine Corps and military personnel and pay. 

Based on the Navy’s fiscal years 2006 to 2011 defense planning budget, the 

Navy ERP will manage approximately $74 billion annually.20 

According to a Navy ERP official, while the Navy ERP would account for 

the total appropriated amount, once transactions occur at the depots, such 

as when a work order is prepared for the repair of an airplane part, the 

respective systems at the depots will execute and maintain the detailed 

transactions. This accounts for about 2 percent, or approximately $1.6 

billion, being executed and maintained in detail by the respective systems 

at the aviation and shipyard depots—not by the Navy ERP. The remaining 

20 percent that the ERP will not manage comprises funds for the Navy 

Installations Command, field support activity, and others.

Navy’s Pilot Projects 

Lacked Coordinated 

Management Oversight

Each of the Navy’s four ERP pilot projects was managed and funded by 

different major commands within the Navy. The pilots, costing over $1 

billion in total, were limited in scope and were not intended to provide 

corporate solutions to any of the Navy’s long-standing financial and 

business management problems. The lack of centralized management 

oversight and control over all four pilots allowed the pilots to be developed 

independently. This resulted in four more DOD stovepiped systems that 

could not operate with each other, even though each carried out many of 

the same functions and were based on the same ERP commercial-off-theshelf (COTS) software. Moreover, due to the lack of high-level 

departmentwide oversight from the start, the pilots were not required to go 

20The amount is based on DOD’s estimated planning budget for fiscal years 2006 through 

2011.
Page 13 GAO-05-858 Navy Enterprise Resource Planning

through the same review process as other acquisition projects of similar 

magnitude.

Pilots Developed 

Independently of Each 

Other

Four separate Navy organizations began their ERP pilot programs 

independently of each other, at different times, and with separate funding. 

All of the pilots implemented the same ERP COTS software, and each pilot 

was small in scale—relative to the entire Navy. For example, one of the 

pilots, SMART, was responsible for managing the inventory items and 

repair work associated with one type of engine, although the organization 

that implemented SMART—the Naval Supply Systems Command—

managed the inventory for several types of engines. As of September 2004, 

the Navy estimated that the total investment in these four pilots was 

approximately $1 billion. Table 1 summarizes each of the pilots, the 

cognizant Navy organization, the business areas they address, and their 

reported costs through September 2004.

Table 1: Navy ERP Pilot Projects

Source: GAO analysis of Navy data.

a

The costs reflect amounts disbursed through September 30, 2004, as reported by the Navy ERP 

program.

Dollars in millions

ERP pilot Organization Area of pilot’s focus Initial start

Costs through

FY 2004a

CABRILLO Space and Naval Warfare 

Systems Command 

Financial Management

• Navy Working Capital Fund

June 2000 $67.4

SMART Naval Supply Systems 

Command

Supply Management

• Intermediate-Level Maintenance 

Management

• Interface to Aviation Depots

August 2000 346.4

NEMAIS Naval Sea Systems Command Regional Maintenance

• Intermediate-Level Maintenance 

Management

• Project Systems

• Workforce Management

June 2000 414.6

SIGMA Naval Air Systems Command Program Management with linkage 

among:

• Contract Management 

• Financial Management

• Workforce Management

May 2001 215.9

Total $1,044.3
Page 14 GAO-05-858 Navy Enterprise Resource Planning

Even after the pilots came under the purview of the CBP Executive 

Steering Group in December 1998, they continued to be funded and 

controlled by their respective organizations. We have previously reported21

that allowing systems to be funded and controlled by component 

organizations has led to the proliferation of DOD’s business systems. 

These four pilots are prime examples. While there was an attempt made to 

coordinate the pilots, ultimately each organization designed its ERP pilot to 

accommodate its specific business needs. The Navy recognized the need 

for a working group that would focus on integration issues among the 

pilots, especially because of the desire to eventually extend the pilot 

programs beyond the pilot organizations to the entire Navy. In this regard, 

the Navy established the Horizontal Integration Team in June 1999, 

consisting of representatives from all of the pilots to address this matter. 

However, one Navy official described this team as more of a “loose 

confederation” that had limited authority. As a result, significant resources 

have been invested that have not and will not result in corporate solutions 

to any of the Navy’s long-standing business and financial management 

problems. This is evident as noted in the DOD Inspector General’s audit 

reports of the Navy’s financial statements discussed above. 

In addition to the lack of centralized funding and control, each of the pilots 

configured the software differently, which, according to Navy ERP program 

officials, caused integration and interoperability problems. While each 

pilot used the same COTS software package, the software offers a high 

degree of flexibility in how similar business functions can be processed by 

providing numerous configuration points.22 According to the Navy, over 2.4 

million configuration points exist within the software. The pilots 

configured the software differently from each other to accommodate 

differences in the way they wanted to manage their functional area focus. 

These differences were allowed even though they perform many of the 

same types of business functions, such as financial management. These 

configuration differences include the levels of complexity in workflow 

activities and the establishment of the organizational structure. For 

example, the primary work order managed by the NEMAIS pilot is an 

intricate ship repair job, with numerous tasks and workers at many levels. 

21GAO, Department of Defense: Financial and Business Management Transformation 

Hindered by Long-standing Problems, GAO-04-941T (Washington, D.C.: July 8, 2004).

22A configuration point is a place at which the system developer must define the business 

flows with inputs, conditions, and criteria that will be used in the application.
Page 15 GAO-05-858 Navy Enterprise Resource Planning

Other pilots had much simpler work order definitions, such as preparing a 

budget document or procuring a single part for an engine.

Because of the various inconsistencies in the design and implementation, 

the pilots were stovepiped and could not operate with each other, even 

though they performed many of the same business functions. Table 2 

illustrates the similar business functions that are performed by more than 

one pilot. 

Table 2: Functions Performed by the Pilot Projects

Source: GAO, based upon information provided by the Navy.

By definition, an ERP solution should integrate the financial and business 

operations of an organization. However, the lack of a coordinated effort 

among the pilots led to a duplication of efforts and problems in 

implementing many business functions and resulted in ERP solutions that 

carry out redundant functions in different ways from one another. 

Type of functions to be performed NEMAIS Cabrillo SIGMA SMART

Materials management 

• Sales and distribution X X X X

• Procurement X X X

Financial management

• Financial accounting X X X X

• Revenue and cost controlling X X X X

• Asset accounting X X X X

• Budgeting and funds management X X X

Program management

• Project management X X X

• Program planning, budgeting, control X X X

Workforce management

• Time and attendance X X X
Page 16 GAO-05-858 Navy Enterprise Resource Planning

The end result of all of the differences was a “system” that could not 

successfully process transactions associated with the normal Navy 

practices of moving ships and aircraft between fleets. Another 

configuration problem occurred because the pilots generally developed 

custom roles23 for systems users. Problems arose after the systems began 

operating. Some roles did not have the correct transactions assigned to 

enable the users with that role to do their entire job correctly. Further, 

other roles violated the segregation-of-duties principle due to the 

inappropriateness of roles assigned to individual users.

The pilots experienced other difficulties with respect to controlling the 

scope and performance schedules due to the lack of disciplined 

processes,24 such as requirements management. For example, the pilots 

did not identify in a disciplined manner the amount of work necessary to 

achieve the originally specified capabilities—even as the end of testing 

approached. There were repeated contract cost-growth adjustments, 

delays in delivery of many planned capabilities, and initial periods of 

systems’ instabilities after the systems began operating. All of these 

problems have been shown as typical of the adverse effects normally 

associated with projects that have not effectively implemented disciplined 

processes.

23Roles are the actions and activities assigned to or required or expected of a person or 

group. A user’s access to the transactions, reports, and applications is controlled by the 

roles assigned to the user.

24Disciplined processes include a wide range of activities, including project planning and 

oversight, requirements management, risk management, and testing. 
Page 17 GAO-05-858 Navy Enterprise Resource Planning

Pilots Lacked 

Departmentwide Oversight 

The Navy circumvented departmentwide policy by not designating the 

pilots as major automated information systems acquisition programs. DOD 

policy in effect at the time25 stipulated that a system acquisition should be 

designated as a major program if the estimated cost of the system exceeds 

$32 million in a single year, $126 million in total program costs, or $378 

million in total life-cycle costs, or if deemed of special interest by the DOD 

Chief Information Officer (CIO). According to the Naval Audit Service,26 all 

four of the pilots should have been designated as major programs based on 

their costs—which were estimated to be about $2.5 billion at the time—and 

their significance to Navy’s operations. More specifically, at the time of its 

review, SMART’s total estimated costs for development, implementation, 

and sustainment was over $1.3 billion—far exceeding the $378 million lifecycle cost threshold. However, because Navy management considered 

each of its ERP programs to be “pilots,” it did not designate the efforts as 

major automated information systems acquisitions, thereby limiting 

departmental oversight.27

25Department of Defense Directive 5000.1, The Defense Acquisition System, DOD 

Instruction 5000.2, Operation of the Defense Acquisition System (Apr. 5, 2002) and DOD 

Regulation 5000.2-R, Mandatory Procedures for Major Defense Acquisition Programs and 

Major Automated Information System Acquisition Programs (Apr. 5, 2002). The DOD 

policy also specifies that the DOD CIO is the milestone decision authority, responsible for 

program approval, for all major automated information systems.

26Naval Audit Service, Department of the Navy Implementation of Enterprise Resource 

Planning Solutions, N2002-0024 (Washington, D.C.: Jan. 25, 2002). 

27Subsequent to the Naval Audit Service’s report, on July 21, 2003, NEMAIS was designated a 

major automated information system. This memorandum also allowed the fielding of 

NEMAIS to four Navy regions. 
Page 18 GAO-05-858 Navy Enterprise Resource Planning

Consistent with the Clinger-Cohen Act of 1996, DOD acquisition guidance28

requires that certain documentation be prepared at each milestone within 

the system life cycle. This documentation is intended to provide relevant 

information for management oversight and in making decisions as to 

whether the investment of resources is cost beneficial. The Naval Audit 

Service reported 29 that a key missing document that should have been 

prepared for each of the pilots was a mission needs statement.30 A mission 

needs statement was required early on in the acquisition process to 

describe the projected mission needs of the user in the context of the 

business need to be met. The mission needs statement should also address 

interoperability needs. As noted by the Naval Audit Service,the result of 

not designating the four ERP pilots as major programs was that program 

managers did not prepare and obtain approval of this required document 

before proceeding into the next acquisition phase. In addition, the pilots 

did not undergo mandatory integrated reviews that assess where to spend 

limited resources departmentwide. The DOD CIO is responsible for 

overseeing major automated information systems and a program executive 

office is required to be dedicated to executive management and not have 

other command responsibilities. However, because the pilots were not 

designated major programs, the oversight was at the organizational level 

that funded the pilots (i.e., command level). Navy ERP officials stated that 

at the beginning of the pilots, investment authority was dispersed 

throughout the Navy and there was no established overall requirement 

within the Navy to address systems from a centralized Navy enterprise 

level. The Navy ERP is now designated a major program under the 

oversight of the DOD CIO. 

28DOD, Operation of the Defense Acquisition System, DOD Instruction 5000.2, (Apr. 5, 

2002); and DOD Regulation 5000.2-R, Mandatory Procedures for Major Defense Acquisition 

Programs and Major Automated Information System Acquisition Programs (Apr. 5, 

2002). The acquisition controls described in this guidance apply to all DOD acquisition 

programs.

29Naval Audit Service N2002-0024. 

30The mission needs statement has been replaced by the requirement for an Initial 

Capabilities Document.
Page 19 GAO-05-858 Navy Enterprise Resource Planning

Experience Has Shown the 

Effects of Not Effectively 

Implementing Disciplined 

Processes

The problems identified in the failed implementation of the four pilots are 

indicative of a system program that did not adhere to the disciplined 

processes. The successful development and implementation of systems is 

dependent on an organization’s ability to effectively implement best 

practices, commonly referred to as disciplined processes, which are 

essential to reduce the risks associated with these projects to acceptable 

levels.31 However, the inability to effectively implement the disciplined 

processes necessary to reduce risks to acceptable levels does not mean 

that an entity cannot put in place a viable system that is capable of meeting 

its needs. Nevertheless, history shows that the failure to effectively 

implement disciplined processes and the necessary metrics to understand 

the effectiveness of processes implemented increases the risk that a given 

system will not meet its cost, schedule, and performance objectives.

In past reports we have highlighted the impact of not effectively 

implementing the disciplined processes. These results are consistent with 

those experienced by the private sector. More specifically: 

• In April 2003, we reported32 that NASA had not implemented an effective 

requirements management process and that these requirement 

management problems adversely affected its testing activities. We also 

noted that because of the testing inadequacies, significant defects later 

surfaced in the production system. In May 2004, we reported33 that 

NASA’s new financial management system, which was fully deployed in 

June 2003 as called for in the project schedule, still did not address 

many of the agency’s most challenging external reporting issues, such as 

external reporting problems related to property accounting and 

budgetary accounting. The system continues to be unable to produce 

reliable financial statements. 

31Acceptable levels refer to the fact that any systems acquisition effort will have risks and 

will suffer the adverse consequences associated with defects in the processes. However, 

effective implementation of the disciplined processes reduces the potential of risks actually 

occurring and prevents significant defects from materially affecting the cost, timeliness, and 

performance of the project. 

32GAO, Business Modernization: Improvements Needed in Management of NASA’s 

Integrated Financial Management Program, GAO-03-507 (Washington, D.C.: Apr. 30, 2003).

33GAO, National Aeronautics and Space Administration: Significant Actions Needed to 

Address Long-standing Financial Management Problems, GAO-04-754T (Washington, D.C.: 

May 19, 2004).
Page 20 GAO-05-858 Navy Enterprise Resource Planning

• In May 2004, we reported34 that the Army’s initial deployments for its 

Logistics Modernization Program (LMP) did not operate as intended and 

experienced significant operational difficulties. In large part, these 

operational problems were due to the Army not effectively 

implementing the disciplined processes that are necessary to manage 

the development and implementation of the systems in the areas of 

requirements management and testing. The Army program officials have 

acknowledged that the problems experienced in the initial deployment 

of LMP could be attributed to requirements and testing. Subsequently, in 

June 2005,35 we reported that the Army still had not put into place 

effective management control and processes to help ensure that the 

problems that have been identified since LMP became operational in 

July 2003 are resolved in an efficient and effective manner. The Army’s 

inability to effectively implement the disciplined processes provides it 

with little assurance that (1) system problems experienced during the 

initial deployment that caused the delay of future deployments have 

been corrected and (2) LMP is capable of providing the promised system 

functionality. The failure to resolve these problems will continue to 

impede operations at Tobyhanna Army Depot, and future deployment 

locations can expect to experience similar significant disruptions in 

their operations, as well as having a system that is unable to produce 

reliable and accurate financial and logistics data.

• We reported in February 200536 that DOD had not effectively managed 

important aspects of the requirements for the Defense Integrated 

Military Human Resources System, which is to be an integrated 

personnel and pay system standardized across all military components. 

For example, DOD had not obtained user acceptance of the detailed 

requirements nor had it ensured that the detailed requirements were 

complete and understandable. Based on GAO’s review of a random 

sample of the requirements documentation, about 77 percent of the 

detailed requirements were difficult to understand. 

34GAO, DOD Business Systems Modernization: Billions Continue to Be Invested with 

Inadequate Management Oversight and Accountability, GAO-04-615 (Washington, D.C.: 

May 27, 2004).

35GAO, Army Depot Maintenance: Ineffective Oversight of Depot Maintenance Operations 

and System Implementation Efforts, GAO-05-441 (Washington, D.C.: June 30, 2005).

36GAO, DOD Systems Modernization: Management of Integrated Military Human Capital 

Program Needs Additional Improvements, GAO-05-189 (Washington, D.C.: Feb. 11, 2005).
Page 21 GAO-05-858 Navy Enterprise Resource Planning

The problems experienced by DOD and other agencies are illustrative of 

the types of problems that can result when disciplined processes are not 

properly implemented. The four Navy pilots provide yet another example. 

As discussed previously, because the pilots were four stovepiped efforts, 

lacking centralized management and oversight, the Navy had to start over 

when it decided to proceed with the current ERP effort after investing 

about $1 billion. Figure 1 shows how organizations that do not effectively 

implement disciplined processes lose the productive benefits of their 

efforts as a project continues through its development and implementation 

cycle. Although undisciplined projects show a great deal of productive 

work at the beginning of the project, the rework associated with defects 

begins to consume more and more resources. In response, processes are 

adopted in the hopes of managing what later turns out to be, in reality, 

unproductive work. However, generally these processes are “too little, too 

late,” and rework begins to consume more and more resources because the 

adequate foundations for building the systems were not done or not done 

adequately. In essence, experience shows that projects that fail to 

implement disciplined processes at the beginning are forced to implement 

them later, when it takes more time and they are less effective.

As can be seen in figure 1, a major consumer of project resources in 

undisciplined efforts is rework (also known as thrashing).
Page 22 GAO-05-858 Navy Enterprise Resource Planning

Figure 1: Percent of Effort Associated with Undisciplined Projects

Rework occurs when the original work has defects or is no longer needed 

because of changes in project direction. Disciplined organizations focus 

their efforts on reducing the amount of rework because it is expensive. 

Studies have shown that fixing a defect during testing is anywhere from 10 

to 100 times more expensive than fixing it during the design or 

requirements phase.37

37Steve McConnell, Rapid Development: Taming Wild Software Schedules (Redmond, 

Wash.: Microsoft Press, 1996).

Percent of Effort

Time

100

0

Lucky projects

finish here

Unlucky projects

get stuck here

Visible progress (coding)

Thrashing (unplanned rework and wasted effort)

Planning and process management

Thrashing and planning combine to

limit ability to make any visible progress

Source: Used by permission of Steve McConnell, Professional Software Development, (Boston, Addison-Wesley, 2004).
Page 23 GAO-05-858 Navy Enterprise Resource Planning

Requirements 

Management Process 

Effective, but 

Implementation 

Challenges and Risks 

Remain 

To date, Navy ERP management has followed a comprehensive and 

disciplined requirements management process, as well as leveraged 

lessons learned from the implementation of the four ERP pilot programs to 

avoid repeating past mistakes. Assuming that the project continues to 

effectively implement the processes it has adopted, the planned 

functionality of the Navy ERP has the potential to address at least some of 

the weaknesses identified in the Navy’s financial improvement plan. 

However, the project faces numerous challenges and risks. Since the 

program is still in a relatively early phase—it will not be fully operational 

until fiscal year 2011, at a currently estimated cost of $800 million—the 

project team must be continually vigilant and held accountable for ensuring 

that the disciplined processes are followed in all phases to help achieve 

overall success. For example, the project management office will need to 

ensure that it effectively oversees the challenges and risks associated with 

developing interfaces with 44 Navy and DOD systems and data 

conversion—areas that were troublesome in other DOD efforts we have 

audited. Considering that the project is in a relatively early phase and 

DOD’s history of not implementing systems on time and within budget, the 

projected schedule and costs estimates are subject to, and very likely will, 

change. Furthermore, a far broader challenge, which lies outside the 

immediate control of the Navy ERP program office, is that the ERP is 

proceeding without DOD having clearly defined its BEA. As we have 

recently reported,38 DOD’s BEA still lacks many of the key elements of a 

well-defined architecture. The real value of a BEA is that it provides the 

necessary content for guiding and constraining system investments in a 

way that promotes interoperability and minimizes overlap and duplication. 

Without it, rework will likely be needed to achieve those outcomes. 

Navy ERP Built on Lessons 

Learned from Pilot Projects

Although the four pilot projects were under the control of different entities 

and had different functional focuses, a pattern of issues emerged that the 

Navy recognized as being critical for effective development of future 

projects. The Navy determined that the pilots would not meet its overall 

requirements and concluded that the best alternative was to develop a new 

ERP system—under the leadership of a central program office—and use 

efforts from the pilots as starting points by performing a review of their 

functionality and lessons learned, eliminating redundancies, and 

developing new functionalities that were not addressed by the pilots. The 

38GAO-05-702.
Page 24 GAO-05-858 Navy Enterprise Resource Planning

lessons learned from the pilots cover technical, organizational, and 

managerial issues and reinforce the Navy’s belief that it must effectively 

implement the processes that are necessary to effectively oversee and 

manage the ERP efforts. Navy ERP project management recognizes that 

the failure to do so would, in all likelihood, result in this ERP effort 

experiencing the same problems as those resulting in the failure of the four 

earlier pilots. 

One of the most important lessons learned from the earlier experiences by 

the Navy ERP project management is the need for following disciplined 

processes to identify and manage requirements. As discussed later in this 

report, the ERP program is following best practices in managing the 

system’s requirements. A key part of requirements identification is to have 

system users involved in the process to ensure that the system will meet 

their needs. Additionally, the inclusion of system users in the detailed 

requirement development process creates a sense of ownership in the 

system, and prepares system users for upcoming changes to the way they 

conduct their business. Moreover, the experience from the pilots 

demonstrated that the working-level reviews must be cross functional. For 

example, the end-to-end process walkthroughs, discussed later, reinforce 

the overall business effect of a transaction throughout the enterprise, and 

help to avoid a stovepiped view of an entity’s operations.

Another lesson learned is the need to adopt business processes to conform 

with the types of business practices on which the standard COTS packages 

are based, along with the associated transaction formats.39 Just the 

opposite approach was pursued for the pilots, during which the Navy 

customized many portions of the COTS software to match the existing 

business process environment. However, the current Navy ERP 

management is restraining customization to the core COTS software to 

allow modifications only where legal or regulatory demands require. 

Obviously, minimizing the amount of customization reduces the complexity 

and costs of development. Perhaps more importantly, holding 

customization to a minimum helps an entity take advantage of two valuable 

benefits of COTS software. First, COTS software provides a mature, 

industry-proven “best practices” approach to doing business. The core 

elements of work-flow management, logistics, financial management, and 

39A transaction format is a logical process, as defined by the ERP. From the user’s point of 

view, a transaction is a self-contained unit, such as changing an address for a customer or 

executing a program. 
Page 25 GAO-05-858 Navy Enterprise Resource Planning

other components have been optimized for efficiency and standardization 

in private industry over many years. According to program officials, the 

Navy ERP will adhere to the fundamental concepts of using a COTS 

package and thus take advantage of this efficiency benefit by modifying 

their business practices to match the COTS software rather than vice versa 

as was done in the four pilots. Having the software dictate processes is a 

difficult transition for users to accept, and Navy ERP officials recognize the 

challenge in obtaining buy-in from system users. To meet this challenge, 

they are getting users involved early in requirements definition, planning 

for extensive training, and ensuring that senior level leadership emphasize 

the importance of process change, so the entire chain of command 

understands and accepts its role in the new environment. In effect, the 

Navy is taking the adopted COTS process and then presenting it to the 

users. As a result, the Navy is attempting to limit the amount of 

customization of the software package. One important consideration in 

doing this is that if the standard COTS components are adopted, the 

maintenance burden of upgrades remains with the COTS vendor.

Finally, the Navy learned from the pilots that it needed to manage its 

system integrators40 better. The ERP officials also found that they could 

significantly reduce their risk by using the implementation methodology of 

the COTS vendor rather than the specific approach of a system integrator. 

Each of the pilots had separate system integrators with their own particular 

methodology for implementing the COTS software. According to Navy 

ERP officials, using the implementation methodology and tool set of the 

COTS vendor maintains a closer link to the underlying software, and 

provides more robust requirements management by easily linking 

requirements from the highest level down to the COTS transaction level. 

Navy ERP is focused on staying as close as possible to the delivered COTS 

package, both in its avoidance of customization and its use of tools 

provided by the COTS vendor. In contrast, with the pilots, the Navy 

allowed the system integrators more latitude in the development process, 

relying on their expertise and experience with other ERP efforts to guide 

the projects. Navy ERP management realized they needed to maintain 

much better control over the integrators’ work. As a result, the Navy 

established the Strategy, Architecture, and Standards Group to structure 

and guide the effort across the Navy. 

40A system integrator is a company that specializes in enabling an organization to use offthe-shelf hardware and software packages to meet the organization’s computing needs.
Page 26 GAO-05-858 Navy Enterprise Resource Planning

Requirements Management 

Process Following Best 

Practices

Our review found that the ERP development team has so far followed an 

effective process for managing its requirements development. 

Documentation was readily available for us to trace selected requirements 

from the highest level to the lowest, detailed transaction level. This 

traceability allows the user to follow the life of the requirement both 

forward and backward through the documentation, and from origin 

through implementation. Traceability is also critical to understanding the 

parentage, interconnections, and dependencies among the individual 

requirements. This information in turn is critical to understanding the 

impact when a requirement is changed or deleted. 

Requirements represent the blueprint that system developers and program 

managers use to design, develop, test, and implement a system. Improperly 

defined or incomplete requirements have been commonly identified as a 

cause of system failure and systems that do not meet their cost, schedule, 

or performance goals. Without adequately defined requirements that have 

been properly reviewed and tested, significant risk exists that the system 

will need extensive and costly changes before it will achieve its intended 

capability. 

Because requirements provide the foundation for system testing, 

specificity and traceability defects in system requirements preclude an 

entity from implementing a disciplined testing process. That is, 

requirements must be complete, clear, and well documented to design and 

implement an effective testing program. Absent this, an organization is 

taking a significant risk that its testing efforts will not detect significant 

defects until after the system is placed into production. Industry 

experience indicates that the sooner a defect is recognized and corrected, 

the cheaper it is to fix. As shown in figure 2, there is a direct relationship 

between requirements and testing.
Page 27 GAO-05-858 Navy Enterprise Resource Planning

Figure 2: Relationship between Requirements Development and Testing

User acceptance testing

Verifies that system operates

correctly with operational hardware

and meets users' needs

Concept of operations

Specifies how system is used in

operation

System acceptance testing

Verifies that the complete system

satisfies functional requirements

Functional requirements

Specifies the high-level functions

of the system

Integration testing

Verifies that units of software, when

combined, work together as

intended

Design requirements

Specifies the tasks each software

component must perform

Unit testing

Verifies that each component of the

software faithfully implements the

detailed design

Detailed design and coding

Specifies the detailed steps for

each software component and

implements those steps

Stages of system development Stages of testing

Source: GAO.

For projects such

as ERPs, these

items are normally

handled by the

COTS vendor
Page 28 GAO-05-858 Navy Enterprise Resource Planning

Although the actual testing activities occur late in the development cycle, 

test planning can help disciplined activities reduce requirements-related 

defects. For example, developing conceptual test cases based on the 

requirements derived from the concept of operations and functional 

requirements stages can identify errors, omissions, and ambiguities long 

before any code is written or a system is configured. Disciplined 

organizations also recognize that planning testing activities in coordination 

with the requirements development process has major benefits. As we 

have previously reported,41 failure to effectively manage requirements and 

testing activities has posed operational problems for other system 

development efforts.

The Navy ERP requirements identification process began with formal 

agreement among the major stakeholders on the scope of the system, 

followed by detailed, working-level business needs from user groups and 

legacy systems. The high-level business or functional requirements 

identified initially are documented in the Operational Requirements 

Document (ORD). The ORD incorporates requirements from numerous 

major DOD framework documents42 and defines the capabilities that the 

system must support, including business operation needs such as 

acquisition, finance, and logistics. In addition, the ORD also identifies the 

numerous policy directives to which the Navy ERP must conform, such as 

numerous DOD infrastructure systems, initiatives, and policies. The ORD 

was distributed to over 150 Navy and DOD reviewers. It went through 

seven major revisions to incorporate the comments and suggestions 

provided by the reviewers before being finalized in April 2004. According 

to Navy ERP program officials, any requested role for the Navy ERP to 

perform that was not included in the ORD will not be supported. This is a 

critical decision that reduces the project’s risks since “requirements creep” 

is another cause of projects that do not meet their cost, schedule, and 

performance objectives. 

41See, for example, GAO-04-615 and GAO, Financial Management Systems: Lack of 

Disciplined Processes Puts Implementation of HHS’ Financial System at Risk, GAO-04-

1008 (Washington, D.C.: Sept. 23, 2004).

42The documents include: DOD Financial Bluebook; Global Information Grid Capstone 

Requirements Document (CRD) Crosswalk; Global Combat Support System CRD 

Crosswalk; Joint Deployment Systems CRD Crosswalk; DoD Joint Technical Architecture; 

DOD 8500.1, Information Assurance; DOD 8510.1-M, Information Technology Security 

Certification and Accreditation Process; and DOD 5000.1, The Defense Acquisition 

System.
Page 29 GAO-05-858 Navy Enterprise Resource Planning

We selected seven requirements43 from the ORD that related to specific 

Navy problem areas, such as financial reporting and asset management, 

and found that the requirements had the expected attributes, including the 

necessary detail one would normally expect to find for the requirement 

being reviewed. For example, a requirement stated that the ERP will 

provide reports of funds expended versus funds allocated. We found this 

requirement was described in a low-level requirement document called a 

Customer Input Template, which included a series of questions that must 

be addressed. The documentation further detailed the standard reports 

that were available based on the selection of configuration options. 

Further, the documentation of the detailed requirements identified the 

specific COTS screen number that would be used and described the screen 

settings that would be used when a screen was “activated.”

While the ORD specifies the overall capabilities of the system at a high 

level, more specific, working-level requirements also had to be developed 

to achieve a usable blueprint for configuration and testing of the system. 

To develop these lower-level requirements, the Navy ERP project held 

detailed working sessions where requirements and design specifications 

were discussed, refined, formalized, and documented. Each high-level 

requirement was broken down into its corresponding business processes, 

which in turn drove the selection of transactions (COTS functions) to be 

used for configuration of the software. For each selected transaction, 

comprehensive documentation was created to capture the source 

information that defines how and why a transaction must be configured. 

This documentation is critical for ensuring accurate configuration of the 

software, as well as for testing the functionality of the software after 

configuration. Table 3 describes the kinds of documentation used to 

maintain these lower-level detailed requirements.

43As discussed in appendix I, our approach to validating the effectiveness of the 

requirements management process relied on a selection of seven requirements from 

different functional areas. From the finance area, we selected the requirement to provide 

reports of funds expended versus funds allocated. From the intermediate-level 

maintenance management area, we selected the requirement related to direct cost per job 

and forecasting accuracy. From the procurement area, we selected the requirement to 

enable monitoring and management of cost versus plan. In the plant supply functions area, 

we reviewed the requirement related to total material visibility and access of material held 

by the activity and the enterprise. From the wholesale supply functions area, we selected 

the requirements of in-transit losses/in-transit write-offs and total material visibility and 

access of material held by the activity and the enterprise. Additionally, we reviewed the 

requirement that the ERP be compliant with federal mandates and requirements and the 

U.S. Standard General Ledger. 
Page 30 GAO-05-858 Navy Enterprise Resource Planning

Table 3: Documentation for Detailed Requirements

Source: GAO, based upon information provided by the Navy.

Additionally, the Navy ERP program is using a requirements management 

tool containing a database that links each requirement from the highest to 

the lowest level and maintains the relationship between the requirements. 

This tool helps to automate the linkage between requirements and helps 

provide the project staff reasonable assurance that its stated processes 

have been effectively implemented. This linkage is critical to 

understanding the scope of any potential change. For example, the users 

can utilize the tool to (1) determine the number of transactions affected by 

a proposed change and (2) identify the detailed documentation necessary 

for understanding how this change will affect each business process. To 

further ensure that the individual transactions ultimately support the 

adopted business process, Navy ERP officials conducted master business 

scenarios44 or end-to-end process walkthroughs. This end-to-end view of 

the business process ensures that the business functionality works across 

the various subsystems of the COTS package. For instance, the 

requirements for a purchase order could be viewed simply from the 

vantage point of a logistics person or the acquisition community. However, 

a purchase order also has financial ramifications, and therefore must be 

posted to financial records, such as the general ledger. The master 

business scenarios provide a holistic review of the business process 

surrounding each transaction. 

Documentation name Documentation description

Customer Input Templates A series of questions completed for every requirement. It enforces a 

comprehensive review of the requirement and documents the reasoning behind 

the answers.

Functional Design Specification A detailed description for each interface.

Technical Functional Script A description of any customization that had to be made to a transaction.

Implementation Guide Automatically documents the actual configuration choices made by the 

developer for each transaction.

44A business scenario is a description of the flow of business processes that runs within a 

particular area of a company process. 
Page 31 GAO-05-858 Navy Enterprise Resource Planning

ERP Has the Potential to 

Address Some of the Navy’s 

Reported Financial 

Management Weaknesses 

The Navy expects the new ERP project to address a number of the 

weaknesses cited in the Department of the Navy Financial Improvement 

Plan—a course of action directed towards achieving better financial 

management and an unqualified audit opinion for the Department of the 

Navy annual financial statements. According to ERP officials, the COTS 

software used for the ERP program will improve the Navy’s current 

financial controls in the areas of asset visibility, financial reporting, and full 

cost accounting. However, the currently planned ERP is not intended to 

provide an all-inclusive end-to-end corporate solution for the Navy.

The COTS software offers the potential for real-time asset visibility for the 

Navy, limited by two factors beyond the project’s scope. First, items in 

transit fall under the authority of the U.S. Transportation Command 

(TRANSCOM). Once the Navy hands off an item to TRANSCOM, it does 

not retain visibility of that asset until it arrives at another Navy location. 

The second factor is the limited ability for communication with ships at 

sea. Once the currently planned ERP is fully implemented, it will cover all 

inventories, including inventory on ships. However, the data for shipboard 

inventory will be current only as of when the ship leaves port. Those data 

will typically not be updated until the ship docks in another port and can 

transmit updated information to the ERP system. This lag time for some 

ships could be as much as 3 to 4 months. While the ERP has the capability 

to maintain real-time shipboard inventory, the Navy has yet to decide 

whether to expand the scope of the ERP and build an interface with the 

ships, which could be extensive and costly, or install the ERP on the ships. 

Both options present additional challenges that necessitate thorough 

analysis of all alternatives before a decision is made. According to the 

program office, a time frame for making this critical decision has not been 

established.

The COTS software is also intended to provide standardized government 

and proprietary financial reporting at any level within the defined 

organization. According to Navy ERP officials, full cost accounting will be 

facilitated by a software component integrated with the ERP. For example, 

the Navy expects that this component will provide up-to-date cost 

information—including labor, materials, and overhead—for its numerous, 

and often complicated, maintenance jobs. Full cost information is 

necessary for effective management of production, maintenance, and other 

activities.
Page 32 GAO-05-858 Navy Enterprise Resource Planning

According to Navy ERP program officials, when fully operational in fiscal 

year 2011, the Navy ERP will be used by organizations comprising 

approximately 80 percent of Navy’s estimated appropriated funds—after 

excluding the Marine Corps and military pay and personnel.45 Based on 

fiscal years’ 2006 through 2011 defense planning budget, the Navy ERP will 

manage approximately $74 billion annually. The organizations that will use 

Navy ERP include the Naval Air Systems, the Naval Sea Systems, the Naval 

Supply Systems, the Space and Naval Warfare Systems, and the Navy 

Facilities Engineering Commands, as well as the Office of Naval Research, 

the Atlantic and Pacific Fleets, and the Strategic Systems Programs. 

However, the Navy ERP will not manage in detail all of the 80 percent. 

About 2 percent, or approximately $1.6 billion, will be executed and 

maintained in detail by respective financial management systems at the 

aviation and shipyard depots. For example, when a work order for a repair 

of an airplane part is prepared, the respective financial management 

system at the depot will execute and maintain the detailed transactions. 

The remaining 20 percent that the Navy ERP will not manage comprises the 

Navy Installations Command, field support activities, and others. Navy 

ERP officials have indicated that it is the Navy’s intent to further expand 

the system in the future to include the aviation and shipyard depots, but 

definite plans have not yet been made. According to Navy ERP officials, 

the software has the capability to be used at the aviation and shipyard 

depots, but additional work would be necessary. For example, the desired 

functionality and related requirements—which as discussed above, are 

critical to the success of any project—would have to be defined for the 

aviation and shipyard depots.

System Interfaces and Data 

Conversion Will Be 

Challenges 

While the Navy’s requirements management process is following 

disciplined processes and comprises one critical aspect of the overall 

project development and implementation, by itself, it is not sufficient to 

provide reasonable assurance of the ERP’s success. Going forward, the 

Navy faces very difficult challenges and risks in the areas of developing and 

implementing 44 system interfaces with other Navy and DOD systems, and 

accurately converting data from the existing legacy systems to the ERP. As 

previously noted, financial management is a high-risk area in the 

department and has been designated as such since 1995. One of the 

45Military pay and personnel will not be included in the Navy ERP because DOD plans to use 

a new system—the Defense Integrated Military Human Resources System (DIMHRS)—to 

process these functions for all DOD components.
Page 33 GAO-05-858 Navy Enterprise Resource Planning

contributing factors has been DOD’s inability to develop integrated 

systems. As a result, the Navy is dependent upon the numerous interfaces 

to help improve the accuracy of its financial management data. Navy ERP 

program managers have recognized the issues of system interfaces and 

data conversion in their current list of key risks. They have identified some 

actions that need to be taken to mitigate the risks; however, they have not 

yet developed the memorandums of agreement with the owners of the 

systems which the Navy ERP will interface. According to the Navy ERP 

program office, they plan to complete these memorandums of agreement 

by October 2005.

Integrated Systems One of the long-standing problems within DOD has been the lack of 

integrated systems. This is evident in the many duplicative, stovepiped 

business systems among the 4,150 that DOD reported as belonging to its 

systems environment. Lacking integrated systems, DOD has a difficult time 

obtaining accurate and reliable information on the results of its business 

operations and continues to rely on either manual reentry of data into 

multiple systems, convoluted system interfaces, or both. These system 

interfaces provide data that are critical to day-to-day operations, such as 

obligations, disbursements, purchase orders, requisitions, and other 

procurement activities. Testing the system interfaces in an end-to-end 

manner is necessary in order for the Navy to have reasonable assurance 

that the ERP will be capable of providing the intended functionality. 

The testing process begins with the initial requirements development 

process. Furthermore, test planning can help disciplined activities reduce 

requirements-related defects. For example, developing conceptual test 

cases based on the requirements can identify errors, omissions, and 

ambiguities long before any code is written or a system is configured. The 

challenge now before Navy ERP is to be sure its testing scenarios 

accurately reflect the activities of the “real users,” and the dependencies of 

external systems.

We previously reported46 that Sears and Wal-Mart, recognized as leadingedge inventory management companies, have automated systems that 

electronically receive and exchange standard data throughout the entire 

inventory management process, thereby reducing the need for manual data 

entry. As a result, information moves through the data systems with 

46GAO, DOD Management: Examples of Inefficient and Ineffective Business Processes, 

GAO-02-873T (Washington, D.C.: June 25, 2002). 
Page 34 GAO-05-858 Navy Enterprise Resource Planning

automated ordering of inventory from suppliers; receiving and shipping at 

distribution centers; and receiving, selling, and reordering at retail stores. 

Unlike DOD, which has a proliferation of nonintegrated systems using 

nonstandard data, Sears and Wal-Mart require all components and 

subsidiaries to operate within a standard systems framework that results in 

an integrated system and does not allow individual systems development.

For the first deployment, the Navy has to develop interfaces that permit the 

ERP to communicate with 44 systems—27 that are Navy specific and 17 

systems belonging to other DOD entities. Figure 3 illustrates the numerous 

required system interfaces.
Page 35 GAO-05-858 Navy Enterprise Resource Planning

Figure 3: Navy ERP Required System Interfaces

Note: See app. III for the definitions of the system acronyms. 
Page 36 GAO-05-858 Navy Enterprise Resource Planning

Long-standing problems regarding the lack of integrated systems and use of 

nonstandard data within DOD pose significant risks for the Navy ERP to 

successfully interface with these systems. Even if integration is successful, 

if the information within the 44 systems is not accurate and reliable, the 

overall information on Navy’s operation provided by the ERP to Navy 

management and the Congress will not be useful in the decision-making 

process. While the Navy ERP project office is working to develop 

agreements with system owners for the interfaces and has been developing 

the functional specifications for each system, officials acknowledged that, 

as of May 2005, they are behind schedule in completing the interface 

agreements due to other tasks. The Navy ERP is dependent on the system 

owners to achieve their time frames for implementation. For example, the 

Defense Travel System (DTS) 47 is one of the DOD systems with which the 

Navy ERP is to interface and exchange data. DTS is currently being 

implemented, and any problems that result in a DTS schedule slippage will, 

in turn, affect Navy ERP’s interface testing. 

We have previously reported that the lack of system interface testing has 

seriously impaired the operation of other system implementation efforts. 

For example, in May 2004, we reported48 that because the system interfaces 

for the Defense Logistics Agency’s Business Systems Modernization (BSM) 

program and the Army’s LMP were not properly tested prior to deployment, 

severe operational problems were experienced. Such problems have led 

BSM, LMP, and organizations with which they interface—such as DFAS—to 

perform costly manual reentry of transactions, which can cause additional 

data integrity problems. For example:

47According to DOD, DTS is expected to reengineer defense travel to a seamless, paperless, 

automated system that meets the needs of individual travelers, force commanders, and 

process owners (such as finance and accounting services). DTS represents a whole new 

way of doing business for government and it is expected to make the travel process faster, 

easier, and better than ever before. During fiscal years 2004-2006, DTS is expected to be 

fielded to more than 250 high-volume sites across the country that serve over 80 percent of 

all DOD travelers.

48GAO-04-615.
Page 37 GAO-05-858 Navy Enterprise Resource Planning

• BSM’s functional capabilities were adversely affected because a 

significant number of interfaces were still in development or were being 

executed manually once the system became operational. Since the 

design of system interfaces had not been fully developed and tested, 

BSM experienced problems with receipts being rejected, customer 

orders being canceled, and vendors not being paid in a timely manner. 

At one point, DFAS suspended all vendor payments for about 2 months, 

thereby increasing the risk of late payments to contractors and 

violations of the Prompt Payment Act.49

• In January 2004, the Army reported that due to an interface failure, LMP 

had been unable to communicate with the Work Ordering and Reporting 

Communications System (WORCS) since September 2003. WORCS is 

the means by which LMP communicates with customers on the status of 

items that have been sent to the depot for repair and initiates 

procurement actions for inventory items. The Army has acknowledged 

that the failure of WORCS has resulted in duplicative shipments and 

billings and inventory items being delivered to the wrong locations. 

Additionally, the LMP program office has stated that it has not yet 

identified the specific cause of the interface failure. The Army is 

currently entering the information manually, which, as noted above, can 

cause additional data integrity errors.

Besides the challenge of developing the 44 interfaces, the Navy ERP must 

also develop the means to be compliant with DOD’s efforts to standardize 

the way that various systems exchange data with each other. As discussed 

in our July 2004 report,50 DOD is undertaking a huge and complex task 

(commonly referred to as the Global Information Grid or GIG) that is 

intended to integrate virtually all of DOD’s information systems, services, 

applications, and data into one seamless, reliable, and secure network. The 

GIG initiative is focused on promoting interoperability throughout DOD by 

building an Internet-like network for DOD-related operations based on 

common standards and protocols rather than on trying to establish 

interoperability after individual systems become operational. DOD 

49The Prompt Payment Act, codified at 31 U.S.C. §§ 3901 – 3907, and as implemented at 5 

C.F.R. pt. 1315 (2005), provides for agencies, among other things, to pay interest and 

penalties under various circumstances for late payments, generally when payments are not 

made within 30 days of the payment due date.

50GAO, Defense Acquisitions: The Global Information Grid and Challenges Facing Its

Implementation, GAO-04-858 (Washington, D.C.: July 28, 2004).
Page 38 GAO-05-858 Navy Enterprise Resource Planning

envisions that this type of network would help ensure systems can easily 

and quickly exchange data and change how military operations are planned 

and executed since much more information would be dynamically available 

to users.

DOD’s plans for realizing the GIG involve building a new core network and 

information capability and successfully integrating the majority of its 

weapon systems; command, control, and communications systems; and 

business systems with the new network. The effort to build the GIG will 

require DOD to make a substantial investment in a new set of core 

enterprise programs and initiatives. To integrate systems such as the Navy 

ERP into the GIG, DOD has developed (1) an initial blueprint or 

architecture for the GIG and (2) new policies, guidance, and standards to 

guide implementation. According to project officials, the Navy ERP system 

will be designed to support the GIG. However, they face challenges that 

can result in significant cost and schedule risks depending on the decisions 

reached. One challenge is the extent to which other DOD applications with 

which the Navy ERP must exchange data are compliant with the GIG. 

While traditional interfaces with systems that are not GIG compliant can be 

developed, these interfaces may suboptimize the benefits expected from 

the Navy ERP. The following is one example of the difficulties faced by the 

Navy ERP project.

As mentioned previously, one system that will need to exchange data with 

the Navy ERP system is DTS. However, the DTS program office and the 

Navy ERP project office hold different views of how data should be 

exchanged between the two systems. The travel authorization process 

exemplifies these differences. DTS requires that funding information and 

the associated funds be provided to DTS in advance of a travel 

authorization being processed. In effect, DTS requires that the financial 

management systems set aside the funds necessary for DTS operations. 

Once a travel authorization is approved, DTS notifies the appropriate 

financial management system that an obligation has been incurred. The 

Navy ERP system, on the other hand, only envisions providing basic 

funding information to DTS in advance, and would delay providing the 

actual funds to DTS until they are needed in order to (1) maintain adequate 

funds control, (2) ensure that the funds under its control are not tied up by 

other systems, and (3) ensure that the proper accounting data are provided 

when an entry is made into its system.

According to the Software Engineering Institute (SEI), a widely recognized 

model evaluating a system of systems interoperability is the Levels of 
Page 39 GAO-05-858 Navy Enterprise Resource Planning

Information System Interoperability. This model focuses on the increasing 

levels of sophistication of system interoperability. According to Navy ERP 

officials, the GIG and the ERP effort are expected to accomplish the 

highest level of this model—enterprise-based interoperability. In essence, 

systems that achieve this level of interoperability can provide multiple 

users access to complex data simultaneously, data and applications are 

fully shared and distributed, and data have a common interpretation 

regardless of format. This is in contrast to traditional interface strategies, 

such as the one used by DTS. The traditional approach is more aligned 

with the lowest level of the SEI model. Data exchanged at this level rely on 

electronic links that result in a simple electronic exchange of data. 

Alignment with DOD’s BEA Is a 

Significant Risk Factor

A broader challenge and risk that is out of the Navy ERP project’s control, 

but could significantly affect it, is DOD’s development of a BEA. As we 

recently reported,51 DOD’s BEA still lacks many of the key elements of a 

well-defined architecture and no basis exists for evaluating whether the 

Navy ERP will be aligned with the BEA and whether it would be a 

corporate solution for DOD in its “To Be” or target environment. 

An enterprise architecture consists of snapshots of the enterprise’s current 

environment and its target environment, as well as a capital investment 

road map for transitioning from the current to the target environment. The 

real value of an enterprise architecture is that it provides the necessary 

content for guiding and constraining system investments in a way that 

promotes interoperability and minimizes overlap and duplication. At this 

time, it is unknown what the target environment will be. Therefore, it is 

unknown what business processes, data standards, and technological 

standards the Navy ERP must align to, as well as what legacy systems will 

be transitioned into the target environment. 

The Navy ERP project team is cognizant of the BEA development and has 

attempted to align to prior versions of it. The project team analyzed the 

BEA requirements and architectural elements to assess Navy ERP’s 

compliance. The project team mapped the BEA requirements to the Navy 

ERP functional areas and the BEA operational activities to the Navy ERP’s 

business processes. The Navy ERP project team recognizes that 

architectures evolve over time, and analysis and assessments will continue 

as requirements are further developed and refined. The scope of the BEA 

and the development approach are being revised. As a result of the new 

51GAO-05-702.
Page 40 GAO-05-858 Navy Enterprise Resource Planning

focus, DOD is determining which products from prior releases of the BEA 

could be salvaged and used. 

Since the Navy ERP is being developed absent the benefit of an enterprise 

architecture, there is limited, if any, assurance that the Navy ERP will be 

compliant with the architecture once it becomes more robust in the future. 

Given this scenario, it is conceivable that the Navy ERP will be faced with 

rework in order to be compliant with the architecture, once it is defined, 

and as noted earlier, rework is expensive. At the extreme, the project could 

fail as the four pilots did. If rework is needed, the overall cost of the Navy 

ERP could exceed the Navy’s current estimate of $800 million. 

Accuracy of Data Conversion Is 

Critical

The ability of the Navy to effectively address its data conversion challenges 

will also be critical to the ultimate success of the ERP effort. A Joint 

Financial Management Improvement Program (JFMIP)52 white paper on 

financial system data conversion53 noted that data conversion (that is, 

converting data in a legacy system to a new system) was one of the critical 

tasks necessary to successfully implement a new financial system. The 

paper further pointed out that data conversion is one of the most frequently 

underestimated tasks.

If data conversion is done right, the new system has a much greater 

opportunity for success. On the other hand, converting data incorrectly or 

entering unreliable data from a legacy system can have lengthy and longterm repercussions. The adage “garbage in, garbage out” best describes the 

adverse impact. Accurately converting data, such as account balances, 

from the pilots, as well as other systems that the Navy ERP is to replace, 

will be critical to the success of the Navy ERP. While data conversion is 

identified in the Navy ERP’s list of key risks, it is too early in the ERP 

system life cycle for the development of specific testing plans. 

52JFMIP was a joint and cooperative undertaking of the Department of the Treasury, GAO, 

the Office of Management and Budget (OMB), and the Office of Personnel Management 

(OPM), working in cooperation with each other and other federal agencies to improve 

financial management practices in the federal government. Leadership and program 

guidance were provided by the four Principals of JFMIP—the Secretary of the Treasury, the 

Comptroller General of the United States, and the Directors of OMB and OPM. Although 

JFMIP ceased to exist as a stand-alone organization as of December 1, 2004, the JFMIP 

Principals will continue to meet at their discretion.

53JFMIP, White Paper: Financial Systems Data Conversion–Considerations (Washington, 

D.C.: Dec. 20, 2002).
Page 41 GAO-05-858 Navy Enterprise Resource Planning

However, our previous audits have shown that if data conversion is not 

done properly, it can negatively impact system efficiency. For example, the 

Army’s LMP data conversion effort has proven to be troublesome and 

continues to affect business operations. As noted in our recent report,54

when the Tobyhanna Army Depot converted ending balances from its 

legacy finance and accounting system—the Standard Depot System 

(SDS)—to LMP in July 2003, the June 30, 2003, ending account balances in 

SDS did not reconcile to the beginning account balances in LMP. Accurate 

account balances are important for producing reliable financial reports. 

Another example is LMP’s inability to transfer accurate unit-of-issue data—

quantity of an item, such as each number, dozen, or gallon—from its legacy 

system to LMP. This resulted in excess amounts of material ordered. 

Similar problems could occur with the Navy ERP if data conversion issues 

are not adequately addressed. The agreements between the Navy ERP and 

the other systems owners, discussed previously, will be critical to 

effectively support Navy’s ERP data conversion efforts.

Additional Actions Can 

be Taken to Improve 

Management Oversight 

of the Navy ERP Effort

Navy officials could take additional actions to improve management 

oversight of the Navy ERP effort. For example, we found that the Navy 

does not have a mechanism in place to capture the data that can be used to 

effectively assess the project management processes. Best business 

practices indicate that a key facet of project management and oversight is 

the ability to effectively monitor and evaluate a project’s actual 

performance, cost, and schedule against what was planned.55 Performing 

this critical task requires the accumulation of quantitative data or metrics 

that can be used to evaluate a project’s performance. This information is 

necessary to understand the risk being assumed and whether the project 

will provide the desired functionality. Lacking such data, the ERP program 

management team can only focus on the project schedule and whether 

activities have occurred as planned, not whether the activities achieved 

their objectives. 

Additionally, although the Navy ERP program has a verification and 

validation function, it relies on in-house subject matter experts and others 

54GAO-05-441.

55GAO, Information Technology: DOD’s Acquisition Policies and Guidance Need to 

Incorporate Additional Best Practices and Controls, GAO-04-722 (Washington, D.C.: July 

30, 2004). 
Page 42 GAO-05-858 Navy Enterprise Resource Planning

who are not independent to provide an assessment of the Navy ERP to 

DOD and Navy management. The use of an IV&V function is recognized as 

a best business practice and can help provide reasonable assurance that 

the system satisfies its intended use and user needs. Further, an 

independent assessment of the Navy ERP would provide information to 

DOD and Navy management on the overall status of the project, including 

the effectiveness of the management processes being utilized and 

identification of any potential risks that could affect the project with 

respect to cost, schedule, and performance. Given DOD’s long-standing 

inability to implement business systems that provide users with the 

promised capabilities, an independent assessment of the ERP’s 

performance is warranted.

Quantitative Data Necessary 

for Assessing Whether the 

System Will Provide the 

Needed Functionality

The Navy’s ability to understand the impact of the weaknesses in its 

processes will be limited because it has not determined the quantitative 

data or metrics that can be used to assess the effectiveness of its project 

management processes. This information is necessary to understand the 

risk being assumed and whether the project will provide the desired 

functionality. The Navy has yet to establish the metrics that would allow it 

to fully understand (1) its capability to manage the entire ERP effort; (2) 

how its process problems will affect the ERP cost, schedule, and 

performance objectives; and (3) the corrective actions needed to reduce 

the risks associated with the problems identified. Experience has shown 

that such an approach leads to rework and thrashing instead of making real 

progress on the project. 

SEI has found that metrics identifying important events and trends are 

invaluable in guiding software organizations to informed decisions. Key 

SEI findings relating to metrics include the following.

• The success of any software organization depends on its ability to make 

predictions and commitments relative to the products it produces.

• Effective measurement processes help software groups succeed by 

enabling them to understand their capabilities so that they can develop 

achievable plans for producing and delivering products and services.
Page 43 GAO-05-858 Navy Enterprise Resource Planning

• Measurements enable people to detect trends and anticipate problems, 

thus providing better control of costs, reducing risks, improving quality, 

and ensuring that business objectives are achieved.56

The lack of quantitative data to assess a project has been a key concern in 

other projects we have reviewed. Without such a process, management 

can only focus on the project schedule and whether activities have 

occurred as planned, not whether the activities achieved their objectives. 

Further, such quantitative data can be used to hold the project team 

accountable for providing the promised capability.

Defect-tracking systems are one means of capturing quantitative data that 

can be used to evaluate project efforts. Although HHS had a system that 

captured the reported defects, we found that the system was not updated in 

a timely manner with this critical information.57 More specifically, one of 

the users identified a process weakness related to grant accounting as a 

problem that will affect the deployment of HHS’s system—commonly 

referred to as a “showstopper.” However, this weakness did not appear in 

the defect-tracking system until about 1 month later. As a result, during 

this interval the HHS defect-tracking system did not accurately reflect the 

potential problems identified by users, and HHS management was unable 

to determine (1) how well the system was working and (2) the amount of 

work necessary to correct known defects. Such information is critical 

when assessing a project’s status. 

We have also reported58 that while NASA had a system that captured the 

defects that have been identified during testing, an analysis was not 

performed to determine the root causes of reported defects. A critical 

element in helping to ensure that a project meets its cost, schedule, and 

performance goals is to ensure that defects are minimized and corrected as 

early in the process as possible. Understanding the root cause of a defect is 

critical to evaluating the effectiveness of a process. For example, if a 

significant number of defects are caused by inadequate requirements 

definition, then the organization knows that the requirements management 

56William A. Florac, Robert E. Park, and Anita D. Carleton, Practical Software 

Measurement: Measuring for Process Management and Improvement (Pittsburgh, Pa.: 

Software Engineering Institute, Carnegie Mellon University, 1997).

57GAO-04-1008.

58GAO-03-507.
Page 44 GAO-05-858 Navy Enterprise Resource Planning

process it has adopted is not effectively reducing risks to acceptable levels. 

Analysis of the root causes of identified defects allows an organization to 

determine whether the requirements management approach it has adopted 

sufficiently reduces the risks of the system not meeting cost, schedule, and 

functionality goals to acceptable levels. Root-cause analysis would also 

help to quantify the risks inherent in the testing process that has been 

selected.

Further, the Navy has not yet implemented an earned value management 

system, which is another metric that can be employed to better manage and 

oversee a system project. Both OMB59 and DOD60 require the use of an 

earned value management system. The earned value management system 

attempts to compare the value of work accomplished during a given period 

with the work scheduled for that period. By using the value of completed 

work as a basis for estimating the cost and time needed to complete the 

program, management can be alerted to potential problems early in the 

program. For example, if a task is expected to take 100 hours to complete 

and it is 50 percent complete, the earned value management system would 

compare the number of hours actually spent to complete the task to the 

number of hours expected for the amount of work performed. In this 

example, if the actual hours spent equaled 50 percent of the hours 

expected, then the earned value would show that the project’s resources 

were consistent with the estimate. Without an effective earned value 

management system, the Navy and DOD management have little assurance 

that they know the status of the various project deliverables in the context 

of progress and the cost incurred in completing each of the deliverables. In 

other words, an effective earned value management system would be able 

to provide quantitative data on the status of a given project deliverable, 

such as a data conversion program. Based on this information, Navy 

management would be able to determine whether the progress of the data 

conversion effort was within the expected parameters for completion. 

Management could then use this information to determine actions to take 

to mitigate risk and manage cost and schedule performance. According to 

Navy ERP officials, they intend to implement the earned value management 

system as part of the contract for the next phase of the project. 

59OMB Circular No. A-11, Part 7, Planning, Budgeting, Acquisition, and Management of 

Capital Assets (June 21, 2005) and the supplement to Part 7, the Capital Programming 

Guide (July 22, 1997).

60Under Secretary of Defense (Acquisition, Technology and Logistics), Department of 

Defense, Revision to DOD Earned Value Management Policy, March 7, 2005.
Page 45 GAO-05-858 Navy Enterprise Resource Planning

Independent Assessment of 

Navy ERP Could Enhance 

DOD and Navy Management 

Oversight of the Project 

The Navy has not established an IV&V function to provide an assessment of 

the Navy ERP to DOD and Navy management. Best business practices 

indicate that use of an IV&V function is a viable means to provide 

management reasonable assurance that the planned system satisfies its 

planned use and users. An effective IV&V review process would provide 

independent information to DOD and Navy management on the overall 

status of the project, including a discussion of any impacts or potential 

impacts to the project with respect to cost, schedule, and performance. 

These assessments involve reviewing project documentation, participating 

in meetings at all levels within the project, and providing periodic reports 

and recommendations, if deemed warranted, to senior management. The 

IV&V function61 should report on every facet of a system project such as:

• Testing program adequacy. Testing activities would be evaluated to 

ensure they are properly defined and developed in accordance with 

industry standard and best practices. 

• Critical-path analysis. A critical path defines the series of tasks that 

must be finished in time for the entire project to finish on schedule. 

Each task on the critical path is a critical task. A critical-path analysis 

helps to identify the impact of various project events, such as delays in 

project deliverables, and ensures that the impact of such delays is 

clearly understood by all parties involved with the project. 

• System strategy documents. Numerous system strategy documents 

that provide the foundation for the system development and operations 

are critical aspects of an effective system project. These documents are 

used for guidance in developing documents for articulating the plans 

and procedures used to implement a system. Examples of such 

documents include the Life-cycle Test Strategy, Interface Strategy, and 

Conversion Strategy.

61According to Institute of Electrical and Electronics Engineers (IEEE), verification and 

validation processes for projects such as Navy ERP can be used to determine whether (1) 

the products of a given activity conform to the requirements of that activity and (2) the 

software satisfies its intended use and user needs. This determination may include 

analyzing, evaluating, reviewing, inspecting, assessing, and testing software products and 

processes. The verification and validation processes should assess the software in the 

context of the system, including the operational environment, hardware, interfacing 

software, operators, and users.
Page 46 GAO-05-858 Navy Enterprise Resource Planning

The IV&V reports should identify the project management weaknesses that 

increase the risks associated with the project to senior management so that 

they can be promptly addressed. The Navy ERP program’s approach to the 

verification and validation of its project management activities relies on inhouse subject matter experts and others who work for the project team’s 

Quality Assurance leader. The results of these efforts are reported to the 

project manager. While various approaches can be used to perform this 

function, such as using the Navy’s approach or hiring a contractor to 

perform these activities, independence is a key component to successful 

verification and validation activities. The system developer and project 

management office may have vested interests and may not be objective in 

their self-assessments. Accordingly, performing verification and validation 

activities independently of the development and management functions 

helps to ensure that verification and validation activities are unbiased and 

based on objective evidence. The Navy’s adoption of verification and 

validation processes is a key component of its efforts to implement the 

disciplined processes necessary to manage this project. However, Navy 

and DOD management cannot obtain reasonable assurance that the 

processes have been effectively implemented since the present verification 

and validation efforts are not conducted by an independent party.

In response to the Ronald W. Reagan National Defense Authorization Act 

for Fiscal Year 2005,62 DOD has established a hierarchy of investment 

review boards from across the department to improve the control and 

accountability over business system investments.63 The boards are 

responsible for reviewing and approving investments to develop, operate, 

maintain, and modernize business systems for their respective business 

62Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005, Pub. L. No. 

108-375, § 332, 118 Stat. 1811, 1851-56 (Oct. 28, 2004) (codified, in part, at 10 U.S.C. §§ 186, 

2222).

63The act requires the use of procedures for ensuring consistency with the guidance issued 

by the Secretary of Defense and the Defense Business Systems Management Committee and 

incorporation of common decision criteria, including standards, requirements, and 

priorities that result in the integration of defense business systems. 
Page 47 GAO-05-858 Navy Enterprise Resource Planning

areas.64 The various boards are to report to the Defense Business Systems 

Management Committee (DBSMC), which is ultimately responsible for the 

review and approval of the department’s investments in its business 

systems. To help facilitate this oversight responsibility, the reports 

prepared by the IV&V function should be provided to the appropriate 

investment review board and the DBSMC to assist them in the decisionmaking process regarding the continued investment in the Navy ERP. The 

information in the reports should provide reasonable assurance that an 

appropriate rate of return is received on the hundreds of millions of dollars 

that will be invested over the next several years and the Navy ERP provides 

the promised capabilities.

To help ensure that the Navy ERP achieves its cost, schedule, and 

performance goals, the investment review should employ an early warning 

system that enables it to take corrective action at the first sign of slippages. 

Effective project oversight requires having regular reviews of the project’s 

performance against stated expectations and ensuring that corrective 

actions for each underperforming project are documented, agreed to, 

implemented, and tracked until the desired outcome is achieved.

Conclusions The lack of management control and oversight and a poorly conceived 

concept resulted in the Navy largely wasting about $1 billion on four ERP 

system projects that had only a limited positive impact on the Navy’s ability 

to produce reliable, useful, and timely information to aid in its day-to-day 

operations. The Navy recognizes that it must have the appropriate 

management controls and processes in place to have reasonable assurance 

that the current effort will be successful. While the current requirements 

management effort is adhering to the disciplined processes, the overall 

effort is still in the early stages and numerous challenges and significant 

risks remain, such as validating data conversion efforts and developing 

numerous systems interfaces. Given that the current effort is not 

scheduled to be complete until 2011 and is currently estimated by the Navy 

64Approval authorities include (1) the Under Secretary of Defense for Acquisition, 

Technology and Logistics; (2) the Under Secretary of Defense (Comptroller); (3) the Under 

Secretary of Defense for Personnel and Readiness; (4) the Assistant Secretary of Defense 

for Networks and Information Integration/Chief Information Officer of the Department of 

Defense; and (5) the Deputy Secretary of Defense or an Under Secretary of Defense, as 

designated by the Secretary of Defense. These approval authorities are responsible for the 

review, approval, and oversight of business systems and must establish investment review 

processes for systems under their cognizance.
Page 48 GAO-05-858 Navy Enterprise Resource Planning

to cost about $800 million, it is incumbent upon Navy and DOD 

management to provide the vigilant oversight that was lacking in the four 

pilots. Absent this oversight, the Navy and DOD run a higher risk than 

necessary of finding, as has been the case with many other DOD business 

systems efforts, that the system may cost more than anticipated, take 

longer to develop and implement, and does not provide the promised 

capabilities. In addition, attempting large-scale systems modernization 

programs without a well-defined architecture to guide and constrain 

business systems investments, which is the current DOD state, presents the 

risk of costly rework or even system failure once the enterprise 

architecture is fully defined. Considering (1) the large investment of time 

and money essentially wasted on the pilots and (2) the size, complexity, 

and estimated costs of the current ERP effort, the Navy can ill afford 

another business system failure.

Recommendations for 

Executive Action

To improve the Navy’s and DOD’s oversight of the Navy ERP effort, we 

recommend that the Secretary of Defense direct the Secretary of the Navy 

to require that the Navy ERP Program Management Office (1) develop and 

implement the quantitative metrics needed to evaluate project performance 

and risks and use the quantitative metrics to assess progress and 

compliance with disciplined processes and (2) establish an IV&V function 

and direct that all IV&V reports be provided to Navy management and to 

the appropriate DOD investment review board, as well as the project 

management. 

Furthermore, given the uncertainty of the DOD business enterprise 

architecture, we recommend that the Secretary of Defense direct the 

DBSMC to institute semiannual reviews of the Navy ERP to ensure that the 

project continues to follows the disciplined processes and meets its 

intended costs, schedule, and performance goals. Particular attention 

should be directed towards system testing, data conversion, and 

development of the numerous system interfaces with the other Navy and 

DOD systems.

Agency Comments and 

Our Evaluation

We received written comments on a draft of this report from the Deputy 

Under Secretary of Defense (Financial Management) and the Deputy Under 

Secretary of Defense (Business Transformation), which are reprinted in 

appendix II. While DOD generally concurred with our recommendations, it 
Page 49 GAO-05-858 Navy Enterprise Resource Planning

took exception to our characterization that the pilots were failures and a 

waste of $1 billion. 

Regarding the recommendations, DOD agreed that it should develop and 

implement quantitative metrics that can be used to evaluate the Navy ERP 

and noted that it intends to have such metrics developed by December 

2005. The department also agreed that the Navy ERP program 

management office should establish an IV&V function and noted that the 

IV&V team will report directly to the program manager. We continue to 

reiterate the need for the IV&V to be completely independent of the project. 

As noted in the report, performing IV&V activities independently of the 

development and management functions helps to ensure that the results 

are unbiased and based on objective evidence. Further, rather than having 

the IV&V reports provided directly to the appropriate DOD investment 

review boards as we recommended, DOD stated that the Navy management 

and/or the project management office shall inform the Office of the Under 

Secretary of Defense for Business Transformation of any significant IV&V 

results. We reiterate our support for the recommendation that the IV&V 

reports be provided to the appropriate investment review board so that it 

can determine whether any of the IV&V results are significant. Again, by 

providing the reports directly to the appropriate investment review board, 

we believe there would be added assurances that the results were objective 

and that the managers who will be responsible for authorizing future 

investments in the Navy ERP will have the information needed to make the 

most informed decision. 

With regard to the reviews by the DBSMC, DOD partially agreed. Rather 

than semiannual reviews by the DBSMC as we recommended, the 

department noted that the components (e.g., the Navy) would provide 

briefings on their overall efforts, initiatives, and systems during meetings 

with the DBSMC. Given the significance of the Navy ERP, in terms of 

dollars and its importance to the overall transformation of the department’s 

business operations, and the failure of the four ERP pilots, we continue to 

support more proactive semiannual reviews by the DBSMC. As noted in 

the report, the Navy’s initial estimate is that the ERP will cost at least $800 

million, and given the department’s past difficulties in effectively 

developing and implementing business systems, substantive reviews by 

individuals outside of the program office that are focused just on the Navy 

ERP by the highest levels of management within the department are 

warranted. Further, we are concerned that the briefings contemplated to 

the DBSMC may not necessarily discuss the Navy ERP, nor provide the 

necessary detailed discussions to offer the requisite level of confidence and 
Page 50 GAO-05-858 Navy Enterprise Resource Planning

assurance that the project continues to follow disciplined processes with 

particular attention to numerous challenges, such as system interfaces and 

system testing.

In commenting on the report, the department depicted the pilots in a much 

more positive light than we believe is merited. DOD pointed out that it 

viewed the pilots as successful, exceeding initial expectations, and forming 

the foundation upon which to build a Navy enterprise solution, and took 

exception to our characterization that the pilots were failures and largely a 

waste of $1 billion. As discussed in the report, the four pilots were narrow 

in scope, and were never intended to be a corporate solution for resolving 

any of the Navy’s long-standing financial and business management 

problems. We characterized the pilots as failures because the department 

spent $1 billion on systems that did not result in marked improvement in 

the Navy’s day-to-day operations. While there may have been marginal 

improvements, it is difficult to ascertain the sustained, long-term benefits 

that will be derived by the American taxpayers for the $1 billion.

Additionally, the pilots present an excellent case study as to why the 

centralization of the business systems funding would be an appropriate 

course of action for the department, as we have previously recommended.65 

Each Navy command was allowed to develop an independent solution that 

focused on its own parochial interest. There was no consideration as to 

how the separate efforts fit within an overall departmental framework, or, 

for that matter, even a Navy framework. As noted in table 2, the pilots 

performed many of the same functions and used the same software, but yet 

were not interoperable because of the various inconsistencies in the design 

and implementation.

Because the department followed the status quo, the pilots, at best, 

provided the department with four more stovepiped systems that perform 

duplicate functions. Such investments are one reason why the department 

reported in February 200566 that it had 4,150 business systems. Further, in 

its comments the department noted one of the benefits of the pilots was 

that they “proved that the Navy could exploit commercial ERP tools 

without significant customization.” Based upon our review and during 

discussions with the program office, just the opposite occurred in the 

65GAO-04-615.

66GAO-05-381.
Page 51 GAO-05-858 Navy Enterprise Resource Planning

pilots. Many portions of the pilots’ COTS software were customized to 

accommodate the existing business processes, which negated the 

advantages of procuring a COTS package. Additionally, the department 

noted that one of the pilots—SMART, on which, as noted in our report, the 

Navy spent approximately $346 million through September 30, 2004—has 

already been retired. We continue to question the overall benefit that the 

Navy and the department derived from these four pilots and the $1 billion it 

spent.

As agreed with your offices, unless you announce the contents of this 

report earlier, we will not distribute it until 30 days after its issuance date. 

At that time, we will send copies to the Chairmen and Ranking Minority 

Members, Senate Committee on Armed Services; Senate Committee on 

Homeland Security and Governmental Affairs; Subcommittee on Defense, 

Senate Committee on Appropriations; House Committee on Armed 

Services; House Committee on Government Reform; and Subcommittee on 

Defense, House Committee on Appropriations. We are also sending copies 

to the Under Secretary of Defense (Comptroller); the Under Secretary of 

Defense (Acquisition, Technology and Logistics); the Under Secretary of 

Defense (Personnel and Readiness); the Assistant Secretary of Defense 

(Networks and Information Integration); and the Director, Office of 

Management and Budget. Copies of this report will be made available to 

others upon request. In addition, the report will be available at no charge 

on the GAO Web site at http://www.gao.gov. If you or your staff have any 
Page 52 GAO-05-858 Navy Enterprise Resource Planning

questions on matters discussed in this report, please contact Gregory D. 

Kutz at (202) 512-9505 or kutzg@gao.gov or Keith A. Rhodes at (202) 512-

6412 or rhodesk@gao.gov. Key contributors to this report are listed in 

appendix IV. Contact points for the Offices of Congressional Relations and 

Public Affairs are shown on the last page of the report.

Gregory D. Kutz

Managing Director

Forensic Audits and Special Investigations

Keith A. Rhodes

Chief Technologist

Applied Research and Methodology Center for Engineering and Technology
Page 53 GAO-05-858 Navy Enterprise Resource Planning

List of Requesters

The Honorable Tom Davis

Chairman

Committee on Government Reform

House of Representatives

The Honorable Christopher H. Shays

Chairman

Subcommittee on National Security, Emerging Threats,

and International Relations

Committee on Government Reform

House of Representatives

The Honorable Todd R. Platts

Chairman

Subcommittee on Government Management, Finance

and Accountability

Committee on Government Reform

House of Representatives

The Honorable Adam H. Putnam

House of Representatives
Page 54 GAO-05-858 Navy Enterprise Resource Planning

Appendix I

Scope and Methodology ApApenednxidexiIs

To obtain a historical perspective on the planning and costs of the Navy’s 

four Enterprise Resource Planning (ERP) pilot projects, and the decision to 

merge them into one program, we reviewed the Department of Defense’s 

(DOD) budget justification materials and other background information on 

the four pilot projects. We also reviewed Naval Audit Service reports on 

the pilots. In addition, we interviewed Navy ERP program management 

and DOD Chief Information Officer (CIO) officials and obtained 

informational briefings on the pilots. 

To determine if the Navy has identified lessons learned from the pilots, how 

they are being used, and the challenges that remain, we reviewed program 

documentation and interviewed Navy ERP program officials. Program 

documentation that we reviewed included concept of operations 

documentation, requirements documents, the testing strategy, and the test 

plan. In order to determine whether the stated requirements management 

processes were effectively implemented, we performed an in-depth review 

and analysis of seven requirements that relate to the Navy’s problem areas, 

such as financial reporting and asset management, and traced them 

through the various requirements documents. These requirements were 

selected in a manner that ensured that the requirements selected were 

included in the Navy’s Financial Improvement Plan. 

Our approach to validating the effectiveness of the requirements 

management process relied on a selection of seven requirements from 

different functional areas. From the finance area, we selected the 

requirement to provide reports of funds expended versus funds allocated. 

From the intermediate-level maintenance management area, we selected 

the requirement related to direct cost per job and forecasting accuracy. 

From the procurement area, we selected the requirement to enable 

monitoring and management of cost versus plan. In the plant supply 

functions area, we reviewed the requirement related to total material 

visibility and access of material held by the activity and the enterprise. 

From the wholesale supply functions area, we selected the requirements of 

in-transit losses/in-transit write-offs and total material visibility and access 

of material held by the activity and the enterprise. Additionally, we 

reviewed the requirement that the ERP be compliant with federal mandates 

and requirements and the U.S. Standard General Ledger. 

In order to provide reasonable assurance that our test results for the 

selected requirements reflected the same processes used to document all 

requirements, we did not notify the project office of the specific 

requirements we had chosen until the tests were conducted. Accordingly, 
Appendix I

Scope and Methodology

Page 55 GAO-05-858 Navy Enterprise Resource Planning

the project office had to be able to respond to a large number of potential 

requests rather than prepare for the selected requirements in advance. 

Additionally, we obtained the list of systems the Navy ERP will interface 

with and interviewed selected officials responsible for these systems to 

determine what activities the Navy ERP program office is working with 

them on and what challenges remain. 

To determine if there are additional business practices that could be used 

to improve management oversight of the Navy ERP, we reviewed industry 

standards and best practices from the Institute of Electrical and 

Electronics Engineers, the Software Engineering Institute, the Joint 

Financial Management Improvement Program, GAO executive guides, and 

prior GAO reports. Given that the Navy ERP effort is still in the early 

stages of development, we did not evaluate all best practices. Rather, we 

concentrated on those that could have an immediate impact in improving 

management’s oversight. We interviewed Navy ERP program officials and 

requested program documentation to determine if the Navy ERP had 

addressed or had plans for addressing these industry standards and best 

practices. 

We did not verify the accuracy and completeness of the cost information 

provided by DOD for the four pilots or the Navy ERP effort. We conducted 

our work from August 2004 through June 2005 in accordance with U.S. 

generally accepted government auditing standards. 

We requested comments on a draft of this report from the Secretary of 

Defense or his designee. We received written comments on a draft of the 

report from the Deputy Under Secretary of Defense (Financial 

Management) and the Deputy Under Secretary of Defense (Business 

Transformation), which are reprinted in appendix II. 
Page 56 GAO-05-858 Navy Enterprise Resource Planning

Appendix II

Comments from the Department of Defense ApendxiI
Appendix II

Comments from the Department of Defense

Page 57 GAO-05-858 Navy Enterprise Resource Planning
Appendix II

Comments from the Department of Defense

Page 58 GAO-05-858 Navy Enterprise Resource Planning
Appendix II

Comments from the Department of Defense

Page 59 GAO-05-858 Navy Enterprise Resource Planning
Page 60 GAO-05-858 Navy Enterprise Resource Planning

Appendix III

Identification of the Navy and Defense 

Systems That Must Interface with the ERP ApendxiI

Acronym System name

Navy systems

AIT Automated Information Technology

CAV II Commercial Asset Visibility System

CDF Consolidated Data File

CDMD-OA Configuration Data Manager’s Database – Open Architecture

CRCS-CADS Common Rates Computation System/Common Allowance Development System

DONIBIS Department of the Navy Industrial Budget Information System

G02APU G02 Annual Pricing Update

ITIMP Integrated Technical Item Management & Procurement

Manugistics Manugistics

MSWP Maintenance and Ship Work Planning

NALCOMIS OMA & OOMA Naval Aviation Logistic Command Management Information System (2 different versions)

NALDA Naval Aviation Logistics Data Analysis

Navy Data Mart Defense Civilian Personnel Data System 

NDMS NAVAIR Depot Maintenance System

NES Navy Enlisted Personnel System

One Touch One Touch Supply System

OPINS Officer Personnel Information System

PBIS Program Budget Information System

RMAIS Regional Maintenance Automated Information System

SAS SAS Activity-Based Management

SDRS Supply Discrepancy Reporting System

SKED Preventative Maintenance Scheduling Program 

SLDP Standard Logistics Data Procedures

TDSA Technical Directive Status Accounting System 

TFMMS Total Force Manpower Management System

UADPS- SP/U2 Uniform Automated Data Processing System - Stock Points
Appendix III

Identification of the Navy and Defense 

Systems That Must Interface with the ERP

Page 61 GAO-05-858 Navy Enterprise Resource Planning

Source: Navy ERP program office.

DOD systems

ADS DFAS Corporate Database

APVM Accounting Pre-Validation Module

CCR Central Contractor Registration System

DAAS Defense Automatic Addressing System

DCAS Defense Cash Accountability System 

DCPS Defense Civilian Pay System

DDRS Defense Departmental Reporting System 

DTS Defense Travel System

FLIS Federal Logistics Information System

FRS Financial Reporting System - Accounting

ICAPS Interactive Computer Aided Provisioning System

MISIL Management Information System for International Logistics

PPVM Payment Pre-Validation Module

SPS Standard Procurement System

VPIS Vendor Pay Inquiry System 

WAWF Wide Area Workflow

WINS Web Based Invoicing System 

(Continued From Previous Page)

Acronym System name
Page 62 GAO-05-858 Navy Enterprise Resource Planning

Appendix IV

GAO Contacts and Acknowledgments ApendxiIV

GAO Contacts Gregory D. Kutz (202) 512-9505 or kutzg@gao.gov

Keith A. Rhodes (202) 512-6412 or rhodesk@gao.gov

Acknowledgments In addition to the contacts above, Darby Smith, Assistant Director; J. 

Christopher Martin, Senior Level Technologist; Francine DelVecchio; Kristi 

Karls; Jason Kelly; Mai Nguyen; and Philip Reiff made key contributions to 

this report.

(192171)
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

Obtaining Copies of 

GAO Reports and 

Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To 

have GAO e-mail you a list of newly posted products every afternoon, go to 

www.gao.gov and select “Subscribe to Updates.”

Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to:

U.S. Government Accountability Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Congressional 

Relations

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548

Public Affairs Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548