Report to Congressional Committees

United States General Accounting Office

GAO

June 2001 INFORMATION

TECHNOLOGY

DLA Should

Strengthen Business

Systems

Modernization

Architecture and

Investment Activities

GAO-01-631
Page i GAO-01-631 Information Technology

Letter 1

Results in Brief 2

Background 4

DLA Has Developed a Strategic Plan and Logistics Transformation

Plan 9

BSM Is Not Being Guided by Either a DLA Architecture or a DODwide Logistics Architecture 14

DLA Has Not Invested Incrementally in BSM 26

Conclusions 31

Recommendations 31

Agency Comments and Our Evaluation 33

Appendix I Objectives, Scope, and Methodology 38

Appendix II Comments From the Department of Defense 40

Appendix III GAO Contact and Staff Acknowledgment 59

Tables

Table 1: DLA Field Command Management Responsibilities 6

Table 2: Materiel Management Functions Performed by Legacy

Systems 7

Table 3: BSM Investment According to Fiscal Year 2001 Budget

Submission 11

Table 4: Seven Essential Products for the DOD C4ISR Architecture

Framework 19

Figures

Figure 1: BSM Timeline 12

Figure 2: BSM Within DLA’s Organization and Oversight Structure 14

Figure 3: C4ISR Architecture Framework: Views and Description 17

Figure 4: DLA’s Approach to Creating an Enterprise Architecture 20

Contents
Page ii GAO-01-631 Information Technology

Abbreviations

BSM Business Systems Modernization

C4ISR Command, Control, Communications, Computers,

 Intelligence, Surveillance, and Reconnaissance

CIO Chief Information Officer

COTS commercial, off-the-shelf

DISMS Defense Integrated Subsistence Management System

DLA Defense Logistics Agency

DLA 21 Strategic Plan 2000: DLA 21

DOD Department of Defense

DPACS Defense Pre-award Contracting System

DT developmental test

ERP enterprise resource planning

FOC full operating capability

IOC initial operating capability

IOT&E initial operational test and evaluation

IT information technology

NATO North Atlantic Treaty Organization

OA operational assessment

OMB Office of Management and Budget

REL release

SAMMS Standard Automated Materiel Management System

SPEDE SAMMS Procurement by Electronic Data Exchange

TRR test readiness review

Y2K Year 2000
Page 1 GAO-01-631 Information Technology

June 29, 2001

The Honorable Carl Levin

Chairman

The Honorable John Warner

Ranking Minority Member

Committee on Armed Services

United States Senate

The Honorable Bob Stump

Chairman

The Honorable Ike Skelton

Ranking Minority Member

Committee on Armed Services

House of Representatives

The Defense Logistics Agency (DLA) plays a critical role in supporting

America’s military forces worldwide. To fulfill this role, DLA employs

about 28,000 civilian and military workers, located at about 500 sites in all

50 states and 28 countries; in round numbers, it manages 4 million supply

items and processes 30 million annual supply distribution actions. In fiscal

year 2000, DLA reported that these operations resulted in sales to the

military services of about $13 billion.

This report is one in a series of products to satisfy our mandate under the

fiscal year 2001 Defense Authorization Act.1

 The act directed that we

review DLA’s efficiency and effectiveness in meeting customer

requirements, application of best business practices, and opportunities for

improving DLA operations. As agreed with your offices, our first review of

DLA’s information technology (IT) management practices focused on

DLA’s $900 million Business Systems Modernization (BSM) acquisition. As

further agreed with your offices, our objectives were to determine

(1) whether DLA is using an enterprise architecture to guide and constrain

its investment in BSM and (2) whether DLA is investing in BSM in an

incremental manner. Using enterprise architectures—institutional

blueprints for business and technological change—and investing

incrementally in large modernization programs are legislative and federal

 1

 P.L. 106-398, Floyd D. Spence National Defense Authorization Act for Fiscal Year 2001,

section 917.

United States General Accounting Office

Washington, DC 20548
Page 2 GAO-01-631 Information Technology

requirements2 and best industry practices. This review did not address

other system modernization best practices, such as whether DLA is

employing effective system acquisition process controls, and did not

evaluate the specific commercial, off-the-shelf product that DLA chose as

its BSM system solution. Details on our objectives, scope, and

methodology are in appendix I.

DLA does not have an enterprise architecture to guide its investment in

BSM, even though the law and Office of Management and Budget (OMB)

guidance recognize the importance of enterprise architectures and

Department of Defense (DOD) policy requires their use. Rather, DLA plans

call for creating an architecture as a by-product of BSM’s implementation.

Moreover, DLA’s architecture development plans address only one, albeit

the largest, of its six primary business areas—materiel management (also

called supply-chain management). According to DLA’s plans, its

architectural products will not be extended to its other business areas

until 5 years from now. This nonagencywide approach to developing and

implementing an enterprise architecture is not consistent with federal

guidance, and it increases the risk that DLA will modernize in a way that

optimizes an individual business area but does not optimize agencywide

logistics management performance and accountability. Further, DLA does

not have effective management structures and processes to support

architecture development, as defined in Chief Information Officers (CIO)

Council guidance.3

 At the completion of our review, DLA officials stated

that they would modify their plans and expeditiously pursue an

agencywide architecture development and implementation approach,

consistent with federal and DOD policies and guidance.

Equally if not more important, DOD has not developed a DOD-wide

logistics management enterprise architecture that would promote

interoperability and avoid duplication among the logistics modernization

efforts now under way in DOD component organizations, such as DLA and

the services. To its credit, the Office of the Deputy Under Secretary of

Defense for Logistics and Materiel Readiness (the office responsible for

DOD-wide logistics policy and strategy) has begun efforts to correct this

 2

 Clinger-Cohen Act of 1996, P.L. 104-106; Management of Federal Information Resources,

Office of Management and Budget Circular A-130 (November 30, 2000).

3 Architecture Alignment and Assessment Guide, Chief Information Officers Council

(October 2000).

Results in Brief
Page 3 GAO-01-631 Information Technology

architectural void, and it has initiated interim steps intended to coordinate

the new logistics management system investments that DLA and the

services have under way. However, according to officials in this office,

plans for completing this architecture are not sufficiently advanced to be

provided to us for evaluation. Moreover, the interim steps that these

officials described do not provide the management control rigor and

discipline that well-defined and effectively implemented enterprise

architectures provide. By allowing the services and DLA (through BSM) to

proceed separately with new logistics management systems in the absence

of a DOD-wide enterprise architecture, DOD will not be in a position to

optimize logistics operations and system performance across the

department, and thus is unlikely to successfully meet its strategic logistics

management goals. At the conclusion of our review, officials in the Deputy

Under Secretary’s office acknowledged the need to move swiftly in

developing and using a DOD-wide logistics management architecture.

In addition to the enterprise architecture issues affecting BSM, DLA has

not been managing its investment in this program in an incremental

manner; that is, DLA has not treated the first of its four planned

incremental releases of BSM as a separate investment decision, justified

on the basis of release-specific analyses of costs, benefits, and risks.

Instead, DLA has thus far treated the entire BSM program as a single

investment decision, justified by a single, “all or nothing” economic

analysis, because doing so, according to BSM officials, was consistent with

DOD policy then in place for major system acquisitions.4

 At the completion

of our review, DLA officials told us that they now plan to take an

incremental investment approach for future releases: that is, they plan to

measure the actual accrual of benefits versus costs on each incremental

release (including the first) and to use this information to make

investment decisions on future increments. This change in BSM

investment strategy is important because an “all or nothing” approach to

investment decisions is not consistent with statutory requirements and

federal guidance for incremental investment management and has led

agencies to invest huge sums in systems without a commensurate return.

To strengthen DLA management of BSM, we are making recommendations

(consistent with officials’ stated commitments) concerning the effective

development and use of the DOD-wide and DLA architectural context

needed for guiding BSM, and concerning the need for incremental

 4

 DOD Instruction 5000.2, Operation of the Defense Acquisition System.
Page 4 GAO-01-631 Information Technology

investment in BSM based on economic justification and validation of

expected return on investment.

In commenting on a draft of this report, DOD agreed with the importance

of using enterprise architectures and making BSM investment decisions

incrementally. DOD also acknowledged that the absence of these IT

management controls increased program risk. Further, DOD agreed with

eight of our ten recommendations. However, DOD disagreed with our

recommendations to limit investment in BSM beyond the first incremental

release and to use actual benefit accrual on the first incremental release in

deciding on investment in the next release, noting that this would delay

BSM progress. According to DOD, our findings and conclusions regarding

the overall level of risk to the successful achievement of BSM and other

DOD logistics modernization program objectives are not supported by the

single risk factor addressed in this report—the use of enterprise

architectures. DOD also contended that all program risks are being

effectively mitigated.

Notwithstanding the considerable level of agreement between DOD and us

on this report, we have a differing view on the level of risk facing DOD

logistics modernization programs in general, and BSM in particular, arising

from the two risk factors addressed in this report: (1) the lack of effective

enterprise architecture development and implementation and (2) the lack

of effective incremental investment management. Our experience

reviewing other federal agency modernization programs is that the use of

enterprise architectures, employed in concert with other important IT

management controls (such as incremental investment management),

greatly increases the chances of successful modernization. Accordingly,

their absence greatly reduces the chance of success. While we have

modified our recommendations slightly and have clarified appropriate

sections in the report to address DOD’s concerns about delaying BSM

progress until these IT management controls are in place, the substance of

the recommendations is unchanged.

DLA is DOD’s logistics manager for all DOD consumable items5

 and some

DOD repair items;6

 its primary business function is providing supply

 5

 Consumable items include such commodities as subsistence (food), fuels, medical

supplies, clothing, and construction equipment.

6

 These repair items are spare and repair parts that support about 1,400 DOD weapon

systems. Each of the military services also manages its own service-unique repair items.

Background
Page 5 GAO-01-631 Information Technology

support to sustain military operations and readiness. In addition to this

primary function, which DLA refers to as either materiel management or

supply-chain management, DLA performs five other major business

functions: distributing materiel ordered from its inventory, purchasing

fuels for DOD and the U.S. government, storing strategic materiel,7

marketing surplus DOD materiel for reuse and disposal, and providing

numerous information services, such as item cataloging,8

 for DOD and the

U.S. and selected foreign governments.

DLA consists of a central command authority supported by a number of

field commands that manage the agency’s six business functions. Table 1

shows DLA’s field commands, their respective business functions, and

examples of the consumable and/or repair items each manages.

 7

 Strategic materiel is defined as any item or materials needed to sustain the United States

in the event of a national emergency.

8

 DLA defines item cataloging as including all activities that describe the technical

characteristics and data for an individual item of supply.
Page 6 GAO-01-631 Information Technology

Table 1: DLA Field Command Management Responsibilities

Field command Business function Examples of items managed

Defense Supply Centersa Materiel management: Procuring consumable items (except

fuels) and some DOD repair parts supporting weapons

systems

Food, medical supplies, clothing,

construction equipment, spare and repair

parts

Defense Distribution Center Distribution: Receiving, storing, packaging, and shipping

consumable items (and some DOD repair parts supporting

weapons systems) procured by the Defense Supply

Centers

(Same as above)

Defense Energy Support

Center

Energy services: Procuring, storing, and shipping fuel and

other energy sources for the federal government

Petroleum, jet fuel, coal

Defense National Stockpile

Center

Strategic materiel management: Procuring, selling, and

storing strategic and critical materiel

Metals, minerals, and ores (e.g.,

aluminum, chromium, diamonds)

Defense Reutilization and

Marketing Services

Materiel reuse: Managing the reuse and disposal of excess

DOD materiel

Aircraft parts, automobiles, clothing,

typewriters, furniture

Defense Logistics

Information Service

Information services: Providing services, such as item

cataloging, for DOD, the U.S. government, and

international organizations (NATO and allied militaries)

(All of the above)

a The three Defense Supply Centers are in Columbus, Philadelphia, and Richmond.

Source: DLA Customer Assistance Handbook, 1998.

To support the current materiel management business function, DLA

reports that it relies on a collection of mainframe-based Cobol systems,

which are not integrated and are more than 30 years old. These legacy

systems support DLA’s traditional mission as a manager of physical

inventory; that is, they support materiel management functions such as

acquisition/ procurement, inventory control, requirements forecasting,

requisition processing, technical data maintenance, and financial

management. The systems consist of two primary systems—the Standard

Automated Materiel Management System (SAMMS) and the Defense

Integrated Subsistence Management System (DISMS)—and two subsidiary

systems—SPEDE (SAMMS Procurement by Electronic Data Exchange)

and DPACS (Defense Pre-award Contracting System). The functions of

these systems are summarized in table 2.
Page 7 GAO-01-631 Information Technology

Table 2: Materiel Management Functions Performed by Legacy Systems

Materiel management function SAMMSaDISMSbSPEDEcDPACSd

Acquisition/procurement x x x x

Inventory control x x x

Requirements forecasting x x

Requisition processing x x x x

Technical data maintenance x x

Financial management x x x x

a Primary system supporting inventory management; provides information regarding stock levels,

acquisition and management of wholesale consumable items, direct support for processing

requisitions, generation of purchase requests, identification of items, asset visibility, and maintenance

of an audit trail of transactions processed.

b Primary system supporting the worldwide wholesale food business in support of troop feeding and

commissary resale; in supporting these commodities, performs the same functions as SAMMS.

c Subsystem giving DLA Supply Centers the capability to interface with vendors electronically; allows

electronic transfer of requests for quotation, vendor responses, invoices, and purchase orders.

d Subsystem performing precontracting functions; provides purchase request management data that

buyers need to complete a solicitation and award package, including item descriptions, applicable

specifications/drawings, synopsis information, histories of past procurements, and other purchase

requests and/or contracts open for the same National Stock Number.

Source: DLA Y2K Program Software Development Plan.

DLA reports that these legacy systems are the product of decades of

accumulated and divergent business practices and use obsolete

technology that is no longer supported by the original equipment

manufacturers and the software support provider. Further, DLA maintains

that these systems, consisting of several million lines of unstructured and

expensive-to-maintain code, provide inadequate analytical capability and

no real-time data access.

DOD’s logistics management strategy is set forth in two strategic planning

documents: Joint Vision 2020 9

 and 21st Century Logistics: DOD

Logistics Strategic Plan.

10 Joint Vision 2020 is DOD’s primary strategic

plan, covering all missions and business areas. This plan introduced what

DOD calls focused logistics, described as the ability to provide the joint

 9 Joint Vision 2020, published in June 2000, was preceded by Joint Vision 2010, published

in 1996. It was issued by the Chairman, Joint Chiefs of Staff.

10 21st Century Logistics, published in August 1999, defines more specifically the strategic

vision and objectives for logistics management. It was issued by the Deputy Under

Secretary of Defense for Logistics and Materiel Readiness.

DOD Has Issued Strategic

Plans for Modernizing

Logistics Management
Page 8 GAO-01-631 Information Technology

force11 the right assets (personnel, equipment, and supplies) in the right

place, at the right time, and in the right quantity. To achieve this goal, DOD

envisions a real-time, Web-based12 information system providing both

logisticians and field commanders with total asset visibility within a given

theater of operations. Additionally, this system is to include decision

support tools to improve requirements analysis and planning, as well as

provide real-time control of the logistics supply chain, regardless of

whether the requirements are to be fulfilled by the commercial sector or

within DOD.

21st Century Logistics calls for the military services and DLA to pursue

system modernization efforts to bring about the focused logistics goals

described in Joint Vision 2020. The systems are to be an integrated

collection of applications sharing a common data environment, which are

to be deployed to all forces by fiscal year 2006.

To fulfill Joint Vision 2020, the Deputy Secretary of Defense issued a

memorandum in March 2000 to the military services and DLA requiring

submission of logistics transformation plans, documenting, on an annual

basis, planned actions and related resources for implementing focused

logistics, as well as any other logistics initiatives. The military services and

DLA were directed to satisfy the 21st Century Logistics objectives by

fiscal year 2006, including developing logistics process and system

modernization plans by the end of fiscal year 2001 and implementing these

plans by fiscal year 2006. DLA was directed to submit its transformation

plan to the Under Secretary of Defense for Acquisition, Technology, and

Logistics by July 1, 2000.

 11 Joint force is a general term applied to a force composed of significant elements,

assigned or attached, of two or more military departments, operating under a single joint

force commander.

12 According to DLA, real-time, Web-based means immediate computer responsiveness

through the Internet or an intranet.
Page 9 GAO-01-631 Information Technology

To fulfill its direction under DOD’s 21st Century Logistics plan, DLA has

outlined strategic goals and objectives in its Strategic Plan 2000: DLA 2113

and Logistics Transformation Plan.

14 Under Goal 2 of DLA 21, DLA plans

to reduce costs, improve efficiency, and increase effectiveness through

organizational redesign, business systems modernization, strategic

sourcing, infrastructure consolidation, and optimally sized inventories.

To achieve this goal, DLA is first focusing on transforming its materiel

management (supply-chain management) function, because according to

the DLA Director, this is DLA’s most critical business function. More

specifically, it plans to implement a Web/network-based logistics system

using commercial, off-the-shelf (COTS) products.

To select the most appropriate COTS products, DLA evaluated potential

solutions against business processes for certain of its commodities

(including clothing, aircraft parts, and medical supplies). According to

DLA, it chose important, widely differing commodities so that its analysis

would be sufficiently comprehensive to be extrapolated to the rest of the

commodities within the materiel management business function. Further,

DLA chose to evaluate solutions against business processes that had been

previously reengineered, so that the system solution chosen would be

based on modernized business processes. Through this analysis, which

used scripted demonstrations of the reengineered requirements, DLA

sought to ensure that the chosen applications could effectively meet DLA’s

materiel management/supply-chain management requirements.

The resulting COTS-based system is to be used to reengineer the agency’s

materiel management business processes. DLA refers to its acquisition and

implementation of this COTS-based system, and the associated business

process reengineering, as Business Systems Modernization (BSM).

DLA’s BSM program is intended to modernize DLA’s materiel management

business function, changing DLA from solely a provider and manager of

physical inventory to primarily a manager of supply chains. In this role,

DLA would link customers with appropriate suppliers and track physical

and financial assets. DLA expects this approach to enhance supply-chain

 13 DLA refers to this plan, published in September 1999, as DLA 21.

14 DLA’s Logistics Transformation Plan was approved by the Under Secretary of Defense

for Acquisition, Technology, and Logistics on July 11, 2000.

DLA Has Developed a

Strategic Plan and

Logistics

Transformation Plan

DLA Has Begun

Implementing its Business

Systems Modernization

Program
Page 10 GAO-01-631 Information Technology

visibility and capabilities, resulting in reduced logistics cycle times,

increased customer satisfaction, reduced customer wait time, and

decreased materiel costs. In short, DLA’s goal is “to do business as

business does business.”

According to BSM program documents, DLA’s transformation from its

current to its future role requires modernization of the IT systems that

now support DLA’s materiel management business function, subfunctions,

and processes. The BSM system solution is envisioned as consisting of

Web- and COTS-based applications supported by an enterprise application

tool.15 Through the use of COTS products, DLA expects to reduce materiel

management systems operating costs and take advantage of the

continuous technology insertion and continuous functional improvement

that COTS packages allow. The BSM COTS will consist of enterprise

resource planning (ERP)16 software.

DLA plans to acquire and deploy its BSM COTS system solution through a

series of four system releases/increments. First, it plans to demonstrate

successful application of its new concept of doing business to selected

commodities, namely, earth-moving equipment, medical/pharmaceutical

supplies, and F/A-18 engine components. These commodities were chosen

for the first release because each is an important responsibility of one of

the three Defense Supply Centers. Thus, the first release will be deployed

to all three Defense Supply Centers at once. Once this first release is

successfully demonstrated, DLA plans to expand the system solution to

other commodities and to other users within the materiel management

business function.

According to DLA’s fiscal year 2001 budget submission, DLA will invest

approximately $897 million to acquire and implement BSM from fiscal

years 2000 through 2007 (see table 3). Thus far, DLA has obligated about

$150 million for BSM. From this total, about $56 million was obligated in

 15 According to DOD, this enterprise application tool is an integration tool that effects

application-to-application interfacing, supports new “business-to-business” relationships,

and mitigates the risk of introducing new business functions into the BSM architecture.

16 ERP software packages provide a suite of software applications and modules, usually

employing a common relational database, that provide functionality for managing a range

of tasks, such as planning, purchasing, maintaining inventories, interacting with suppliers,

providing customer service, and tracking orders. These packages are intended to provide a

generic set of enterprise management applications that can be used to reengineer business

operations.
Page 11 GAO-01-631 Information Technology

fiscal year 2000 to prepare and validate functional requirements, conduct

pre-award contract functions for selection of the systems integration

contractor, perform initial work on the program by awarding the

integration contract, and begin detailed business process reengineering

planning for the first increment of the program.

Table 3: BSM Investment According to Fiscal Year 2001 Budget Submission

Fiscal year

Dollarsa

(in millions)

Cumulative percentage

of total

2000 $56 6

2001 93 17

2002 104 28

2003 151 45

2004 176 65

2005 116 78

2006 101 89

2007 100 100

Total $897

a Costs include $183 million for maintenance.

Source: DLA budget for fiscal year 2001.

The remaining approximately $93 million was obligated in fiscal year 2001

for the development of a materiel management enterprise architecture

(referred to by DLA as the BSM Business Architecture Blueprint) and for

the initial implementation, including testing, of the BSM concept

demonstration. Of the unobligated portion of the BSM budget, about $549

million is to fund implementation and sustainment of the COTS solution

(system releases 1 through 4) within the materiel management business

area, including its extension to the commodities not covered by the

concept demonstration. DLA officials told us that the remaining

unobligated funds for fiscal years 2006 and 2007, about $201 million, are to

be used for extending the BSM ERP software to the distribution and

cataloging business areas.

Only about 1 percent of the BSM budget is for new network and

computing platform infrastructure (hardware and systems software). This

is because DLA’s current and future investments in IT infrastructure,

which will support ongoing DLA operations as well as the BSM program,

are funded separately. The infrastructure investments in fiscal year 2001

amount to about 45 percent of DLA’s approximately $680 million IT

budget.
Page 12 GAO-01-631 Information Technology

BSM is expected to take 6 years to acquire and implement for materiel

management (fiscal years 2000 through 2005). DLA has structured the BSM

life cycle to comply with DOD 5000.217 milestone phases for major systems

acquisitions. Figure 1 summarizes the BSM development timeline.

Figure 1: BSM Timeline

Legend

IOC: initial operating capability FOC: full operating capability

REL: release OA: operational assessment

TRR: test readiness review DT: developmental test

IOT&E: initial operational test and evaluation

Source: BSM program documents.

The first milestone phase (phase 0, December 1999 through July 2000) of

the BSM program focused on evaluation of alternative concepts and

potential system integrators.

The second and current milestone phase (phase I/IIA18) covers program

definition (design), risk reduction, and concept demonstration. The

current phase includes implementation and testing/evaluation of the

system solution for certain commodities, as well as such implementation

 17 DOD Instruction 5000.2, Operation of the Defense Acquisition System.

18 BSM is following a modified acquisition process, as allowed by DOD Instruction 5000.2.

The traditional acquisition process consists of four discrete phases: 0 (concept

exploration), I (program definition and risk reduction), II (engineering and manufacturing

development), and III (production, fielding/deployment, and operational support).

FY 2000

Quarters:

Process setting:

Audits/reviews:

2 123 3 123 123 123 1234 4 4 4 4 4

BSM major

milestones

Concept

Demo (CD)

Milestone

II/III

Milestone I/IIA

Contract award

Milestone 0 FOC

Design

OA TRR

IOC

REL 2 REL 3 REL 4

FY 2001 FY 2002 FY 2003 FY 2004 FY 2005

REL 1

REL 4 test cycle

CD DT IOT&E REL 3 test cycle

REL 2 test cycle
Page 13 GAO-01-631 Information Technology

activities as training and site preparation. This phase is scheduled to run

from September 2000 through September 2002.

The third phase (phase II/III) is referred to as initial fielding/deployment

and operational support. This phase focuses on implementing the system

beyond the scope of the concept demonstration, operating and

maintaining the system, and modifying and upgrading the system as

required. This phase is scheduled to run from October 2002 through March

2005.

Investment decisionmaking for BSM (and other DLA major modernization

programs) is vested in the DLA 21 Executive Board, chaired by the Chief

Acquisition Executive of DLA, who is the Vice Director. BSM life-cycle

management is the responsibility of the BSM Program Manager, who runs

the BSM program office and reports to the DLA Program Executive

Officer,19 who oversees the program in coordination with the DLA Chief

Information Officer (CIO). Figure 2 shows this BSM program management

and oversight structure.

 19 Under DOD 5000 regulations, Program Executive Officers are to oversee major systems

acquisitions.
Page 14 GAO-01-631 Information Technology

Figure 2: BSM Within DLA’s Organization and Oversight Structure

Source: BSM program documents.

Using enterprise architectures to guide and constrain information

technology investments is called for by law, OMB, and DOD, and is an

industry best practice. An enterprise architecture is a blueprint for guiding

and constraining business and technological change for the enterprise,

which can be an organization (e.g., DLA or the services) or a functional or

mission area spanning more than one organization (e.g., logistics

management or financial management). In some cases, both organizational

and functional/mission area architectures are appropriate, because

organizations interrelate closely, sharing functional and mission area

responsibilities. This is the case for DOD and its component organizations.

DLA is managing its BSM program without having either a DLA enterprise

architecture or a DOD-wide logistics management enterprise architecture.

In doing so, DLA risks investing in significant business process and system

change that does not optimally support enterprise (DLA and DOD)

requirements (operational, system, and technical). To their credit, officials

from both DLA and the Office of the Deputy Under Secretary for Logistics

BSM Is Not Being

Guided by Either a

DLA Architecture or a

DOD-wide Logistics

Architecture

Business process reengineering

Transformation management

Knowledge transfer

System deployment Acquisition planning

Requirements management

Contractor management

Testing

Logistics

Operations

Business

Modernization

BSM

Stakeholders

Group:

DLA Senior Executives

Primary External Stakeholders

CIO/Information

Operations

Program

Executive Officer

DLA-21 Executive Board

Program

Manager, BSM

DLA Director

Vice Director

Executive Director

DLA Field 

Commands

Organizations within grey box are represented on the DLA-21 Executive Board.

Dotted lines represent communication and coordination.

Financial

Operations
Page 15 GAO-01-631 Information Technology

and Materiel Readiness recognize the need to take an enterprise approach

to transforming logistics management, and at the conclusion of our work,

they committed to doing so.

Enterprise architectures are essential tools for effectively and efficiently

reengineering business processes and for implementing and evolving their

supporting systems. Enterprise architectures systematically capture—in

useful models, diagrams, and narrative—the full breadth and depth of the

mission-based mode of operation for a given enterprise, which can be (1) a

single organization or (2) a functional or mission area that transcends

more than one organizational boundary (e.g., financial management,

acquisition management, logistics management). An architecture describes

the enterprise’s operations in both (1) logical terms, such as interrelated

business processes, information needs and flows, work locations, and

system applications, and (2) technical terms, such as hardware, software,

data, communications, and security attributes and performance standards.

It provides these perspectives both for the enterprise’s current or “as is”

environment and for its target or “to be” environment, as well as an IT

capital investment road map for moving between the two environments.

The development, implementation, and maintenance of enterprise

architectures are recognized hallmarks of successful public and private

sector organizations. Managed properly, an enterprise architecture can

clarify and help optimize the interdependencies and interrelationships

among an organization’s business operations and the underlying IT

infrastructure and applications that support these operations. Employed in

concert with other important IT management controls, such as portfolio

investment management (selection, control, and evaluation) practices20

and continuous information security management practices,21 enterprise

architectures can greatly increase the chances that modernization

programs will succeed. Our experience with federal agencies has shown

that attempting to define and build major IT systems without first

completing an enterprise architecture often results in IT systems that are

duplicative, are not well integrated, are unnecessarily costly to maintain

and interface, and do not effectively optimize mission performance.

 20 Information Technology Investment Management: A Framework for Assessing and

Improving Process Maturity (Exposure Draft) (GAO/AIMD-10.1.23, May 2000).

21 Executive Guide: Information Security Management (GAO/AIMD-98-68, May 1998).

Enterprise Architectures:

A Cornerstone of

Successful Modernization

Programs
Page 16 GAO-01-631 Information Technology

Congress, OMB, and the federal CIO Council have recognized the

importance of enterprise architectures. The Clinger-Cohen Act, for

example, requires that agency CIOs develop, maintain, and facilitate the

implementation of enterprise architectures as a means of integrating

business processes and agency goals with IT. Further, OMB has issued

guidance on the development and implementation of agency IT

architectures. Among other things, OMB guidance directs that agency

investments in information systems be based on these architectures.22

Similarly, the CIO Council has issued guidance providing (1) a federal

framework for the content and structure of an enterprise architecture,23

(2) a process for assessing investment compliance with an enterprise

architecture,24 and (3) a set of management controls for developing,

implementing, and maintaining an enterprise architecture.25

According to CIO Council guidance, it is critically important that an

enterprise architecture be derived through a “top-down” incremental

approach, consistent with the hierarchical architectural views that are the

building blocks of published architecture frameworks, including DOD’s. It

is equally important, according to this guidance, that the higher level views

span the entire enterprise. Only through such an approach can an

organization develop enterprisewide understanding of the

interrelationships and interdependencies among business operations and

supporting technology. Such understanding is vital for informed

decisionmaking about whether the enterprise, and thus the enterprise

architecture, can be divided into segments without sacrificing the goal of

optimizing enterprisewide performance and accountability.

DOD has also issued enterprise architecture policy, including a framework

defining an architecture’s content and structure. Specifically, in February

 22 Funding Information Technology Systems Investments, OMB Memorandum M-97-02

(October 25, 1996); Information Technology Architectures, OMB Memorandum M-97-16

(June 18, 1997).

23 Federal Architecture Framework, Version 1.1, Chief Information Officers Council

(September 1999).

24 Architecture Alignment and Assessment Guide, Chief Information Officers Council

(October 2000).

25 A Practical Guide to Federal Enterprise Architecture, Version 1.0, Chief Information

Officers Council (February 2001).
Page 17 GAO-01-631 Information Technology

1998,26 DOD directed its components and activities to use the C4ISR

(Command, Control, Communications, Computers, Intelligence,

Surveillance, and Reconnaissance) Architecture Framework, Version 2.0.

According to DOD, this framework is a critical tool in achieving its

strategic direction, and all DOD components and activities should use the

framework for all functional areas and domains within the department.

This framework is also recognized in the CIO Council published guide as a

model architecture framework. Figure 3 shows the views required by the

C4ISR framework and their relationships.

Figure 3: C4ISR Architecture Framework: Views and Description

Source: C4ISR Architecture Framework, Version 2.

 26 The February 28, 1998, memorandum was jointly signed by the Under Secretary of

Defense for Acquisition and Technology, the Acting Assistant Secretary of Defense for

Command, Control, Communications, and Intelligence, and the Director for C4 Systems,

Joint Chiefs of Staff.

Technical

view

Specific capabilities identified

to satisfy information exchange

levels and other operational

requirements

Technical criteria governing

interoperable implementation/

procurement of the selected 

system capabilities

Relates capabilities and characteristics

to operational requirements

Systems

view

Processing and internodal

levels of information

exchange requirements

Systems associations to 

nodes, activities, needlines 

and requirements

Processing and levels of

information exchange 

requirements

Basic technology

supportability and 

new capabilities

Identifies warfighter

relationships and information needs

Operational

view

Prescribes standards and

conventions
Page 18 GAO-01-631 Information Technology

Briefly, the C4ISR Architecture Framework decomposes an enterprise

architecture into three primary views (windows into how the enterprise

operates): the operational, systems, and technical views. According to

DOD, these three interdependent views are needed to ensure that IT

systems are developed and implemented in an interoperable and costeffective manner. Each of these views is summarized below.

• The operational architecture view defines the operational elements,

activities, tasks, and information flows required to accomplish or support

an organizational mission or business function. According to DOD, it is

useful for facilitating a number of actions and assessments across DOD,

such as examining business processes for reengineering or defining

operational requirements to be supported by physical resources and

systems.

• The systems architecture view defines the systems and their

interconnections supporting the organizational or functional mission,

including how multiple systems link and interoperate, and may describe

the internal construction and operations of particular systems. According

to DOD, this view has many uses, such as helping managers to evaluate

interoperability improvement and to make investment decisions

concerning cost-effective ways to satisfy operational requirements.

• The technical architecture view defines a minimum set of standards and

rules governing the arrangement, interaction, and interdependence of

system applications and infrastructure. It provides the technical standards,

criteria, and reference models upon which engineering specifications are

based, common building blocks are established, and applications are

developed.

Within the three architectural views, the C4ISR Architecture Framework

identifies 26 graphical, textual, and tabular architectural artifacts or

products. Of the 26 products, DOD specifies that 7 are essential and must

be developed for each enterprise architecture. Table 4 briefly describes

the content of each essential product.
Page 19 GAO-01-631 Information Technology

Table 4: Seven Essential Products for the DOD C4ISR Architecture Framework

Essential product Description

Overview and summary information Serves as planning guide and summarizes “who, what, when, why, and how” for

architecture to be developed

Integrated dictionary Provides a central source for definitions of all terms used in all architecture products

High-level operational concept graphic Shows a high-level graphic description of operational concept, including organizations,

missions, and geographic distribution of assets

Operational node connectivity description Identifies organizational elements that produce, process, and consume information; need

to exchange information between elements; and characteristics of information exchanged

(content, media, volume requirements, security classification, timeliness, and

interoperability requirements)

Operational information exchange matrix Provides information exchange requirements, identifying who exchanges what information

with whom, why information is necessary, and how it is needed

System interface description Links operational and systems architecture views by depicting information systems and

their interfaces to organizational elements that produce, process, and consume information

Technical architecture profile Establishes a set of rules governing system implementation and operation; normally,

references existing technical guidance and discusses how that guidance has been or

needs to be implemented

Source: C4ISR Architecture Framework, Version 2.

BSM officials, including the BSM program manager and chief architect,

have acknowledged that DLA does not have an enterprise architecture.

However, these officials also maintain that BSM still has had

enterprisewide architectural direction, because although the program did

not have C4ISR-compliant architectural artifacts, the strategic direction of

the agency (as defined in strategic plans) was the primary focus of BSM.27

Nevertheless, these officials also stated that they planned to develop a

C4ISR-compliant DLA enterprise architecture.

Subsequently, the BSM program office prepared a draft enterprise

architecture development plan that focused first on developing a materiel

management architecture, which DLA calls the BSM Business Architecture

Blueprint, as a by-product of acquiring and implementing its BSM COTS

solution. According to the draft plan, once BSM is completed (scheduled

for fiscal year 2005), DLA will add other business functions to the BSM

 27 Strategic Plan 2000: DLA 21 (September 1999); Logistics Transformation Plan (June

28, 2000).

DLA’s Approach to

Developing an

Architecture Is Not

Enterprise Focused
Page 20 GAO-01-631 Information Technology

Business Architecture Blueprint so that the end result is an enterprisewide

architecture (see fig. 4).28

Figure 4: DLA’s Approach to Creating an Enterprise Architecture

Note: Base support, although not included by DLA among its major business functions, is included in

this diagram because it is supported by a legacy system that would have to be modernized.

Source: Enterprise Information Architecture Plan (draft, 28 July 2000), DLA.

DLA’s approach to developing and implementing a C4ISR-based,

agencywide enterprise architecture is thus one of compliance after the fact

(that is, after acquiring and implementing its system solution), rather than

development before the fact (and application during the fact). DLA plans

to fill out the essential products of the C4ISR framework as by-products of

implementing its COTS solution, rather than developing these products

beforehand, so that they could be used as the basis for guiding and

constraining its acquisition and implementation of system solutions.

 28 DLA’s plan shows each of the business functions being incorporated separately into the

enterprise architecture; however, for materiel management operations that are to be

fulfilled through commercial supply chains, some aspects of other functions (such as

distribution) will be included in the BSM Business Architecture Blueprint.

Materiel management

Fuels

Distribution

Cataloging

Base support

Reutilization

Hazardous material

aTo be determined.

BSM

FAS

CRS

Phased development

Systems:

BSM = Business Systems Modernization

FAS = Fuels Automated System

DSS = Distribution Standard System

CRS = Cataloging Reengineering System

TBDa

DSS

TBD

TBD

DLA functions:

BSM Baseline 1BSM Baseline 2BSM Baseline 3BSM Baseline 4
Page 21 GAO-01-631 Information Technology

Although DLA has not yet formally developed the seven essential products

in the C4ISR format, BSM program officials point out that some of the

information needed for these products is available, and DLA is relying on

that information in its direction of BSM. For example, BSM officials

acknowledged as enterprise requirements the following: technology

infrastructure standards, a transition strategy, technical reference models,

standards profiles, and information assurance policies at the enterprise

level. In our view, such information would be valuable and useful in

building a DLA enterprise architecture.

However, DLA nevertheless does not have a complete enterprise

architecture as defined by DOD’s C4ISR framework, and its plans for

developing one are not consistent with a fundamental best practice in CIO

Council guidance. That is, DLA’s approach is focused on only part of its

enterprise (albeit the largest), whereas CIO Council guidance promotes a

process to ensure development of an enterprisewide perspective.

According to this guidance, the scope of the architecture for the

operational view needs to encompass the entire enterprise. Only then can

the agency understand the relationships and dependencies among its

business areas and position itself to make informed decisions about the

level of depth and detail to include in the architecture. Necessarily, this

level of depth and detail will be tailored to each enterprise based on its

size and complexity, as well as the purpose of the architecture and the

riskiness of the modernization effort to be undertaken.

Although DLA’s approach does not preclude success, it does introduce a

risk of misalignment between the COTS solution and the enterprise’s

strategic operational vision. To manage this risk, the enterprise would

have to fully evaluate the COTS solution against the business requirements

for the whole enterprise, not just a portion of it. Such an enterprisewide

perspective is what Strategic Plan 2000: DLA 21 and the Logistics

Transformation Plan envision. (For instance, DLA 21 refers to logistics

solutions that are integrated from two standpoints: throughout the supply

chain and throughout the agency.) An approach that lacks an

enterprisewide perspective does not adequately ensure that the COTS

system solution being acquired and implemented will optimally support all

logistics business functions.

Despite the existence of some enterprisewide architectural information,

DLA’s plan for building an architecture for its future operational, systems,

and technical environment does not have an enterprisewide perspective.

Instead, DLA is defining and implementing its architecture by first

focusing on one of its six business functions (materiel management)
Page 22 GAO-01-631 Information Technology

separately from the others. For DLA, this is particularly problematic

because DLA’s six business functions are interrelated and interdependent.

For example:

• Materiel management and cataloging are related, because how an item is

procured is determined in part by how an item is classified during the

cataloging process.

• Materiel management and distribution are related, because decisions

about whether DLA will distribute the materiel (or the vendor will deliver

directly to the requisitioner) are made at the same time the materiel is

bought.

• Distribution and cataloging are related, because how materiel is stored

and transported is determined by how it is classified during cataloging

(e.g., hazardous materiel has special storage and transportation needs).

DLA adopted its approach to developing and implementing an enterprise

architecture because the agency believed that using materiel

management—its largest business process—as a starting point would lead

to a solution with a baseline functionality that could be augmented as DLA

evolves. Further, DLA expected that addressing materiel management

would lead to the inclusion of some aspects of its other business

functions, such as distribution and cataloging, for that part of materiel

management operations to be performed via private sector supply chains.

However, DLA also recognized that postponing consideration of some

business functions might lead to disconnects later in the development.

According to DLA, if the later DLA business functions could not be

incorporated into the BSM Business Architecture Blueprint without major

redesign, then DLA would construct linkages between BSM and the other

business areas to produce an enterprisewide architecture. This approach

introduces the risk that DLA will sacrifice optimizing enterprisewide

performance and accountability in order to optimize the performance and

accountability of its individual components—the risk that well developed

and implemented enterprise architectures are intended to prevent.

At the conclusion of our review, DLA officials stated that they intend to

expand the scope of their enterprise architecture program to provide for

an agencywide, C4ISR-compliant architecture.
Page 23 GAO-01-631 Information Technology

CIO Council guidance defines a set of recognized key practices

(management structures and processes) for developing and implementing

an enterprise architecture that are hallmarks of successful public and

private sector organizations. Among other things, these practices include

the following:

• Because the enterprise architecture is a corporate asset for systematically

managing institutional change, the head of the enterprise should support

and sponsor the architecture, giving it a clear mandate in the form of an

enterprise policy statement. Such support is crucial to gaining the

commitment of all organizational components of the enterprise, all of

whom should participate in developing and implementing the enterprise

architecture.

• The enterprise architecture effort should be directed and overseen by an

executive body, empowered by the head of the enterprise, with members

who represent all stakeholder organizations and have the authority to

commit resources and to make and enforce decisions for their respective

organizations.

• The enterprise architecture effort should be led by a Chief Architect who

reports to the enterprise CIO, and it should be managed as a formal

program. A formal program entails creating a program office, committing

core staff, implementing a program management plan that details work

breakdown structure and schedules, allocating resources and tools,

performing basic program management functions (e.g., risk management,

change control, quality assurance, configuration management), and

tracking and reporting progress against measurable goals.

• The enterprise architecture should conform to a specified framework.

DLA is following some of these recognized key practices: it plans to follow

the DOD C4ISR Architecture Framework, and a BSM Chief Architect has

been designated. However, DLA is not following most key practices. It is

not, for example,

• approaching the architecture as a corporate endeavor, with explicit

support and sponsorship from the DLA Director in the form of an agency

policy statement;

• using an executive body consisting of stakeholders from across DLA to

direct and oversee the architecture effort; or

• managing the architecture effort as a formal program.

Additionally, while an architecture development plan was drafted

following our inquiries, neither the DLA CIO nor any DLA executive body

has taken any action to approve the draft plan (dated July 2000), and

DLA Does Not Have

Effective Management

Structures and Processes

for Developing and

Implementing an

Enterprise Architecture
Page 24 GAO-01-631 Information Technology

thereby demonstrate their commitment and accountability for the

architecture.

Moreover, the Chief Architect is assigned to the BSM program rather than

to an organization with authority and responsibility for DLA-wide logistics

modernization, and the draft architecture development plan focuses solely

on materiel management, only providing a notional sequence for possibly

incorporating DLA’s five other business functions at some time after 2005.

Further, no work breakdown structures and schedules are provided for

adding these other business functions, no estimates of resources are

provided for implementing the plan, no measures are provided for

managing progress in developing the architecture, and no enterprise

stakeholders outside the BSM program participated in drafting the plan.

At the conclusion of our review, DLA officials stated that they will expand

the architecture program to provide an agencywide focus, assign

responsibility and accountability for the program accordingly, and

formalize their approach to its management.

Beyond the need for a DLA organization-based enterprise architecture,

DLA needs a DOD-wide logistics enterprise architecture, which would also

serve DOD’s other component organizations that have interrelated and

interdependent logistics management missions. However, DOD does not

yet have such an architecture, although it has begun efforts to develop

one.

Despite the lack of a DOD-wide logistics architecture, in March 2000, the

former Deputy Secretary of Defense directed DLA and the military

services to develop plans for modernizing their respective logistics

processes and systems by July 2001 and to implement their respective

plans by September 2006. No DOD-wide architectural blueprint to promote

operational and system commonality and integration accompanied this

direction.

According to the Director of Logistics Systems Modernization within the

Office of the Deputy Under Secretary of Defense for Logistics and Materiel

Readiness (the DOD organization responsible for overseeing the

implementation of systems modernization efforts in 21st Century

Logistics), DOD needs a departmental logistics management enterprise

architecture to achieve its logistics management vision and optimize DODwide logistics management performance and accountability. Without such

an architecture, the Director stated that the services’ and DLA’s

DOD Does Not Have a

Departmental Logistics

Management Enterprise

Architecture
Page 25 GAO-01-631 Information Technology

investments in new logistics management systems would result in

operations and systems that, although modernized, would continue to be

“stovepiped,” rather than optimized and integrated across DOD.

Accordingly and as an outgrowth of Year 2000 testing, efforts were started

to develop a DOD-wide logistics enterprise architecture concurrently with

DOD components’ efforts to modernize their respective operations and

systems. According to the Director, modernization of component

operations and systems is the single most important factor in being able to

effectively and efficiently optimize and integrate operations and systems

across DOD. The Director’s position is that components’ modernization

efforts and DOD-wide logistics architecture efforts should proceed in

parallel.

To date, according to the Director,

• a plan has been drafted for creating a departmental logistics management

enterprise architecture;

• a description of the department’s current or “as is” logistics management

enterprise architecture is being developed; and

• descriptions of both near-term and long-term future (“to be”) logistics

management enterprise architectures are being developed.

However, the Director would not provide us a copy of the enterprise

architecture development plan, noting that it has yet to be approved and

that a milestone for completing the enterprise architecture was not

available. He also would not provide us copies of any “as is” architectural

products, acknowledging that these products were based on data obtained

under DOD’s Year 2000 Conversion Program and thus needed to be

modified in light of the ongoing modernization activities in the military

service and DLA. He added that the near-term and long-term “to be”

architecture efforts are being managed by two different organizations

within the Office of the Deputy Under Secretary for Logistics and Materiel

Readiness, and that these architectures are to be based on yet-to-bedeveloped business rules and operational and technical agreements among

the services and DLA. The Director specifically cited efforts under way to

develop an architecture for DOD-wide management of conventional

ammunition, DOD’s most critical commodity for the warfighter, as the

prototype effort for developing a logistics management architecture and

policies for architecture-based portfolio investment management.

The Director also told us that the Office of the Deputy Under Secretary of

Defense for Logistics and Materiel Readiness would control these

investments using traditional means, such as exercising oversight of the
Page 26 GAO-01-631 Information Technology

acquisition and budget processes. This oversight role will be augmented,

according to the Director, by a DOD enterprise integration consortium,

established in December 2000, made up of component organizations that

are pursuing logistics enterprise resource planning system solutions, and

led by the Director. The purpose of this consortium is to update plans for

community services and DOD-wide logistics rules, develop enterprisewide

interfaces between logistics and other functional areas, and develop

mechanisms for sharing development plans and technical artifacts among

programs and for sharing lessons learned.

These controls, however, are not sufficient substitutes for having a welldefined enterprise architecture. As discussed earlier, an enterprise

architecture defines the business and technical rules, standards, and

protocols that govern how the entire enterprise will operate in the future,

and provides a common road map for getting to this future operational

state. It thus serves as the explicit, documented change management tool

that the Office of the Deputy Under Secretary for Logistics and Materiel

Readiness needs to effectively discharge the very acquisition and budget

oversight role that the Director cited. Without an enterprise architecture to

guide and constrain the components’ modernization efforts, there is not

adequate assurance of common understanding of the nature and content

of the change.

At the conclusion of our review, an official in the Deputy Under

Secretary’s office acknowledged that DOD needs to move swiftly to

develop and use a DOD-wide logistics management architecture.

Incremental investment management involves three fundamental

components: (1) developing/acquiring a large system in a series of smaller

projects or system increments; (2) individually justifying investment in

each separate increment on the basis of costs, benefits, and risks; and

(3) monitoring actual benefits achieved and costs incurred on ongoing

increments and modifying subsequent increments/investments to reflect

lessons learned.29 Using this approach prevents agencies from discovering

too late that their systems are not cost beneficial and allows them to

reduce the enormous risks associated with large, expensive projects. Such

an approach does not preclude overlap and smooth transition among

 29 Customs Service Modernization: Serious Management and Technical Weaknesses Must

Be Corrected (GAO/AIMD-99-41, February 26, 1999).

DLA Has Not Invested

Incrementally in BSM
Page 27 GAO-01-631 Information Technology

increments, because lessons learned from the actual results of ongoing

increments should be monitored and evaluated continuously so that these

results are available for use in defining and justifying future increments.

DLA has not yet followed an incremental approach to investment in BSM.

To date, DLA has treated the entire BSM program as one investment

decision, justified by a single economic analysis, because this approach

was consistent with DOD policy for major system acquisitions30 that were

in effect until January 2001. DLA invested in its preliminary work (the

analysis of candidate COTS systems against previously reengineered

business processes) and in the initial release of BSM on the basis of this

economic analysis. As a result, DLA did not justify and decide on investing

about $150 million thus far in BSM on the basis of release-specific analyses

of costs, benefits, and risks.

This kind of approach to making investment decisions has historically

resulted in agencies investing huge sums of money in systems that do not

provide commensurate benefits, and thus has been abandoned by

successful organizations. The need to avoid this pitfall was a major

impetus for the Clinger-Cohen Act investment management reforms.

At the conclusion of our review, DLA officials told us that in future, they

plan to make investment decisions on BSM incrementally.

The Clinger-Cohen Act of 1996 requires agencies to follow, to the

maximum extent practicable, an incremental approach to investing in IT

development/acquisition projects. Additionally, OMB policy requires that

investments in major information systems be implemented in increments,

with each increment delivering measurable benefits.31 More specifically,

OMB’s Capital Programming Guide 32 describes the use of modular

contracting or incremental investment, including its application and

benefits. In particular, OMB states that project increments should provide

for the following:

 30 DOD Instruction 5000.2, Operation of the Defense Acquisition System.

31 Memorandum for Heads of Executive Departments and Agencies, OMB Memorandum

M-97-02 (October 25, 1996); Capital Programming Guide, Version 1.0, OMB Circular A-11,

part 3 (July 1997), pp. 545–572.

32 Capital Programming Guide, Version 1.0, OMB Circular A-11, part 3, Supplement,

Planning, Budgeting, and Acquisition of Capital Assets (July 2000), pp. 35–37.

Incremental Investment

Management Spreads the

Risk of Large Programs

Across Smaller,

Incremental Parts
Page 28 GAO-01-631 Information Technology

• Separability: Each increment should be fully funded, have substantial

programmatic use that is not dependent on any subsequent increment, and

be capable of performing its principal functions even if no subsequent

increments are acquired.

• Interoperability: Each increment should comply with a common

architecture or commercially acceptable technology standards and should

be compatible with and capable of being integrated with other increments.

• Performance requirements: Each increment’s performance requirement

should be consistent with the performance requirements of the completed

overall system and should address interface requirements with other

increments.

In short, incremental investment helps to mitigate the risks inherent in

large IT acquisitions/developments by breaking apart a single large project

into smaller, independently useful components with known and defined

relationships and dependencies. Making investment decisions up front on

large-scale, long-term projects is generally risky: their economic

justification is based on costs, benefits, and risks that are difficult to

forecast reliably, partially because later increments are not well

understood or defined, and partially because they are subject to change in

light of experience on nearer term increments and changing business

needs. Through incremental investment management, organizations can

• reduce the level of project risk and complexity faced at any one time by

spreading this risk and complexity across a series of smaller investments;

• permit the delivery of some part of the expected business value earlier

rather than waiting until later for the total, but more uncertain, business

value;

• continuously monitor and evaluate the delivery of cost and benefit

expectations on ongoing increments and use this information to better

define and economically justify these increments; and

• permit later increments to exploit technology advances or accommodate

evolving business needs.

In January 2001, DOD issued a change to its major system acquisition

policy33 requiring incremental investment management; specifically, the

policy notes that a program’s milestone decision authority must verify that

each increment meets part of the mission need and delivers a measurable

benefit, independent of future increments.

 33 DOD Instruction 5000.2, Change 1, Operation of the Defense Acquisition System.
Page 29 GAO-01-631 Information Technology

Although DLA plans to acquire and implement its BSM system solution in

four increments, it has not so far managed BSM investments

incrementally. Specifically, (1) DLA has not justified proposed investment

in the first BSM increment on the basis of costs and benefits, (2) the DLA

21 Executive Board34 has not made decisions about whether to invest in

each discrete BSM increment, and (3) the BSM program office has not

measured actual return on investment from each increment and used the

results to better inform decisionmaking about future increments.

DLA’s position is that BSM was being managed according to DOD

Instruction 5000.2, which until January 2001 required a single economic

analysis of a system’s life-cycle costs and benefits before the system

entered the concept demonstration phase. Accordingly, DLA prepared a

program life-cycle economic analysis for BSM, according to which the

program is expected to produce a positive net present value over its 15-

year life cycle and improve military readiness. As a result, DLA officials

stated that investing about $900 million to acquire and implement the BSM

system solution is prudent, and DLA committed itself to funding,

acquiring, and implementing all four BSM phases or increments.

Relying on a single economic analysis to make an investment decision for

a program that is large and risky, involving many things to be done over

many years, is neither prudent nor consistent with the principles of

incremental investment management embodied in the Clinger-Cohen Act

and OMB requirements. Approaching the BSM investment decision in this

way was especially risky, since at the time that DLA prepared the BSM

program life-cycle economic analysis, it had defined BSM business

requirements for only the reengineered part of the materiel management

business area. Thus, DLA was not only attempting the daunting task of

accurately estimating the costs and benefits of many things over many

years, it was doing so without knowing its BSM requirements beyond the

reengineered parts of the materiel management business area. Our

experience in reviewing IT investments across the federal government has

 34 This board is DLA’s senior investment decisionmaking body. It is responsible for

investment selection, control, and evaluation decisions for all investments.

DLA Has Not Yet Made

Investment Decisions

Incrementally
Page 30 GAO-01-631 Information Technology

shown that such estimating does not produce reliable results and cannot

provide a sufficient basis for informed investment decisionmaking.35

To date, the BSM program office has not analyzed the costs, benefits, and

risks of what DLA defines as its first BSM release. Program officials told us

that they justified this release, which is to cost $93 million (not including

the $56 million that was spent on BSM before the first release), on the

basis of the program life-cycle economic analysis.

At the conclusion of our review, however, DLA officials informed us that

they do now plan to make decisions on the last three releases

incrementally: they plan to justify release 2 on the basis of its costs,

benefits, and risks, and to verify that the first release produced benefits

commensurate with costs before deciding to invest in release 2. They also

stated that they would follow this approach to investing in later releases.

According to program officials, because DLA is still working on its first

release (concept demonstration), the investment decision for release 2 will

not be made until fiscal year 2002.

However, the program office does not yet have plans or measures to

determine, once an increment is implemented, whether its expected value

is actually accruing. The implementation plans for the first BSM increment

(release 1) were completed in April 2001, but no specific plans or

measures were developed to determine whether benefit versus cost

expectations would be met for this increment. Clinger-Cohen requires that

agencies identify quantifiable measurements for determining costs and

benefits. Until the program office has some means of measuring expected

value, DLA will not be able to determine whether this first increment is

performing at a level equal to or better than its current materiel

management systems—SAMMS and DISMS.

According to the BSM program manager, the investment management

plans and measures are not in place because DLA’s policies and

procedures on investment portfolio management are still in draft. When

 35 Air Traffic Control: Complete and Enforced Architecture Needed for FAA Systems

Modernization (GAO/AIMD-97-30, February 3, 1997); Air Traffic Control: FAA’s

Modernization Investment Management Approach Could Be Strengthened (RCED/AIMD99-88, April 1999); Customs Service Modernization: Architecture Must Be Complete and

Enforced to Effectively Build and Maintain Systems (GAO/AIMD-98-70, May 5, 1998);

Information Technology: INS Needs to Strengthen Its Investment Management Capability

(GAO-01-146, December 2000); Information Technology Management: Coast Guard

Practices Can Be Improved (GAO-01-190, December 2000).
Page 31 GAO-01-631 Information Technology

these are completed, they should govern, for example, (1) how investment

increments are to be defined, prioritized, and sequenced, (2) how

performance criteria are to be applied to each increment, and (3) how

accrued earned value from each investment increment will be measured.

Until these investment management plans and measures are in place, the

program manager stated that incremental investment will be accomplished

through a series of 11 planned contract task orders, consisting of 105 more

specific subtasks, which are to span the four BSM increments. However,

merely structuring contract task orders into incremental pieces is not a

sufficient substitute for making investment approval and funding decisions

incrementally.

DLA has already invested significant sums of money in its BSM program,

and its plans call for investing hundreds of millions of dollars more. To

invest this money wisely and in a way that minimizes risks, DLA will need

to acquire BSM within the context of an enterprise architecture and on an

incremental basis. To date, DLA has not used an enterprise architecture or

applied management controls for developing, implementing, and

maintaining one, as described in CIO Council guidance. Additionally, DLA

has not employed incremental investment practices.

Officials of DLA and the Office of the Secretary of Defense have

committed to correcting these management weaknesses. We view these

commitments as positive first steps for two primary reasons. First, if DLA

continues to invest large sums of money in BSM without a DLA- or DODwide logistics enterprise architecture, it runs the serious risk that it will

acquire a system solution that focuses narrowly on DLA materiel

management performance and accountability at the expense of DLA- and

DOD-wide performance and accountability. Second, until DLA begins

justifying the return on its BSM investment incrementally, making BSM

investment decisions incrementally, and measuring actual return on this

investment incrementally, it will not be able to make well-informed and

prudent investment decisions, and it will not know whether its BSM

system solution is cost-effective until it has already spent hundreds of

millions of dollars.

To address DLA’s need for an enterprise architecture to guide and

constrain its IT capital investments, including BSM, we recommend that

the Secretary of Defense direct the DLA Director to designate and treat

Conclusions

Recommendations
Page 32 GAO-01-631 Information Technology

development, implementation, and maintenance of a DLA enterprise

architecture as an agency priority.

We recommend that in fulfilling this direction from the Secretary, the DLA

Director (1) issue a policy governing development, implementation, and

maintenance of an enterprise architecture and (2) establish a DLA

enterprise architecture steering committee, chaired by the DLA Vice

Director, to provide a DLA-wide direction and focus to the enterprise

architecture, and to ensure that one is developed and maintained in a

manner consistent with the CIO Council published guide on managing

enterprise architectures.

To provide for the effective development and maintenance of the DLA

enterprise architecture, we recommend that the DLA Director make the

CIO accountable to the enterprise architecture steering committee for

developing and maintaining the agency’s enterprise architecture. We

recommend that in fulfilling this responsibility, the CIO appoint a Chief

Architect for DLA and establish a program office responsible for

developing and maintaining a DLA-wide enterprise architecture. We

further recommend that the CIO direct the Chief Architect to work

collaboratively with the appropriate offices within the Office of the Deputy

Under Secretary for Logistics and Materiel Readiness to appropriately

align the DLA enterprise architecture with this office’s ongoing efforts to

develop a DOD-wide logistics enterprise architecture. Additionally, we

recommend that the CIO have the Chief Architect, as appropriate, follow

the steps outlined in the CIO Council’s published guide for managing

enterprise architectures, including

• obtaining executive buy-in and support,

• establishing architecture management structure and controls,

• defining the architecture process and approach,

• developing the baseline architecture, the target architecture, and the

sequencing plan,

• facilitating the use of the architecture, and

• maintaining the architecture.

To ensure that, once developed, the enterprise architecture is effectively

implemented, we also recommend that the Director require the DLA 21

Executive Board and its subsidiary investment review boards to ensure

that only architecturally compliant IT investments are approved and

funded, unless the investment decisionmaking authority issues a written

waiver in response to a written justification.
Page 33 GAO-01-631 Information Technology

Until a DLA enterprise architecture is developed and can be used to

effectively guide and constrain DLA IT investment, we also recommend

that the DLA Director limit future investment in BSM to only its first

incremental release.

To address DLA’s need to incrementally invest in BSM, we recommend

that the DLA Director impose three further conditions on investment in

BSM beyond its first increment. Specifically, such investment should not

occur until plans have been developed and processes implemented for

(1) measuring and validating whether ongoing BSM increments are

actually producing benefits commensurate with costs, (2) developing

economic justifications for each subsequent increment that consider the

actual performance of ongoing increments, and (3) ensuring that decisions

on each subsequent increment are based on these incremental economic

justifications.

To ensure that the approach to DLA’s logistics modernization (as well as

that of other DOD component organizations) recognizes the logistics

management interdependencies and interrelationships among DOD

components and aims to optimize departmentwide performance and

accountability, we recommend that the Secretary of Defense direct the

Under Secretary of Defense for Acquisition, Technology, and Logistics to

(1) expedite development of a DOD-wide logistics management enterprise

architecture in a manner consistent with CIO Council published guidance

and (2) establish effective controls for ensuring that component

investments in modernized logistics systems are compliant with the

architecture or are otherwise granted an explicit waiver to this

requirement on the basis of analytical justification.

In written comments on a draft of this report (reprinted in appendix II),

the acting Deputy Under Secretary of Defense for Logistics and Materiel

Readiness agreed with us on the importance of developing and

maintaining enterprise architectures and making BSM investment

decisions incrementally, concurring with eight of our ten

recommendations. DOD also acknowledged that the absence of these IT

management controls increased program risk. Despite these areas of

substantial agreement, the Deputy Under Secretary disagreed with our

recommendations that the DLA Director limit future investment in BSM to

the first incremental release until (1) a “full-blown” (as DOD termed it)

DLA enterprise architecture is developed and (2) the BSM program

manager has the means in place to measure actual benefit accrual on the

Agency Comments

and Our Evaluation
Page 34 GAO-01-631 Information Technology

first incremental release and to use this information in deciding on

investment in the next release.

According to DOD, our findings and conclusions regarding the overall

level of risk to the successful achievement of BSM and other DOD logistics

modernization program objectives are not supported by the single risk

factor addressed in this report—the use of enterprise architectures. DOD’s

stated view is that the risks associated with delaying BSM are greater than

the risks of modernizing while concurrently developing the architecture.

DOD also stated that while the conditions that we recommended regarding

measurement and use of actual benefit accrual are valuable and will be

performed by DOD, they should not be performed by the program

manager and should not be on the critical path for undertaking future

incremental releases.

While DOD agrees that the absence of an “end-to-end” enterprise

architecture for DLA is a risk, DOD strongly believes that this risk is amply

mitigated by measures already taken. As examples, the Deputy Under

Secretary’s comments refer to the preparation of an enterprisewide

logistics implementation plan for community data management services

that defines the future information and business rules interaction among

DOD components; the existence of an “as is” logistics architecture

(documented during the Y2K campaign); the recent establishment of an

OSD-led enterprise integration consortium that, according to the

comments, will ensure optimal collaboration among programs; and the

commercial architecture and process reengineering procedures being

followed by DLA’s BSM program.

Notwithstanding the considerable level of agreement between DOD and us

on this report, we have a differing view on the level of risk facing DOD

logistics modernization programs in general, and BSM in particular, in the

absence of the two risk mitigators addressed in this report: (1) effective

enterprise architecture development and implementation and (2) effective

incremental investment management.

Our experience in reviewing other federal agency modernization programs

has shown that the absence of these program management controls are

two of the root causes for other programs’ lack of success.36 The two

recommendations that DOD disagrees with are intended to strike a

 36 GAO/AIMD-97-30, RCED/AIMD-99-88, GAO/AIMD-98-70, GAO-01-146, and GAO-01-190.
Page 35 GAO-01-631 Information Technology

reasonable balance between the risk that DOD will suffer the same fate as

these other agencies and the risk that DOD points out of delaying BSM.

These recommendations permit DLA to make progress on the first BSM

increment while developing and implementing the two missing

management controls and thus becoming better equipped to move forward

on subsequent increments. Therefore, we continue to believe that DOD

should limit investment in BSM beyond the first increment until certain

conditions are met.

We agree with the Deputy Under Secretary that the risk mitigation

strategies that DOD has followed can lower the risk associated with the

lack of a complete enterprise architecture. However, we continue to

believe that these efforts are not a sufficient substitute for having an

explicit enterprise architecture and effectively using it to guide and

constrain modernization investment decisions. At most, these mitigation

strategies should be viewed as temporary controls until the architecture is

available, as well as useful starting points for the development and use of

an enterprise architecture.

With regard to risk mitigation, the Deputy Under Secretary comments

further that commercial implementation of enterprise resource planning

(ERP) is a proven means for achieving an enterprise business strategy,

adding that a key requirement of DLA’s BSM program is to move to a

commercial approach for achieving its end-state enterprise architecture.

We agree that the use of commercial products can be a less risky approach

than the development of custom applications, but we believe that success

still depends on effective implementation and in particular on ensuring

that the chosen products meet the needs of the enterprise; this

requirement is best fulfilled by having and using an enterprise

architecture. The main point of our recommendations is that DLA has yet

to define its end-state enterprise architecture and therefore cannot know

whether its chosen ERP solution will satisfy requirements beyond the

materiel management business area. Accordingly, we stand by our

recommendation in this area, which is intended to significantly lower the

risk for the BSM program in relation to its role in developing DLA’s

enterprise architecture, not just in achieving DLA’s requirements for

modernizing its materiel management systems.

We would add that DOD’s characterization of the enterprise architecture

that we recommend as “full-blown” and “end-to-end” suggests that we are

prescribing the full depth and detail of this architecture. We are not. In

fact, effective architecture management practices recognize that there is

no one-size-fits-all architecture solution and that the driving goal in
Page 36 GAO-01-631 Information Technology

developing an architecture is to make it useful for the task at hand,

meaning that its depth and detail will vary from enterprise to enterprise.

The Deputy Under Secretary also disagreed with our recommendation to

impose additional conditions on investment in BSM beyond its first

increment, specifically concerning the development of processes for

performing cost/benefit analyses of ongoing and completed increments.

According to the Deputy Under Secretary, the BSM program does plan to

prepare a business case to support each of four scheduled releases and to

include in each of these increments a projection of business process

performance and an evaluation of empirical results of the program to that

point. These are the very activities that we recommend as conditions for

investing beyond the first increment. The Deputy Under Secretary’s

disagreement with our recommendation appears to center on who should

be required to measure and use actual benefit accrual in making decisions

on continuing the program to the next increment. It is not our intent to

suggest that only the BSM program manager should measure and use

actual benefit accrual or that one increment must be fully completed and

fully evaluated before another can begin. In fact, effective incremental

investment management necessitates some overlap among increments,

and our recommendations do not contradict this. On the matter of who

should measure and validate actual benefit accrual, we are not opposed to

DLA assigning this responsibility to someone other than the program

manager, and we acknowledge that the program manager should not be

solely responsible for ensuring that investment decisions on subsequent

increments are based on economic justifications. Accordingly, we have

modified the recommendation to reflect this.

DOD also provided specific comments on other aspects of the report.

These comments have been incorporated throughout the report where

appropriate.

We are sending copies of this report to the Chairmen and Ranking

Minority Members of the Senate Appropriations Subcommittee on

Defense; the Subcommittee on Readiness and Management Support,

Senate Committee on Armed Services; the House Appropriations

Subcommittee on Defense; and the Subcommittee on Readiness, House

Committee on National Security. We are also sending copies to the

Director, Office of Management and Budget, and the Director, Defense

Logistics Agency. Copies will be made available to others upon request.
Page 37 GAO-01-631 Information Technology

If you have any questions regarding this report, please contact me at (202)

512-6256 or Carl M. Urie, Assistant Director, at (202) 512-6231. We can also

be reached by e-mail at hiter@gao.gov and uriec@gao.gov. Other key

contributors to this report are listed in appendix III.

Randolph C. Hite

Director, Information Technology Systems Issues
Appendix I: Objectives, Scope, and

Methodology

Page 38 GAO-01-631 Information Technology

Our objectives were to determine (1) whether DLA is using an enterprise

architecture to guide and constrain its investment in its Business Systems

Modernization (BSM) program and (2) whether DLA is investing in BSM in

an incremental manner. Using enterprise architectures—institutional

blueprints for business and technological change—and investing

incrementally in large modernization programs are legislative and federal

requirements and best industry practices. This review did not address

other system modernization best practices, such as whether DLA is

employing effective system acquisition process controls, and did not

evaluate the specific commercial, off-the-shelf product that DLA chose as

its BSM system solution.

To determine whether DLA has an enterprise architecture to guide BSM,

we identified the DOD organizations involved in efforts to reform and

modernize DOD logistics management operations and systems, as well as

organizations responsible for DOD policy and guidance on enterprise

architectures, including the Office of the Under Secretary of Defense

(Acquisition, Technology, and Logistics); the Director of Logistics Systems

Modernization; the DLA Logistics Operations Directorate; and the DLA

Director of Information Operations, who is in charge of corporate

information policy. From each of these organizations, we solicited

information on plans and activities that defined the form and content of

these reform and modernization efforts. We then questioned officials from

each organization about planned and existing architectural artifacts and

obtained copies of all such plans and artifacts. Next, we analyzed the

information provided, including DOD’s Joint Vision 2020, 21st Century

Logistics: DOD Logistics Strategic Plan, and available DLA enterprise

architecture artifacts, against DOD’s C4ISR Architecture Framework to

determine the extent to which these organizations individually or

collectively had produced architectural artifacts that satisfied DOD

requirements. In our analysis, we also considered other published

architectural guidance, including Office of Management and Budget

memorandums and the Federal Enterprise Architecture Framework of

the Chief Information Officers Council.1

We also obtained and reviewed the federal CIO Council’s A Practical

Guide to Federal Enterprise Architecture, Version 1.0, as well as

 1 Funding Information Systems Investments, OMB Memorandum M-97-02 (October 25,

1996); Information Technology Architectures, OMB Memorandum M-97-16 (June 18, 1997);

Federal Enterprise Architecture Framework, Version 1.1, Chief Information Officers

Council (September 1999).

Appendix I: Objectives, Scope, and

Methodology
Appendix I: Objectives, Scope, and

Methodology

Page 39 GAO-01-631 Information Technology

published information on the enterprise architecture best practices that

the guide is based upon. We compared DLA’s Enterprise Information

Architecture Plan (draft, 28 July 2000) with this guidance.

To determine whether DLA is following an incremental investment

strategy for BSM, we compared DLA’s Single Acquisition Management

Plan, Operational Requirements Document, and other program-related

material against the Clinger-Cohen Act, OMB’s Capital Programming

Guide, and the associated assessment method. We evaluated policies,

procedures, and guidance related to DLA’s IT modular contracting

activities. We evaluated task order plans and performance data against

commonly accepted IT investment methodologies.

Additionally, we reviewed economic analysis of DLA’s BSM program and

the related cost justifications. We also used DOD’s Logistics

Transformation guide and the Defense Fiscal Year 2001 Budget to obtain

information about other DOD and military service supply-chain logistics

initiatives.

We conducted our work at DLA headquarters, located at Fort Belvoir, VA,

from February to April 2001 in accordance with generally accepted

government auditing standards.
Appendix II: Comments From the Department

of Defense

Page 40 GAO-01-631 Information Technology

Appendix II: Comments From the

Department of Defense

See comment 2.

See comment 1.
Appendix II: Comments From the Department

of Defense

Page 41 GAO-01-631 Information Technology

See comment 5.

See comment 4.

See comment 3.
Appendix II: Comments From the Department

of Defense

Page 42 GAO-01-631 Information Technology

See comment 7.

See comment 6.
Appendix II: Comments From the Department

of Defense

Page 43 GAO-01-631 Information Technology

See comment 8.
Appendix II: Comments From the Department

of Defense

Page 44 GAO-01-631 Information Technology

See comment 9.
Appendix II: Comments From the Department

of Defense

Page 45 GAO-01-631 Information Technology

See comment 10.
Appendix II: Comments From the Department

of Defense

Page 46 GAO-01-631 Information Technology

See comment 13.

See comment 12.

See comment 11.
Appendix II: Comments From the Department

of Defense

Page 47 GAO-01-631 Information Technology

See comment 15.

See comment 14.
Appendix II: Comments From the Department

of Defense

Page 48 GAO-01-631 Information Technology

See comment 17.

See comment 16.
Appendix II: Comments From the Department

of Defense

Page 49 GAO-01-631 Information Technology

See comment 19.

See comment 18.
Appendix II: Comments From the Department

of Defense

Page 50 GAO-01-631 Information Technology

See comment 21.

See comment 20.
Appendix II: Comments From the Department

of Defense

Page 51 GAO-01-631 Information Technology

See comment 24.

See comment 23.

See comment 22.
Appendix II: Comments From the Department

of Defense

Page 52 GAO-01-631 Information Technology

See comment 26.

See comment 25.
Appendix II: Comments From the Department

of Defense

Page 53 GAO-01-631 Information Technology

See comment 28.

See comment 27.
Appendix II: Comments From

the Department of Defense

Page 54 GAO-01-631 Information Technology

1. We agree that the BSM program should not be held “hostage” to the

architecture and that the program and the architecture can proceed in

parallel, but only to a point. As we have recommended, this point is the

start of the next BSM increment, which according to DLA is

August/September 2002. Our recommendation affords DLA ample time

to develop and use the missing architectural definition without

delaying the program.

2. We disagree that this report draws conclusions about the program’s

“overall” risk based on a single risk factor. Our report is not intended

to and does not provide a comprehensive BSM risk profile. Rather, it

identifies important risks associated with the two IT management

controls that were the focus of our work: (1) the use of an enterprise

architecture and (2) the use of incremental investment controls.

Further, while we acknowledge that DLA has taken risk mitigation

measures, as cited in the draft report provided to DOD for comment,

these measures are not a sufficient substitute for having an explicit

enterprise architecture and effectively using it to guide and constrain

modernization investment decisions. At most, they should be viewed

as temporary controls until the architecture is available. In fact, while

DOD believes that it cannot delay modernization, it nevertheless states

in its comments that it supports the need for enterprise architectures

and the integration of these architectures into its investment

management processes.

3. We agree that one should not conclude from this report that there is

unacceptable risk associated with modernizing systems while

developing an enterprise architecture. We also agree that there are

risks associated with delaying modernization. Accordingly, our

recommendation strikes a reasonable balance between the need for an

architecture and the need to modernize legacy systems by allowing

DOD to continue modernization while it develops an architecture.

However, it does not allow the two to proceed in parallel indefinitely,

because the longer DLA proceeds without an architecture, the greater

the risks will become.

4. We do not disagree and our report does not question the use of an ERP

solution. Our point is that DLA has yet to define its end-state enterprise

architecture and therefore cannot be sure that it is implementing its

chosen ERP solution in a manner that will not suboptimize DLA

enterprisewide performance and accountability, in order to optimize

performance and accountability of one business area.

GAO Comments
Appendix II: Comments From

the Department of Defense

Page 55 GAO-01-631 Information Technology

5. See comments 1 and 2. Also, we do not agree that BSM is currently

guided by sound architectural discipline. Such discipline would be

evidenced by the existence and use of architectural artifacts developed

in accordance with the practices outlined in CIO Council guidance.1

While we acknowledge the existence of some enterprise architecture

information, as cited in the draft report provided to DOD for comment,

and state in this report that that information would be valuable and

useful in building a DLA-wide architecture, the fact remains that DLA

does not currently have such an architecture.

6. We acknowledge that the Deputy Under Secretary of Defense for

Logistics and Materiel Readiness has pursued risk-reducing strategies,

as noted in this report. However, they are not a sufficient substitute for

a DOD-wide logistics architecture.

7. We agree. The incremental investment actions described, if effectively

implemented, are consistent with our recommendations.

8. See comments 1 and 2.

9. See comment 1.

10. We generally agree. The activities described in the comment are the

very activities that we recommend as conditions for investing beyond

the first increment. Further, it was not our intent to suggest that only

the BSM program manager should measure and use actual benefit

accrual or that one increment must be fully completed and fully

evaluated before another can begin. In fact, effective incremental

investment management necessitates some overlap among increments,

and our recommendations do not preclude this. On the matter of who

should measure and validate actual benefit accrual, we are not

opposed to DLA assigning this responsibility to someone other than

the program manager, and we acknowledge that the program manager

is not the appropriate person to ensure that investment decisions on

subsequent increments are based on economic justifications.

Accordingly, we have slightly modified the recommendation and

clarified language elsewhere in the report to reflect this.

11. We agree. We have changed the title of the report.

 1 A Practical Guide to Federal Enterprise Architecture, Version 1.0, Chief Information

Officers Council (February 2001).
Appendix II: Comments From

the Department of Defense

Page 56 GAO-01-631 Information Technology

12. We disagree. The statement in our report is supported by DLA’s draft

Enterprise Information Architecture Plan, which clearly shows that the

initial component of DLA’s enterprise architecture will be, by design,

limited to the materiel management business area. The plan also shows

that DLA’s other business areas are to be incorporated into the

enterprise architecture following completion of the BSM program,

which, according to BSM documents, is 5 years from now.

13. Our report has given the department credit for its DOD Logistics

Strategic Plan, as well as Joint Vision 2020 and other strategic planning

directions. However, as previously stated, these plans are not a

sufficient substitute for an effective DOD-wide logistics management

enterprise architecture. Moreover, in its comments, DOD states that

having an architecture-based approach to investing in modernized

systems is a more efficient means to optimizing enterprise

performance and accountability. DOD also states that having an

architecture would provide a more efficient and effective means for

developing and overseeing the implementation of a common vision.

14. We disagree. While we have recognized these “interim measures” in

our report, our position is that these are just what DOD terms them—

temporary controls until DOD has in place the architecture-based

approach to modernization that its comments state are needed to

achieve end-to-end optimization. See also comment 3.

15. We disagree. Notwithstanding DLA’s recent commitments to

incrementally invest in BSM, DLA has so far treated BSM as a single

investment decision. The economic analysis used to justify DLA’s

decision to invest in BSM is dated April 2000, and this analysis was for

the entire program. We requested any further economic analyses

developed to justify the ongoing BSM increment as well as any plans to

produce such analyses. DLA did not provide either. In June 2001, DLA

provided an updated economic analysis, dated March 30, 2001, which

continued to treat BSM as a whole. DOD’s comments are consistent

with statements made by BSM officials at the conclusion of our review,

which were cited in the draft report. We are encouraged by DOD’s

comment that DLA will prepare a business case for each release and

will include a projection of business process performance as well as a

retrospective evaluation of empirical results of the program to that

point.

16. We have modified our report to reflect this change.
Appendix II: Comments From

the Department of Defense

Page 57 GAO-01-631 Information Technology

17. We disagree. As we state in the report, enterprise architectures are

essential tools for effectively and efficiently reengineering business

processes and for implementing and evolving their supporting systems.

Their development, implementation, and maintenance are recognized

hallmarks of successful public and private sector organizations.

Congress, OMB, and the federal CIO Council have recognized the

importance of enterprise architectures.

18. We disagree. See comments 2 and 6. Further, our report does not state

that the lack of an enterprise architecture will result in “bad” effects.

Our report concludes that this lack increases the probability of such

effects, and our report recognizes that other IT management

weaknesses can also be contributing factors.

19. See comment 5.

20. See comment 12.

21. We disagree that our report implies that there is little interaction and

consideration of the enterprise cataloging requirements. Our report

refers to DLA’s expectation that some aspects of other business

functions would be included in the BSM blueprint. At the time of our

review, however, DLA recognized that postponing some business

functions might lead to disconnects, requiring later linkages between

BSM and other business areas to produce an architecture with an

enterprisewide scope. Our point is that because these functions are

interrelated, it makes sense to construct an enterprise architecture to

guide and constrain BSM implementation and thereby optimize DLAwide performance and accountability.

22. See comments 2 and 21.

23. See comment 13.

24. We have modified the report to reflect this comment.

25. We have modified the report to incorporate the additional context

provided in this comment and to recognize when efforts to develop a

DOD-wide logistics enterprise architecture began.

26. We disagree that we have been provided a copy of the document

defining the mission-critical logistics “threads” used in DOD’s Year

2000 testing “in their current status.” While we did receive in 1999 a
Appendix II: Comments From

the Department of Defense

Page 58 GAO-01-631 Information Technology

copy of the document defining these mission-critical logistics threads,

the Director also told us that these were being modified in light of

ongoing modernizations, and DOD’s comments acknowledge that they

have since been modified.

27. We have modified the report to reflect this comment.

28. See comment 23. Also, we have slightly modified the language of our

report.
Appendix III: GAO Contact and

Staff Acknowledgment

Page 59 GAO-01-631 Information Technology

Carl Urie, (202) 512-6231

In addition to the person named above, Katherine Chu-Hickman, John

Christian, Barbara Collier, and Greg Donnellon made key contributions to

this report.

Appendix III: GAO Contact and Staff

Acknowledgment

GAO Contact

Acknowledgments

(310206)

The first copy of each GAO report is free. Additional copies of reports are

$2 each. A check or money order should be made out to the

Superintendent of Documents. VISA and MasterCard credit cards are also

accepted.

Orders for 100 or more copies to be mailed to a single address are

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St., NW (corner of 4th and G Sts. NW)

Washington, DC 20013

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and testimony. To

receive facsimile copies of the daily list or any list from the past 30 days,

please call (202) 512-6000 using a touchtone phone. A recorded menu will

provide information on how to obtain these lists.

Orders by Internet

For information on how to access GAO reports on the Internet, send an email message with “info” in the body to:

Info@www.gao.gov

or visit GAO’s World Wide Web home page at:

http://www.gao.gov

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• E-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

Ordering Information

To Report Fraud,

Waste, and Abuse in

Federal Programs