Information Technology: DLA Needs to Strengthen Its Investment Management Capability
Published: 2002-03-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Defense Logistics Agency (DLA) relies extensively on information technology (IT) to carry out its logistics support mission. This report focuses on DLA's processes for making informed IT investment decisions.
section_title: What GAO Found
paragraphs: Because IT investment management has only recently become an area of management focus and commitment at DLA, the agency's ability to effectively manage IT investments is limited. The first step toward establishing effective investment management is putting in place foundational, project-level control and selection processes. The second step toward effective investment management is to continually assess proposed and ongoing projects as an integrated and competing set of investment options. Accomplishing these two steps requires effective development and implementation of a plan, supported by senior management, which defines and prioritizes investment process improvements. Without a well-defined process improvement plan and controls for implementing it, it is unlikely that the agency will establish a mature investment management capability. As a result, GAO continues to question DLA's ability to make informed and prudent investment decisions in IT.

section_title: Letter
paragraphs: The Defense Logistics Agency (DLA) plays a critical role in supporting America’s military forces worldwide. To fulfill this role, DLA employs about 28,000 civilian and military workers, located at about 500 sites in all 50 states and in 28 countries; in round numbers, it manages 4 million supply items and processes 30 million annual supply distribution actions. In fiscal year 2001, DLA reported that these operations resulted in sales to the military services of about $15.2 billion. To perform its logistics support mission, the agency relies extensively on information technology (IT). For fiscal year 2002, DLA’s IT budget is about $654 million.
This report is one in a series of products to respond to the fiscal year 2001 Defense Authorization Act. That act directs that GAO review DLA’s efficiency and effectiveness in meeting customer requirements, application of best business practices, and opportunities for improving agency operations.
This report focuses on DLA’s processes for making informed IT investment decisions. As agreed with your offices, our objectives were to determine (1) whether DLA has effective IT investment management processes needed to modernize and maintain systems and (2) what actions the agency has planned to improve these processes.
subsections: section_title: Background
paragraphs: DLA is DOD’s logistics manager for all departmental consumable items  and some repair parts. Its primary business function is materiel  management: providing supply support to sustain military operations and  readiness. In addition, DLA performs five other supply-related business  functions: distributing materiel from DLA and service-owned inventories,  purchasing fuels for DOD and the U.S. government, storing strategic  materiel, marketing surplus DOD materiel for reuse and disposal, and  providing numerous information services, such as item cataloging, for  DOD and the U.S. government, as well as selected foreign governments.  These six business functions are managed by field commands that report  to and support the agency’s central command authority.
In 2000, DLA refocused its logistics mission from that of a supplier of  materiel to a manager of supply chain relationships. To support this  transition, the agency developed a strategic plan (known as DLA 21) to  reengineer and modernize its operations. Among the goals of DLA 21 are  to optimize inventories, improve efficiency, increase effectiveness through  organizational redesign, reduce inventories, and modernize business  systems.
DLA relies on over 650 systems to support warfighters by allowing access  to global inventories. Whether it is ensuring that there is enough fuel to  service an aircraft fleet, providing sufficient medical supplies to protect  and treat military personnel, or supplying ample food rations to our  soldiers on the frontlines, information technology plays a key role in  ensuring that Defense Department agencies are prepared for their  missions. Because of its heavy reliance on IT to accomplish its mission,  DLA invests extensively in this area. For fiscal year 2002, DLA’s IT budget  is about $654 million.
subsections: section_title: Prior Reviews Identified IT Management Weaknesses
paragraphs: Our recent reviews of DLA’s IT management have identified weaknesses in  such important areas as enterprise architecture management, incremental  investment management, and software acquisition management.
In June 2001, we reported that DLA did not have an enterprise architecture  to guide the agency’s investment in its Business Systems Modernization  (BSM) project—the agency’s largest IT project. The use of an enterprise  architecture, which describes an organization’s mode of operation in  useful models, diagrams, and narrative, is required by the OMB guidance  that implements the Clinger-Cohen Act of 1996 and is a commercial best  practice. Such a “blueprint” can help clarify and optimize the  dependencies and relationships among an agency’s business operations  and the IT infrastructure and applications supporting them. An effective  architecture describes both the environment as it is and the target  environment that an organization is aiming for (as well as a plan for the  transition from one to the other). We concluded that without this  architecture, DLA will be challenged in its efforts to successfully acquire  and implement BSM.
Further, we reported that DLA was not managing its investment in BSM in  an incremental manner, as required by the Clinger-Cohen Act of 1996 and  OMB guidance and in accordance with best commercial practices. An  incremental approach to investment helps to minimize the risk associated  with such large-scale projects as BSM. Accordingly, we recommended that  DLA make the development, implementation, and maintenance of an  enterprise architecture an agency priority and take steps to incrementally  justify and validate its investment in BSM. According to DLA officials, the  agency is addressing these issues.
In January 2002, we reported a wide disparity in the rigor and discipline of  software acquisition processes between two DLA systems. Such  inconsistency in processes for acquiring software (the most costly and  complex component of systems) can lead to the acquisition of systems  that do not meet the information needs of management and staff, do not  provide support for necessary programs and operations, and cost more  and take longer than expected to complete.
We also reported that DLA did not have a software process-improvement  program in place to effectively strengthen its corporate software  acquisition processes, having eliminated the program in 1998. Without a  management-supported software process-improvement program, it is  unlikely that DLA can effectively improve its institutional software  acquisition capabilities, which in turn means that the agency’s software  projects will be at risk of not delivering promised capabilities on time and  within budget. Accordingly, we recommended that DLA institute a  software process-improvement program and correct the software  acquisition process weaknesses that we identified. According to DLA  officials, the agency is addressing each of these issues.
section_title: Information Technology Investment Management (ITIM) Maturity Framework
paragraphs: In May 2000, we issued the Information Technology Investment  Management (ITIM) maturity framework, which identifies critical  processes for successful IT investment and organizes these processes into  an assessment framework comprising five stages of maturity. This  framework supports the fundamental requirements of the Clinger-Cohen  Act of 1996, which requires IT investment and capital planning processes  and performance measurement. Additionally, ITIM can provide a useful  roadmap for agencies when they are implementing specific, fundamental  IT capital planning and investment management practices. The federal  Chief Information Officers Council has favorably reviewed the framework,  and it is also being used by a number of executive agencies and  organizations for designing related policies and procedures and self-led or  contractor-based assessments.
ITIM establishes a hierarchical set of five different maturity stages. Each  stage builds upon the lower stages and represents increased capabilities  toward achieving both stable and effective (and thus mature) IT  investment management processes. Except for the first stage—which  largely reflects ad hoc, undefined, and undisciplined decision and  oversight processes—each maturity stage is composed of critical  processes essential to satisfy the requirements of that stage. These critical  processes are defined by core elements that include organizational  commitment (for example, policies and procedures), prerequisites (for  example, resource allocation), and activities (for example, implementing  procedures). Each core element is composed of a number of key practices.  Key practices are the specific tasks and conditions that must be in place  for an organization to effectively implement the necessary critical  processes.
Figure 1 shows the five ITIM stages and a brief description of each stage.
section_title: Scope and Methodology
paragraphs: Using ITIM, we assessed the extent to which DLA satisfied the five critical  processes in stage 2 of the framework. Based on DLA’s acknowledgment  that it had not executed any of the key practices in stage 3, we did not  independently assess the agency’s capabilities in this stage or stages 4 and  5. To determine whether DLA had implemented the stage 2 critical  processes, we compared relevant DLA policies, procedures, guidance, and  documentation associated with investment management activities to the  key practices and critical processes in ITIM. We rated the key practices as  “executed” based on whether the agency demonstrated (by providing  evidence of performance) that it had met the criteria of the key practice. A  key practice was rated as “not executed” when we found insufficient  evidence of a practice during the review, or when we determined that  there were significant weaknesses in DLA’s execution of the key practice.
As part of our analysis, we selected four IT projects as case studies to  verify application of the critical processes and practices. We selected  projects that (1) supported different DLA business areas (such as materiel  management), (2) were in different lifecycle phases (for example,  requirements definition, design, operations and maintenance),  (3) represented different levels of risk (such as low or medium) as  designated by the agency, and (4) included at least one investment that  required funding approval by a DOD authority outside of DLA (for  example, the Office of the Secretary of Defense (OSD)). The four projects  are the following:
Business Systems Modernization: This system, which supports DLA’s  materiel management business area, is in the concept demonstration  phase of development. DLA reported that it spent about $136 million on  this system in fiscal year 2001, and it has budgeted about $133 million for  fiscal year 2002. BSM is intended to modernize DLA’s materiel  management business function, replacing two of its standard systems (the  Standard Automated Materiel Management System and the Defense  Integrated Subsistence Management System). The project is also intended  to enable the agency to reengineer its logistics practices to reflect best  commercial business practices. For example, in support of DLA’s goal of  reducing its role as a provider and manager of materiel and increasing its  role as a manager of supply chain relationships, BSM is to help link  customers with appropriate suppliers and to incorporate commercial  business practices regarding physical distribution and financial  management. The agency has classified this project as high risk, and OSD  has funding approval authority for this project.
Hazardous Materials Information System (HMIS): This system, which  supports DLA’s logistics operations function, was implemented in 1978. In  fiscal year 2001, DLA reported that it spent about $1 million on this system  and budgeted about $2.4 million for fiscal year 2002. In 1999 DLA began a  redesign effort to transform HMIS into a Web-based system with a direct  interface to the manufacturers and suppliers of hazardous material. The  project is in the development stage. It contains data on the chemical  composition of materials classified as “hazardous” for the purposes of  usage, storage, and transportation. The system is used by Emergency  Response Teams whenever a spill or accident occurs involving hazardous  materials. The agency classified this project as low risk, and funding  approval occurs within DLA.
The Defense Reutilization and Marketing Automated Information System  (DAISY): This system, which supports DLA’s materiel reuse and disposal  mission, is in the operations and maintenance lifecycle phase. The agency  reported that it spent approximately $4.4 million on DAISY in fiscal year  2001, and it has budgeted about $7 million for fiscal year 2002. This system  is a repository for transactions involving the reutilization, transfer,  donation, sale, or ultimate disposal of excess personal property from DOD,  federal, and state agencies. The excess property includes spare and repair  parts, scrap and recyclable material, precious metals recovery, hazardous  material, and hazardous waste disposal. Operated by the Defense  Reutilization and Marketing Service, the system is used at 190 locations  worldwide. The agency classified this project as low risk, and funding  approval occurs within DLA.
Standard Automated Materiel Management System (SAMMS): This system,  which supports DLA’s materiel management business area, is 30 years old  and approaching the end of its useful life. The agency reports that  investment in SAMMS (budgeted at approximately $19 million for fiscal  year 2002) is directed toward keeping the system operating until its  replacement, BSM, becomes fully operational (scheduled for fiscal year  2005). This system provides the Inventory Control Points with information  regarding stock levels, as well as with the capabilities required for  (1) acquisition and management of wholesale consumable items, (2) direct  support for processing requisitions, (3) forecasting of requirements,  (4) generation of purchase requests, (5) maintenance of technical data,  (6) financial management, (7) identification of items, and (8) asset  visibility. The agency has classified the maintenance of SAMMS as a low  risk effort, and funding approval occurs within DLA.
For these projects, we reviewed project management documentation, such  as mission needs statements, project plans, and status reports. We also  analyzed charters and meeting minutes for DLA oversight boards, DLA’s  draft Automated Information System Emerging Program Life Management  (LCM) Review and Milestone Approval Directive and Portfolio  Management and Oversight Directives, and DOD’s 5000 series guidance on  systems acquisition. In addition, we reviewed documentation related to  the agency’s self-assessment of its IT investment operations.
To supplement our document reviews, we interviewed senior DLA  officials, including the vice director (who sits on the Corporate Board,  DLA’s highest level investment decisionmaking body), the chief  information officer (CIO), the chief financial officer, and oversight board  members. We also interviewed the program managers of our four case  study projects, as well as officials responsible for managing the IT  investment process  and other staff within Information Operations.
To determine what actions DLA has taken to improve its IT investment  management processes, we interviewed the CIO and officials of the Policy,  Plans, and Assessments and the program executive officer (PEO)  operations groups within the Information Operations Directorate. These  groups are primarily responsible for implementing investment  management process improvements. We also reviewed a draft list of IT  investment management improvement tasks.
We conducted our work at DLA headquarters in Fort Belvoir, Virginia,  from June 2001 through January 2002, in accordance with generally  accepted government auditing standards.
section_title: DLA’s Capabilities to Effectively Manage IT Investments Are Limited
paragraphs: In order to have the capabilities to effectively manage IT investments, an  agency should (1) have basic, project-level control and selection practices  in place and (2) manage its projects as a portfolio of investments, treating  them as an integrated package of competing investment options and  pursuing those that best meet the strategic goals, objectives, and mission  of the agency.
DLA has a majority of the project-level practices in place. However, it is  missing several crucial practices, and it is not performing portfolio-based  investment management. According to the CIO, the evolving state of its  investment management capabilities is the result of agency leadership’s  recently viewing IT investment management as an area of management  focus and priority. Without having crucial processes and related practices  in place, DLA lacks essential management controls over its sizable IT  investments.
subsections: section_title: Many Stage 2 Strengths, but Key Weaknesses Exist in Project-Level Control and Selection Processes
paragraphs: At ITIM stage 2 maturity, an organization has attained repeatable,  successful IT project-level investment control processes and basic  selection processes. Through these processes, the organization can  identify expectation gaps early and take appropriate steps to address  them. According to ITIM, critical processes at stage 2 include (1) defining  investment board operations, (2) collecting information about existing  investments, (3) developing project-level investment control processes,  (4) identifying the business needs for each IT project, and (5) developing a  basic process for selecting new IT proposals. Table 1 discusses the  purpose for each of the stage 2 critical processes.
To its credit, DLA has put in place about 75 percent of the key practices  associated with stage 2 critical processes. For example, DLA has oversight  boards to perform investment management functions, and it has basic  project-level control processes to help ensure that IT projects are meeting  cost and schedule expectations.
However, DLA has not executed several crucial stage 2 investment  practices. For example, the business needs for IT projects are not always  clearly identified and defined, basic investment selection processes are  still being developed, and policies and procedures for project oversight are  not documented. Table 2 summarizes the status of DLA’s stage 2 critical  processes, showing how many associated key practices the agency has  executed. DLA’s actions in each of the critical processes are discussed in  the sections that follow.
To help ensure executive management accountability for IT capital  planning and investment decisions, an organization should establish a  governing board or boards responsible for selecting, controlling, and  evaluating IT investments. According to ITIM, effective IT investment  board operations require, among other things, that (1) board membership  have both IT and business knowledge, (2) board members understand the  investment board’s policies and procedures and exhibit core competencies  in using the agency’s IT investment policies and procedures, (3) the  organization’s executives and line managers support and carry out board  decisions, (4) the organization create organization-specific process  guidance that includes policies and procedures to direct the board’s  operations, and (5) the investment board operate according to written  policies and procedures. (The full list of key practices is provided in  table 3.)
DLA has established several oversight boards that perform IT investment  management functions. These boards include the following:
The DLA Investment Council, which is intended to review, evaluate, and  approve new IT and non-IT investments between $100,000 and $1,000,000.
The Program Executive Officer Review Board, which is intended to review  and approve the implementation of IT investments that are budgeted for  over $25 million in all or over $5 million in any one year.
The Corporate Board, which is intended to review, evaluate, and approve  all IT and non-IT investments over $1 million.
DLA is executing four of the six key practices needed for these boards to  operate effectively. For example, the membership of these boards  integrates both IT and business knowledge. In addition, board members  informed us of their understanding of their board’s informal practices.  Further, according to IT investment officials, project managers, and  agency documentation, the boards have a process for ensuring that their  decisions are supported and carried out by organization executives and  line managers. This process involves documenting board decisions in  meeting minutes, assigning staff to carry out the decisions, and tracking  the actions taken on a regular basis until the issues are addressed.
Nonetheless, DLA is missing the key ingredient associated with two of the  board oversight practices that are needed to operate effectively—  organization-specific guidance. This guidance, which serves as official  operations documentation, should (1) clearly define the roles of key  people within its IT investment process, (2) delineate the significant  events and decision points within the processes, (3) identify the external  and environmental factors that will influence the processes (that is, legal  constraints, the behavior of key subordinate agencies and military  customers, and the practices of commercial logistics that DLA is trying to  emulate as part of DLA 21); and (4) explain how IT investment-related  processes will be coordinated with other organizational plans and  processes. DLA does not have guidance that sufficiently addresses these  issues. Policies and procedures governing operations are in draft for one  board and have not been developed for the two other boards. Without this  guidance governing the operations of the investment boards, the agency is  at risk of performing key investment decisionmaking activities  inconsistently. Such guidance would also provide a degree of transparency  that is helpful in both communicating and demonstrating how these  decisions are made.
Table 3 summarizes the ratings for each key practice and the specific  findings supporting the ratings.
An IT project inventory provides information to investment decision- makers to help evaluate the impacts and opportunities created by  proposed or continuing investments. This inventory (which can take many  forms) should, at a minimum, identify the organization’s IT projects  (including new and existing systems) and a defined set of relevant  investment management information about them (for example, purpose,  owner, lifecycle stage, budget cost, physical location, and interfaces with  other systems). Information from the IT project inventory can, for  example, help identify systems across the organization that provide similar  functions and help avoid the commitment of additional funds for  redundant systems and processes. It can also help determine more precise  development and enhancement costs by informing decisionmakers and  other managers of interdependencies among systems and how potential  changes in one system can affect the performance of other systems.
According to ITIM, effectively managing an IT project inventory requires,  among other things, (1) identifying IT projects, collecting relevant  information about them, and capturing this information in a repository,  (2) assigning responsibility for managing the IT project inventory process  to ensure that the inventory meets the needs of the investment  management process, (3) developing written policies and procedures for  maintaining the IT project inventory, (4) making information from the  inventory available to staff and managers throughout the organization so  they can use it, for example, to build business cases and to support project  selection and control activities, and (5) maintaining the IT project  inventory and its information records to contribute to future investment  selections and assessments. (The full list of key practices is provided in  table 4.)
DLA has executed many of the key practices in this critical process. For  example, according to DLA’s CIO, IT projects are identified and specific  information about them is entered into a central repository called the DLA  Profile System (DPS). DPS includes, among other things, project  descriptions, key contact information, lifecycle stage, and system  interfaces. In addition, the CIO is responsible for managing the IT project  identification process to ensure that DPS meets the needs of the  investment management process. However, DLA has not defined written  policies and procedures for how and when users should add to or update  information in the DPS. In addition, DLA is not maintaining DPS records,  which would be useful during future project selections and investment  evaluations, and for documenting the evolution of a project’s development.  Without appropriate policies and procedures in place to describe the  objectives and information requirements of the inventory, DPS is not being  maximized as an effective tool to assist in the fundamental analysis  essential to effective decisionmaking.
Table 4 summarizes the ratings for each key practice and the specific  findings supporting the ratings.
Investment review boards should effectively oversee IT projects  throughout all lifecycle phases (concept, design, development, testing,  implementation, and operations/maintenance). At stage 2 maturity,  investment review boards should review each project’s progress toward  predefined cost and schedule expectations, using established criteria and  performance measures, and should take corrective actions to address cost  and milestone variances.
According to ITIM, effective project oversight requires, among other  things, (1) having written polices and procedures for project management,  (2) developing and maintaining an approved management plan for each IT  project, (3) having written policies and procedures for oversight of IT  projects, (4) making up-to-date cost and schedule data for each project  available to the oversight boards, (5) reviewing each project’s  performance by regularly comparing actual cost and schedule data to  expectations, (6) ensuring that corrective actions for each under- performing project are documented, agreed to, implemented, and tracked  until the desired outcome is achieved, and (7) using information from the  IT project inventory. (The complete list of key practices is provided in  table 5.)
DLA has executed most of the key practices in this area. In particular, DLA  relies on the guidance in the Department of Defense 5000 series directives  for project management and draft guidance in an Automated Information  System (AIS) Emerging Program Life-Cycle Management (LCM) Review  and Milestone Approval Directive for specific IT project management. In  addition, for each of the four projects we reviewed, a project management  plan had been approved, and cost and schedule controls were addressed  during project review meetings. Further, based on our review of project  documentation and in discussion with project managers, up-to-date cost  and schedule project data were provided to the PEO Review Board. This  board oversees project performance regularly by comparing actual cost  and schedule data to expectations and has a process for ensuring that, for  underperforming projects, corrective actions are documented, agreed to,  and tracked.
Notwithstanding these strengths, DLA has some weaknesses in project  oversight. Specifically, although the Corporate Board and the Investment  Council have written charters, there are no written policies or procedures  that define their role in collectively overseeing IT projects. Without these  policies and procedures, project oversight may be inconsistently applied,  leading to the risk that performance problems, such as cost overruns and  schedule slippages, may not be identified and resolved in a timely manner.
In addition, according to representatives from the oversight boards, they  do not use information from the IT project inventory to oversee projects  because they are more comfortable using more traditional methods of  obtaining and using information (that is, informally talking with subject  matter experts and relying on experience). The inventory is of value only  to the extent that decisionmakers use it. As discussed earlier, while the  inventory need not be the only source of information, it should  nevertheless serve as a reliable and consistent tool for understanding  project and overall portfolio decisions.
Table 5 summarizes the ratings for each key practice and the specific  findings supporting the ratings.
Defining business needs for each IT project helps ensure that projects  support the organization’s mission goals and meets users’ needs. This  critical process creates the link between the organization’s business  objectives and its IT management strategy. According to ITIM, effectively  identifying business needs requires, among other things, (1) defining the  organization’s business needs or stated mission goals, (2) identifying users  for each project who will participate in the project’s development and  implementation, (3) training IT staff adequately in identifying business  needs, and (4) defining business needs for each project. (The complete list  of key practices is provided in table 6.)
DLA has executed all but one of the key practices associated with  effectively defining business needs for IT projects. For example, DLA’s  mission goals are described in DLA’s strategic plan. In addition, according  to IT investment management officials, the IT staff is adequately trained in  identifying business needs because they generally have prior functional  unit experience. In addition, according to DLA directives, IT projects are  assigned an Integrated Process Team (IPT) to guide and direct the project  through the development lifecycle. The IPTs are composed of IT and  functional staff. Moreover, DOD and DLA directives require that business  requirements and system users be identified and that users participate in  the lifecycle management of the project. According to an IT investment  official, each IT project has a users’ group that meets throughout the  lifecycle to discuss problems and potential changes related to the system.  We verified that this was the case for the four projects we reviewed.
While the business needs for three of the four projects we reviewed were  clearly identified and defined,  DLA has reported that this has not been  consistently done for all IT projects. According to IT investment  management officials, this inconsistency arose because policies and  procedures for developing business needs were not always followed or  required.
DLA officials have stated that they are developing new guidance to  address this problem. However, until this guidance is implemented and  enforced, DLA cannot effectively demonstrate that priority mission and  business improvement needs are forming the basis for all its IT investment  decisions.
Table 6 summarizes the ratings for each key practice and the specific  findings supporting the ratings.
Selecting new IT proposals requires an established and structured process  to ensure informed decisionmaking and infuse management  accountability. According to ITIM, this critical process requires, among  other things, (1) making funding decisions for new IT proposals according  to an established process, (2) providing adequate resources for proposal  selection activities, (3) using an established proposal selection process,  (4) analyzing and ranking new IT proposals according to established  selection criteria, including cost and schedule criteria, and (5) designating  an official to manage the proposal selection process. (The complete list of  key practices is provided in table 7.)
DLA has executed some of the key practices for investment proposal  selection. For example, DLA executives make funding decisions for IT  investments using DOD’s Program Objective Memorandum (POM)  process, which is part of DOD’s annual budgeting process. Through this  process, proposals for new projects or enhancements to ongoing projects  are evaluated by DLA’s IT and financial groups and submitted to OSD  through DLA’s Corporate Board with recommendations for funding  approval. In addition, according to the CIO, adequate resources have been  provided to carry out activities related to the POM process.
Nonetheless, DLA has yet to execute some of the critical practices related  to this process area. Specifically, DLA acknowledges that the agency is not  analyzing and prioritizing new IT proposals according to established  selection criteria. Instead, the Corporate Board uses the expertise from the  IT organization and its own judgment to analyze and prioritize projects. To  its credit, DLA recognizes that it cannot continue to rely solely on the POM  process to make sound IT investment selection decisions. Therefore, the  agency has been working to establish an IT selection process over the past  two budget cycles that is more investment-focused and includes increased  involvement from IT Operations staff, necessary information, and  established selection criteria.
Until DLA implements an effective IT investment selection process that is  well established and understood throughout the agency, executives cannot  be adequately assured that they are consistently and objectively selecting  proposals that best meet the needs and priorities of the agency.
Table 7 summarizes the ratings for each key practice and the specific  findings supporting the ratings.
section_title: IT Investments Are Not Being Managed as a Complete Portfolio
paragraphs: An IT investment portfolio is an integrated, enterprisewide collection of  investments that are assessed and managed collectively based on common  criteria. Managing investments within the context of such a portfolio is a  conscious, continuous, and proactive approach to expending limited  resources on an organization’s competing initiatives in light of the relative  benefits expected from these investments. Taking an enterprisewide  perspective enables an organization to consider its investments  comprehensively so that the collective investments optimally address its  mission, strategic goals, and objectives. This portfolio approach also  allows an organization to determine priorities and make decisions about  which projects to fund based on analyses of the relative organizational  value and risks of all projects, including projects that are proposed, under  development, and in operation.
According to ITIM, stage 3 maturity includes (1) defining portfolio  selection criteria, (2) engaging in project-level investment analysis,  (3) developing a complete portfolio based on the investment analysis,  (4) maintaining oversight over the investment performance of the  portfolio, and (5) aligning the authority of IT investment boards.
Table 8 describes the purposes for the critical processes in stage 3.
According to DLA officials, they are currently focusing on implementing  stage 2 processes and have not implemented any of the critical processes  in stage 3. Until the agency fully implements both stage 2 and 3 processes,  it cannot consider investments in a comprehensive manner and determine  whether it has the appropriate mix of IT investments to best meet its  mission needs and priorities.
section_title: DLA Lacks a Plan to Guide Improvement Efforts
paragraphs: DLA recognizes the need to improve its IT investment processes, but it has  not yet developed a plan for systematically correcting weaknesses. To  properly focus and target IT investment process improvements, an  organization should fully identify and assess current process strengths and  weaknesses (that is, create an investment management capability  baseline) as the first step in developing and implementing an improvement  plan. As we have previously reported, this plan should, at a minimum,  (1) specify measurable goals, objectives, milestones, and needed  resources, and (2) clearly assign responsibility and accountability for  accomplishing well-defined tasks. The plan should also be documented  and approved by agency leadership. In implementing the plan, it is  important that DLA measure and report progress against planned  commitments, and that appropriate corrective action be taken to address  deviations.
DLA does not have such a plan. In March 2001, it attempted to baseline  agency IT operations by reviewing its project-level investment  management practices using ITIM. This effort identified practice strengths  and weaknesses, but DLA considered the assessment to be preliminary (to  be followed by a more comprehensive assessment at an unspecified later  date) and limited in scope. DLA used the assessment results to establish  broad milestones for strengthening its investment management process.  The agency did not, however, develop a complete process improvement  plan. For example, it did not (1) specify required resources to accomplish  the various tasks, (2) clearly assign responsibility and accountability for  accomplishing the tasks, (3) obtain support from senior level officials, and  (4) establish performance measures to evaluate the effectiveness of the  completed tasks. At the same time, the agency has separately begun other  initiatives to improve its investment management processes, but these  initiatives are not aligned with the established milestones or with each  other. The DLA CIO characterizes the agency’s approach to its various  process improvement efforts as a necessary progression that includes  some inevitable “trial and error” as it moves toward a complete process  improvement plan.
Without such a plan that allows the agency to systematically prioritize,  sequence, and evaluate improvement efforts, DLA jeopardizes its ability to  establish a mature investment process that includes selection and control  capabilities that result in greater certainty about future IT investment  outcomes.
section_title: Conclusions
paragraphs: Until recently, IT investment management has not been an area of DLA  management attention and focus. As a result, DLA currently finds itself  without some of the capabilities that it needs to ensure that its mix of IT  investments best meets the agency’s mission and business priorities. To its  credit, DLA now recognizes the need to strengthen its IT investment  management and has taken positive steps to begin doing so. However,  several critical IT investment management capabilities need to be  enhanced before DLA can have reasonable assurance that it is maximizing  the value of its IT investment dollar and minimizing the associated risks.  Moreover, DLA does not yet have a process improvement plan that is  endorsed and supported by agency leadership. The absence of such a plan  limits DLA’s prospects for introducing the management capabilities  necessary for making prudent decisions that maximize the benefits and  minimize the risks of its IT investment.
section_title: Recommendations
paragraphs: To strengthen DLA’s investment management capability and address the  weaknesses discussed in this report, we recommend that the secretary of  defense direct the DLA director to designate the development and  implementation of effective IT investment management processes as an  agencywide priority.
Further, we recommend that the secretary of defense have the DLA  director do the following:
Develop a plan, within 6 months, for implementing IT investment  management process improvements that is based on GAO’s ITIM stage 2  and 3 critical processes.
Ensure that the plan specifies measurable goals and time frames, defines a  management structure for directing and controlling the improvements, and  establishes review milestones.
Ensure that the plan focuses first on correcting the weakness in the ITIM  stage 2 critical processes, because these processes collectively provide the  foundation for building a mature IT investment management process.  Specifically:
Develop and issue guidance covering the scope and operations of  DLA’s investment review boards. Such guidance should include, at a  minimum, specific definitions of the roles and responsibilities within  the IT investment process; an outline of the significant events and  decision points within the processes; an identification of the external  and environmental factors that will influence the processes (for  example, legal constraints, the behavior of key suppliers or customers,  or industry norms), and the manner in which IT investment-related  processes will be coordinated with other organization plans and  processes.
Develop and issue policies and procedures for maintaining DLA’s IT  projects inventory for investment management purposes.
Finalize and issue policies and procedures (including the use of  information from the IT systems and project inventory) for the PEO  Review Board’s oversight of IT projects.
Develop and issue similar policies and procedures for the other  investment boards.
Finalize and issue guidance supporting the identification of business  needs and implementing management controls to ensure that proposals  submitted to DLA for review clearly identify and define business  requirements.
Develop and issue guidance for the proposal selection process in such  a way that the criteria for selection are clearly set forth, including  formally assigning responsibility for managing the proposal selection  process and establishing management controls to ensure that the  proposal selection process is working effectively.
Ensure that the plan next focuses on stage 3 critical processes, which are  necessary for portfolio management, because along with the stage 2  foundational processes, these processes are necessary for effective  management of IT investments.  Implement the approved plan and report on progress made against the  plan’s goals and time frames to the secretary of defense every 6 months.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided what it termed “official oral comments” from the director  for acquisition resources and analysis on a draft of this report. In its  comments, DOD concurred with our recommendations and described  efforts under way and planned to implement them. However, it  recommended that two report captions be changed to more accurately  reflect, in DOD’s view, the contents of the report and to eliminate false  impressions.
Specifically, DOD recommended that we change one caption from “DLA’s  Capabilities to Effectively Manage IT Investments Are Limited” to “DLA’s  Capabilities to Effectively Manage IT Investments Should Be Improved.”
DOD stated that this change is needed to recognize the fact that DLA has  completed about 75 percent of the practices associated with stage 2  critical processes. We do not agree. As stated in our report, to effectively  manage IT investments an agency should (1) have basic, project-level  control and selection practices in place (stage 2 processes) and  (2) manage its projects as a portfolio of investments (stage 3 processes).  Although DLA has executed most of the key practices associated with  stage 2 processes, the agency acknowledges that it has not implemented  any of the stage 3 processes. Therefore, our caption as written describes  DLA’s IT investment management capabilities appropriately.
In addition, DOD recommended that we change the caption “DLA Lacks a  Plan to Guide Improvement Efforts” to “DLA Lacks a Published Plan to  Guide Improvement Efforts.” DOD stated that this change is needed  because DLA has developed some elements of an implementation plan. We  do not agree. Our point is that DLA did not have a complete process  improvement plan, not that it has yet to publish the plan that it has. As we  describe in the report, a complete plan should, at a minimum, (1) be based  on a full assessment of process strengths and weaknesses, (2) specify  measurable goals, objectives, milestones, and needed resources,  (3) clearly assign responsibility and accountability for accomplishing well- defined tasks, and (4) be documented and approved by agency leadership.
In contrast, DLA’s planning document was based on a preliminary  assessment of only stage 2 critical processes and lacked several of the  critical attributes listed above. Moreover, DOD stated in its comments that  DLA has not completed a formally documented and prioritized  implementation plan to resolve stage 2 and 3 practice weaknesses and has  yet to complete the self-assessment and gap analysis necessary to define  planned action items. Accordingly, it is clear that DLA has not satisfied the  tenets of a complete plan, and thus our caption is accurate as written.
DOD provided additional comments that we have incorporated as  appropriate in the report.
We are sending copies of this report to the chairmen and ranking minority  members of the Subcommittee on Defense, Senate Committee on  Appropriations; the Subcommittee on Readiness and Management  Support, Senate Committee on Armed Services; the Subcommittee on  Defense, House Committee on Appropriations; and the Subcommittee on  Military Readiness, House Committee on Armed Services. We are also  sending copies to the director, Office of Management and Budget; the  secretary of defense; the under secretary of defense for acquisition, technology, and logistics; the deputy under secretary of defense for logistics and materiel readiness; and the director, Defense Logistics Agency. Copies will be made available to others upon request.
If you have any questions regarding this report, please contact us at (202) 512-3439 and (202) 512-7351, respectively, or by e-mail at hiter@gao.gov and mcclured@gao.gov. An additional GAO contact and staff acknowledgments are listed in appendix II.
section_title: Appendix I: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the individual named above, key contributors to this report  were Barbara Collier, Lester Diamond, Gregory Donnellon, Sabine Paul,  and Eric Trout.