Information Technology: DLA Needs to Strengthen Its Investment Management Capability
Released: 2002-03-15T00:00:00

Letter

Background

DLA is DOD’s logistics manager for all departmental consumable items and some repair parts. Its primary business function is materiel management: providing supply support to sustain military operations and readiness. In addition, DLA performs five other supply-related business functions: distributing materiel from DLA and service-owned inventories, purchasing fuels for DOD and the U.S. government, storing strategic materiel, marketing surplus DOD materiel for reuse and disposal, and providing numerous information services, such as item cataloging, for DOD and the U.S. government, as well as selected foreign governments. These six business functions are managed by field commands that report to and support the agency’s central command authority.

In 2000, DLA refocused its logistics mission from that of a supplier of materiel to a manager of supply chain relationships. To support this transition, the agency developed a strategic plan (known as DLA 21) to reengineer and modernize its operations. Among the goals of DLA 21 are to optimize inventories, improve efficiency, increase effectiveness through organizational redesign, reduce inventories, and modernize business systems.

DLA relies on over 650 systems to support warfighters by allowing access to global inventories. Whether it is ensuring that there is enough fuel to service an aircraft fleet, providing sufficient medical supplies to protect and treat military personnel, or supplying ample food rations to our soldiers on the frontlines, information technology plays a key role in ensuring that Defense Department agencies are prepared for their missions. Because of its heavy reliance on IT to accomplish its mission, DLA invests extensively in this area. For fiscal year 2002, DLA’s IT budget is about $654 million.

Prior Reviews Identified IT Management Weaknesses

Our recent reviews of DLA’s IT management have identified weaknesses in such important areas as enterprise architecture management, incremental investment management, and software acquisition management.

In June 2001, we reported that DLA did not have an enterprise architecture to guide the agency’s investment in its Business Systems Modernization (BSM) project—the agency’s largest IT project. The use of an enterprise architecture, which describes an organization’s mode of operation in useful models, diagrams, and narrative, is required by the OMB guidance that implements the Clinger-Cohen Act of 1996 and is a commercial best practice. Such a “blueprint” can help clarify and optimize the dependencies and relationships among an agency’s business operations and the IT infrastructure and applications supporting them. An effective architecture describes both the environment as it is and the target environment that an organization is aiming for (as well as a plan for the transition from one to the other). We concluded that without this architecture, DLA will be challenged in its efforts to successfully acquire and implement BSM.

Further, we reported that DLA was not managing its investment in BSM in an incremental manner, as required by the Clinger-Cohen Act of 1996 and OMB guidance and in accordance with best commercial practices. An incremental approach to investment helps to minimize the risk associated with such large-scale projects as BSM. Accordingly, we recommended that DLA make the development, implementation, and maintenance of an enterprise architecture an agency priority and take steps to incrementally justify and validate its investment in BSM. According to DLA officials, the agency is addressing these issues.

In January 2002, we reported a wide disparity in the rigor and discipline of software acquisition processes between two DLA systems. Such inconsistency in processes for acquiring software (the most costly and complex component of systems) can lead to the acquisition of systems that do not meet the information needs of management and staff, do not provide support for necessary programs and operations, and cost more and take longer than expected to complete.

We also reported that DLA did not have a software process-improvement program in place to effectively strengthen its corporate software acquisition processes, having eliminated the program in 1998. Without a management-supported software process-improvement program, it is unlikely that DLA can effectively improve its institutional software acquisition capabilities, which in turn means that the agency’s software projects will be at risk of not delivering promised capabilities on time and within budget. Accordingly, we recommended that DLA institute a software process-improvement program and correct the software acquisition process weaknesses that we identified. According to DLA officials, the agency is addressing each of these issues.

Information Technology Investment Management (ITIM) Maturity Framework

In May 2000, we issued the Information Technology Investment Management (ITIM) maturity framework, which identifies critical processes for successful IT investment and organizes these processes into an assessment framework comprising five stages of maturity. This framework supports the fundamental requirements of the Clinger-Cohen Act of 1996, which requires IT investment and capital planning processes and performance measurement. Additionally, ITIM can provide a useful roadmap for agencies when they are implementing specific, fundamental IT capital planning and investment management practices. The federal Chief Information Officers Council has favorably reviewed the framework, and it is also being used by a number of executive agencies and organizations for designing related policies and procedures and self-led or contractor-based assessments.

ITIM establishes a hierarchical set of five different maturity stages. Each stage builds upon the lower stages and represents increased capabilities toward achieving both stable and effective (and thus mature) IT investment management processes. Except for the first stage—which largely reflects ad hoc, undefined, and undisciplined decision and oversight processes—each maturity stage is composed of critical processes essential to satisfy the requirements of that stage. These critical processes are defined by core elements that include organizational commitment (for example, policies and procedures), prerequisites (for example, resource allocation), and activities (for example, implementing procedures). Each core element is composed of a number of key practices. Key practices are the specific tasks and conditions that must be in place for an organization to effectively implement the necessary critical processes.

Figure 1 shows the five ITIM stages and a brief description of each stage.

Scope and Methodology

Using ITIM, we assessed the extent to which DLA satisfied the five critical processes in stage 2 of the framework. Based on DLA’s acknowledgment that it had not executed any of the key practices in stage 3, we did not independently assess the agency’s capabilities in this stage or stages 4 and 5. To determine whether DLA had implemented the stage 2 critical processes, we compared relevant DLA policies, procedures, guidance, and documentation associated with investment management activities to the key practices and critical processes in ITIM. We rated the key practices as “executed” based on whether the agency demonstrated (by providing evidence of performance) that it had met the criteria of the key practice. A key practice was rated as “not executed” when we found insufficient evidence of a practice during the review, or when we determined that there were significant weaknesses in DLA’s execution of the key practice.

As part of our analysis, we selected four IT projects as case studies to verify application of the critical processes and practices. We selected projects that (1) supported different DLA business areas (such as materiel management), (2) were in different lifecycle phases (for example, requirements definition, design, operations and maintenance), (3) represented different levels of risk (such as low or medium) as designated by the agency, and (4) included at least one investment that required funding approval by a DOD authority outside of DLA (for example, the Office of the Secretary of Defense (OSD)). The four projects are the following:

Business Systems Modernization: This system, which supports DLA’s materiel management business area, is in the concept demonstration phase of development. DLA reported that it spent about $136 million on this system in fiscal year 2001, and it has budgeted about $133 million for fiscal year 2002. BSM is intended to modernize DLA’s materiel management business function, replacing two of its standard systems (the Standard Automated Materiel Management System and the Defense Integrated Subsistence Management System). The project is also intended to enable the agency to reengineer its logistics practices to reflect best commercial business practices. For example, in support of DLA’s goal of reducing its role as a provider and manager of materiel and increasing its role as a manager of supply chain relationships, BSM is to help link customers with appropriate suppliers and to incorporate commercial business practices regarding physical distribution and financial management. The agency has classified this project as high risk, and OSD has funding approval authority for this project.

Hazardous Materials Information System (HMIS): This system, which supports DLA’s logistics operations function, was implemented in 1978. In fiscal year 2001, DLA reported that it spent about $1 million on this system and budgeted about $2.4 million for fiscal year 2002. In 1999 DLA began a redesign effort to transform HMIS into a Web-based system with a direct interface to the manufacturers and suppliers of hazardous material. The project is in the development stage. It contains data on the chemical composition of materials classified as “hazardous” for the purposes of usage, storage, and transportation. The system is used by Emergency Response Teams whenever a spill or accident occurs involving hazardous materials. The agency classified this project as low risk, and funding approval occurs within DLA.

The Defense Reutilization and Marketing Automated Information System (DAISY): This system, which supports DLA’s materiel reuse and disposal mission, is in the operations and maintenance lifecycle phase. The agency reported that it spent approximately $4.4 million on DAISY in fiscal year 2001, and it has budgeted about $7 million for fiscal year 2002. This system is a repository for transactions involving the reutilization, transfer, donation, sale, or ultimate disposal of excess personal property from DOD, federal, and state agencies. The excess property includes spare and repair parts, scrap and recyclable material, precious metals recovery, hazardous material, and hazardous waste disposal. Operated by the Defense Reutilization and Marketing Service, the system is used at 190 locations worldwide. The agency classified this project as low risk, and funding approval occurs within DLA.

Standard Automated Materiel Management System (SAMMS): This system, which supports DLA’s materiel management business area, is 30 years old and approaching the end of its useful life. The agency reports that investment in SAMMS (budgeted at approximately $19 million for fiscal year 2002) is directed toward keeping the system operating until its replacement, BSM, becomes fully operational (scheduled for fiscal year 2005). This system provides the Inventory Control Points with information regarding stock levels, as well as with the capabilities required for (1) acquisition and management of wholesale consumable items, (2) direct support for processing requisitions, (3) forecasting of requirements, (4) generation of purchase requests, (5) maintenance of technical data, (6) financial management, (7) identification of items, and (8) asset visibility. The agency has classified the maintenance of SAMMS as a low risk effort, and funding approval occurs within DLA.

For these projects, we reviewed project management documentation, such as mission needs statements, project plans, and status reports. We also analyzed charters and meeting minutes for DLA oversight boards, DLA’s draft Automated Information System Emerging Program Life Management (LCM) Review and Milestone Approval Directive and Portfolio Management and Oversight Directives, and DOD’s 5000 series guidance on systems acquisition. In addition, we reviewed documentation related to the agency’s self-assessment of its IT investment operations.

To supplement our document reviews, we interviewed senior DLA officials, including the vice director (who sits on the Corporate Board, DLA’s highest level investment decisionmaking body), the chief information officer (CIO), the chief financial officer, and oversight board members. We also interviewed the program managers of our four case study projects, as well as officials responsible for managing the IT investment process and other staff within Information Operations.

To determine what actions DLA has taken to improve its IT investment management processes, we interviewed the CIO and officials of the Policy, Plans, and Assessments and the program executive officer (PEO) operations groups within the Information Operations Directorate. These groups are primarily responsible for implementing investment management process improvements. We also reviewed a draft list of IT investment management improvement tasks.

We conducted our work at DLA headquarters in Fort Belvoir, Virginia, from June 2001 through January 2002, in accordance with generally accepted government auditing standards.

DLA’s Capabilities to Effectively Manage IT Investments Are Limited

In order to have the capabilities to effectively manage IT investments, an agency should (1) have basic, project-level control and selection practices in place and (2) manage its projects as a portfolio of investments, treating them as an integrated package of competing investment options and pursuing those that best meet the strategic goals, objectives, and mission of the agency.

DLA has a majority of the project-level practices in place. However, it is missing several crucial practices, and it is not performing portfolio-based investment management. According to the CIO, the evolving state of its investment management capabilities is the result of agency leadership’s recently viewing IT investment management as an area of management focus and priority. Without having crucial processes and related practices in place, DLA lacks essential management controls over its sizable IT investments.

Many Stage 2 Strengths, but Key Weaknesses Exist in Project-Level Control and Selection Processes

At ITIM stage 2 maturity, an organization has attained repeatable, successful IT project-level investment control processes and basic selection processes. Through these processes, the organization can identify expectation gaps early and take appropriate steps to address them. According to ITIM, critical processes at stage 2 include (1) defining investment board operations, (2) collecting information about existing investments, (3) developing project-level investment control processes, (4) identifying the business needs for each IT project, and (5) developing a basic process for selecting new IT proposals. Table 1 discusses the purpose for each of the stage 2 critical processes.

To its credit, DLA has put in place about 75 percent of the key practices associated with stage 2 critical processes. For example, DLA has oversight boards to perform investment management functions, and it has basic project-level control processes to help ensure that IT projects are meeting cost and schedule expectations.

However, DLA has not executed several crucial stage 2 investment practices. For example, the business needs for IT projects are not always clearly identified and defined, basic investment selection processes are still being developed, and policies and procedures for project oversight are not documented. Table 2 summarizes the status of DLA’s stage 2 critical processes, showing how many associated key practices the agency has executed. DLA’s actions in each of the critical processes are discussed in the sections that follow.

To help ensure executive management accountability for IT capital planning and investment decisions, an organization should establish a governing board or boards responsible for selecting, controlling, and evaluating IT investments. According to ITIM, effective IT investment board operations require, among other things, that (1) board membership have both IT and business knowledge, (2) board members understand the investment board’s policies and procedures and exhibit core competencies in using the agency’s IT investment policies and procedures, (3) the organization’s executives and line managers support and carry out board decisions, (4) the organization create organization-specific process guidance that includes policies and procedures to direct the board’s operations, and (5) the investment board operate according to written policies and procedures. (The full list of key practices is provided in table 3.)

DLA has established several oversight boards that perform IT investment management functions. These boards include the following:

The DLA Investment Council, which is intended to review, evaluate, and approve new IT and non-IT investments between $100,000 and $1,000,000.

The Program Executive Officer Review Board, which is intended to review and approve the implementation of IT investments that are budgeted for over $25 million in all or over $5 million in any one year.

The Corporate Board, which is intended to review, evaluate, and approve all IT and non-IT investments over $1 million.

DLA is executing four of the six key practices needed for these boards to operate effectively. For example, the membership of these boards integrates both IT and business knowledge. In addition, board members informed us of their understanding of their board’s informal practices. Further, according to IT investment officials, project managers, and agency documentation, the boards have a process for ensuring that their decisions are supported and carried out by organization executives and line managers. This process involves documenting board decisions in meeting minutes, assigning staff to carry out the decisions, and tracking the actions taken on a regular basis until the issues are addressed.

Nonetheless, DLA is missing the key ingredient associated with two of the board oversight practices that are needed to operate effectively— organization-specific guidance. This guidance, which serves as official operations documentation, should (1) clearly define the roles of key people within its IT investment process, (2) delineate the significant events and decision points within the processes, (3) identify the external and environmental factors that will influence the processes (that is, legal constraints, the behavior of key subordinate agencies and military customers, and the practices of commercial logistics that DLA is trying to emulate as part of DLA 21); and (4) explain how IT investment-related processes will be coordinated with other organizational plans and processes. DLA does not have guidance that sufficiently addresses these issues. Policies and procedures governing operations are in draft for one board and have not been developed for the two other boards. Without this guidance governing the operations of the investment boards, the agency is at risk of performing key investment decisionmaking activities inconsistently. Such guidance would also provide a degree of transparency that is helpful in both communicating and demonstrating how these decisions are made.

Table 3 summarizes the ratings for each key practice and the specific findings supporting the ratings.

An IT project inventory provides information to investment decision- makers to help evaluate the impacts and opportunities created by proposed or continuing investments. This inventory (which can take many forms) should, at a minimum, identify the organization’s IT projects (including new and existing systems) and a defined set of relevant investment management information about them (for example, purpose, owner, lifecycle stage, budget cost, physical location, and interfaces with other systems). Information from the IT project inventory can, for example, help identify systems across the organization that provide similar functions and help avoid the commitment of additional funds for redundant systems and processes. It can also help determine more precise development and enhancement costs by informing decisionmakers and other managers of interdependencies among systems and how potential changes in one system can affect the performance of other systems.

According to ITIM, effectively managing an IT project inventory requires, among other things, (1) identifying IT projects, collecting relevant information about them, and capturing this information in a repository, (2) assigning responsibility for managing the IT project inventory process to ensure that the inventory meets the needs of the investment management process, (3) developing written policies and procedures for maintaining the IT project inventory, (4) making information from the inventory available to staff and managers throughout the organization so they can use it, for example, to build business cases and to support project selection and control activities, and (5) maintaining the IT project inventory and its information records to contribute to future investment selections and assessments. (The full list of key practices is provided in table 4.)

DLA has executed many of the key practices in this critical process. For example, according to DLA’s CIO, IT projects are identified and specific information about them is entered into a central repository called the DLA Profile System (DPS). DPS includes, among other things, project descriptions, key contact information, lifecycle stage, and system interfaces. In addition, the CIO is responsible for managing the IT project identification process to ensure that DPS meets the needs of the investment management process. However, DLA has not defined written policies and procedures for how and when users should add to or update information in the DPS. In addition, DLA is not maintaining DPS records, which would be useful during future project selections and investment evaluations, and for documenting the evolution of a project’s development. Without appropriate policies and procedures in place to describe the objectives and information requirements of the inventory, DPS is not being maximized as an effective tool to assist in the fundamental analysis essential to effective decisionmaking.

Table 4 summarizes the ratings for each key practice and the specific findings supporting the ratings.

Investment review boards should effectively oversee IT projects throughout all lifecycle phases (concept, design, development, testing, implementation, and operations/maintenance). At stage 2 maturity, investment review boards should review each project’s progress toward predefined cost and schedule expectations, using established criteria and performance measures, and should take corrective actions to address cost and milestone variances.

According to ITIM, effective project oversight requires, among other things, (1) having written polices and procedures for project management, (2) developing and maintaining an approved management plan for each IT project, (3) having written policies and procedures for oversight of IT projects, (4) making up-to-date cost and schedule data for each project available to the oversight boards, (5) reviewing each project’s performance by regularly comparing actual cost and schedule data to expectations, (6) ensuring that corrective actions for each under- performing project are documented, agreed to, implemented, and tracked until the desired outcome is achieved, and (7) using information from the IT project inventory. (The complete list of key practices is provided in table 5.)

DLA has executed most of the key practices in this area. In particular, DLA relies on the guidance in the Department of Defense 5000 series directives for project management and draft guidance in an Automated Information System (AIS) Emerging Program Life-Cycle Management (LCM) Review and Milestone Approval Directive for specific IT project management. In addition, for each of the four projects we reviewed, a project management plan had been approved, and cost and schedule controls were addressed during project review meetings. Further, based on our review of project documentation and in discussion with project managers, up-to-date cost and schedule project data were provided to the PEO Review Board. This board oversees project performance regularly by comparing actual cost and schedule data to expectations and has a process for ensuring that, for underperforming projects, corrective actions are documented, agreed to, and tracked.

Notwithstanding these strengths, DLA has some weaknesses in project oversight. Specifically, although the Corporate Board and the Investment Council have written charters, there are no written policies or procedures that define their role in collectively overseeing IT projects. Without these policies and procedures, project oversight may be inconsistently applied, leading to the risk that performance problems, such as cost overruns and schedule slippages, may not be identified and resolved in a timely manner.

In addition, according to representatives from the oversight boards, they do not use information from the IT project inventory to oversee projects because they are more comfortable using more traditional methods of obtaining and using information (that is, informally talking with subject matter experts and relying on experience). The inventory is of value only to the extent that decisionmakers use it. As discussed earlier, while the inventory need not be the only source of information, it should nevertheless serve as a reliable and consistent tool for understanding project and overall portfolio decisions.

Table 5 summarizes the ratings for each key practice and the specific findings supporting the ratings.

Defining business needs for each IT project helps ensure that projects support the organization’s mission goals and meets users’ needs. This critical process creates the link between the organization’s business objectives and its IT management strategy. According to ITIM, effectively identifying business needs requires, among other things, (1) defining the organization’s business needs or stated mission goals, (2) identifying users for each project who will participate in the project’s development and implementation, (3) training IT staff adequately in identifying business needs, and (4) defining business needs for each project. (The complete list of key practices is provided in table 6.)

DLA has executed all but one of the key practices associated with effectively defining business needs for IT projects. For example, DLA’s mission goals are described in DLA’s strategic plan. In addition, according to IT investment management officials, the IT staff is adequately trained in identifying business needs because they generally have prior functional unit experience. In addition, according to DLA directives, IT projects are assigned an Integrated Process Team (IPT) to guide and direct the project through the development lifecycle. The IPTs are composed of IT and functional staff. Moreover, DOD and DLA directives require that business requirements and system users be identified and that users participate in the lifecycle management of the project. According to an IT investment official, each IT project has a users’ group that meets throughout the lifecycle to discuss problems and potential changes related to the system. We verified that this was the case for the four projects we reviewed.

While the business needs for three of the four projects we reviewed were clearly identified and defined, DLA has reported that this has not been consistently done for all IT projects. According to IT investment management officials, this inconsistency arose because policies and procedures for developing business needs were not always followed or required.

DLA officials have stated that they are developing new guidance to address this problem. However, until this guidance is implemented and enforced, DLA cannot effectively demonstrate that priority mission and business improvement needs are forming the basis for all its IT investment decisions.

Table 6 summarizes the ratings for each key practice and the specific findings supporting the ratings.

Selecting new IT proposals requires an established and structured process to ensure informed decisionmaking and infuse management accountability. According to ITIM, this critical process requires, among other things, (1) making funding decisions for new IT proposals according to an established process, (2) providing adequate resources for proposal selection activities, (3) using an established proposal selection process, (4) analyzing and ranking new IT proposals according to established selection criteria, including cost and schedule criteria, and (5) designating an official to manage the proposal selection process. (The complete list of key practices is provided in table 7.)

DLA has executed some of the key practices for investment proposal selection. For example, DLA executives make funding decisions for IT investments using DOD’s Program Objective Memorandum (POM) process, which is part of DOD’s annual budgeting process. Through this process, proposals for new projects or enhancements to ongoing projects are evaluated by DLA’s IT and financial groups and submitted to OSD through DLA’s Corporate Board with recommendations for funding approval. In addition, according to the CIO, adequate resources have been provided to carry out activities related to the POM process.

Nonetheless, DLA has yet to execute some of the critical practices related to this process area. Specifically, DLA acknowledges that the agency is not analyzing and prioritizing new IT proposals according to established selection criteria. Instead, the Corporate Board uses the expertise from the IT organization and its own judgment to analyze and prioritize projects. To its credit, DLA recognizes that it cannot continue to rely solely on the POM process to make sound IT investment selection decisions. Therefore, the agency has been working to establish an IT selection process over the past two budget cycles that is more investment-focused and includes increased involvement from IT Operations staff, necessary information, and established selection criteria.

Until DLA implements an effective IT investment selection process that is well established and understood throughout the agency, executives cannot be adequately assured that they are consistently and objectively selecting proposals that best meet the needs and priorities of the agency.

Table 7 summarizes the ratings for each key practice and the specific findings supporting the ratings.

IT Investments Are Not Being Managed as a Complete Portfolio

An IT investment portfolio is an integrated, enterprisewide collection of investments that are assessed and managed collectively based on common criteria. Managing investments within the context of such a portfolio is a conscious, continuous, and proactive approach to expending limited resources on an organization’s competing initiatives in light of the relative benefits expected from these investments. Taking an enterprisewide perspective enables an organization to consider its investments comprehensively so that the collective investments optimally address its mission, strategic goals, and objectives. This portfolio approach also allows an organization to determine priorities and make decisions about which projects to fund based on analyses of the relative organizational value and risks of all projects, including projects that are proposed, under development, and in operation.

According to ITIM, stage 3 maturity includes (1) defining portfolio selection criteria, (2) engaging in project-level investment analysis, (3) developing a complete portfolio based on the investment analysis, (4) maintaining oversight over the investment performance of the portfolio, and (5) aligning the authority of IT investment boards.

Table 8 describes the purposes for the critical processes in stage 3.

According to DLA officials, they are currently focusing on implementing stage 2 processes and have not implemented any of the critical processes in stage 3. Until the agency fully implements both stage 2 and 3 processes, it cannot consider investments in a comprehensive manner and determine whether it has the appropriate mix of IT investments to best meet its mission needs and priorities.

DLA Lacks a Plan to Guide Improvement Efforts

DLA recognizes the need to improve its IT investment processes, but it has not yet developed a plan for systematically correcting weaknesses. To properly focus and target IT investment process improvements, an organization should fully identify and assess current process strengths and weaknesses (that is, create an investment management capability baseline) as the first step in developing and implementing an improvement plan. As we have previously reported, this plan should, at a minimum, (1) specify measurable goals, objectives, milestones, and needed resources, and (2) clearly assign responsibility and accountability for accomplishing well-defined tasks. The plan should also be documented and approved by agency leadership. In implementing the plan, it is important that DLA measure and report progress against planned commitments, and that appropriate corrective action be taken to address deviations.

DLA does not have such a plan. In March 2001, it attempted to baseline agency IT operations by reviewing its project-level investment management practices using ITIM. This effort identified practice strengths and weaknesses, but DLA considered the assessment to be preliminary (to be followed by a more comprehensive assessment at an unspecified later date) and limited in scope. DLA used the assessment results to establish broad milestones for strengthening its investment management process. The agency did not, however, develop a complete process improvement plan. For example, it did not (1) specify required resources to accomplish the various tasks, (2) clearly assign responsibility and accountability for accomplishing the tasks, (3) obtain support from senior level officials, and (4) establish performance measures to evaluate the effectiveness of the completed tasks. At the same time, the agency has separately begun other initiatives to improve its investment management processes, but these initiatives are not aligned with the established milestones or with each other. The DLA CIO characterizes the agency’s approach to its various process improvement efforts as a necessary progression that includes some inevitable “trial and error” as it moves toward a complete process improvement plan.

Without such a plan that allows the agency to systematically prioritize, sequence, and evaluate improvement efforts, DLA jeopardizes its ability to establish a mature investment process that includes selection and control capabilities that result in greater certainty about future IT investment outcomes.

Conclusions

Until recently, IT investment management has not been an area of DLA management attention and focus. As a result, DLA currently finds itself without some of the capabilities that it needs to ensure that its mix of IT investments best meets the agency’s mission and business priorities. To its credit, DLA now recognizes the need to strengthen its IT investment management and has taken positive steps to begin doing so. However, several critical IT investment management capabilities need to be enhanced before DLA can have reasonable assurance that it is maximizing the value of its IT investment dollar and minimizing the associated risks. Moreover, DLA does not yet have a process improvement plan that is endorsed and supported by agency leadership. The absence of such a plan limits DLA’s prospects for introducing the management capabilities necessary for making prudent decisions that maximize the benefits and minimize the risks of its IT investment.

Recommendations

To strengthen DLA’s investment management capability and address the weaknesses discussed in this report, we recommend that the secretary of defense direct the DLA director to designate the development and implementation of effective IT investment management processes as an agencywide priority.

Further, we recommend that the secretary of defense have the DLA director do the following:

Develop a plan, within 6 months, for implementing IT investment management process improvements that is based on GAO’s ITIM stage 2 and 3 critical processes.

Ensure that the plan specifies measurable goals and time frames, defines a management structure for directing and controlling the improvements, and establishes review milestones.

Ensure that the plan focuses first on correcting the weakness in the ITIM stage 2 critical processes, because these processes collectively provide the foundation for building a mature IT investment management process. Specifically:

Develop and issue guidance covering the scope and operations of DLA’s investment review boards. Such guidance should include, at a minimum, specific definitions of the roles and responsibilities within the IT investment process; an outline of the significant events and decision points within the processes; an identification of the external and environmental factors that will influence the processes (for example, legal constraints, the behavior of key suppliers or customers, or industry norms), and the manner in which IT investment-related processes will be coordinated with other organization plans and processes.

Develop and issue policies and procedures for maintaining DLA’s IT projects inventory for investment management purposes.

Finalize and issue policies and procedures (including the use of information from the IT systems and project inventory) for the PEO Review Board’s oversight of IT projects.

Develop and issue similar policies and procedures for the other investment boards.

Finalize and issue guidance supporting the identification of business needs and implementing management controls to ensure that proposals submitted to DLA for review clearly identify and define business requirements.

Develop and issue guidance for the proposal selection process in such a way that the criteria for selection are clearly set forth, including formally assigning responsibility for managing the proposal selection process and establishing management controls to ensure that the proposal selection process is working effectively.

Ensure that the plan next focuses on stage 3 critical processes, which are necessary for portfolio management, because along with the stage 2 foundational processes, these processes are necessary for effective management of IT investments. Implement the approved plan and report on progress made against the plan’s goals and time frames to the secretary of defense every 6 months.

Agency Comments and Our Evaluation

DOD provided what it termed “official oral comments” from the director for acquisition resources and analysis on a draft of this report. In its comments, DOD concurred with our recommendations and described efforts under way and planned to implement them. However, it recommended that two report captions be changed to more accurately reflect, in DOD’s view, the contents of the report and to eliminate false impressions.

Specifically, DOD recommended that we change one caption from “DLA’s Capabilities to Effectively Manage IT Investments Are Limited” to “DLA’s Capabilities to Effectively Manage IT Investments Should Be Improved.”

DOD stated that this change is needed to recognize the fact that DLA has completed about 75 percent of the practices associated with stage 2 critical processes. We do not agree. As stated in our report, to effectively manage IT investments an agency should (1) have basic, project-level control and selection practices in place (stage 2 processes) and (2) manage its projects as a portfolio of investments (stage 3 processes). Although DLA has executed most of the key practices associated with stage 2 processes, the agency acknowledges that it has not implemented any of the stage 3 processes. Therefore, our caption as written describes DLA’s IT investment management capabilities appropriately.

In addition, DOD recommended that we change the caption “DLA Lacks a Plan to Guide Improvement Efforts” to “DLA Lacks a Published Plan to Guide Improvement Efforts.” DOD stated that this change is needed because DLA has developed some elements of an implementation plan. We do not agree. Our point is that DLA did not have a complete process improvement plan, not that it has yet to publish the plan that it has. As we describe in the report, a complete plan should, at a minimum, (1) be based on a full assessment of process strengths and weaknesses, (2) specify measurable goals, objectives, milestones, and needed resources, (3) clearly assign responsibility and accountability for accomplishing well- defined tasks, and (4) be documented and approved by agency leadership.

In contrast, DLA’s planning document was based on a preliminary assessment of only stage 2 critical processes and lacked several of the critical attributes listed above. Moreover, DOD stated in its comments that DLA has not completed a formally documented and prioritized implementation plan to resolve stage 2 and 3 practice weaknesses and has yet to complete the self-assessment and gap analysis necessary to define planned action items. Accordingly, it is clear that DLA has not satisfied the tenets of a complete plan, and thus our caption is accurate as written.

DOD provided additional comments that we have incorporated as appropriate in the report.

We are sending copies of this report to the chairmen and ranking minority members of the Subcommittee on Defense, Senate Committee on Appropriations; the Subcommittee on Readiness and Management Support, Senate Committee on Armed Services; the Subcommittee on Defense, House Committee on Appropriations; and the Subcommittee on Military Readiness, House Committee on Armed Services. We are also sending copies to the director, Office of Management and Budget; the secretary of defense; the under secretary of defense for acquisition, technology, and logistics; the deputy under secretary of defense for logistics and materiel readiness; and the director, Defense Logistics Agency. Copies will be made available to others upon request.

If you have any questions regarding this report, please contact us at (202) 512-3439 and (202) 512-7351, respectively, or by e-mail at hiter@gao.gov and mcclured@gao.gov. An additional GAO contact and staff acknowledgments are listed in appendix II.

Appendix I: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the individual named above, key contributors to this report were Barbara Collier, Lester Diamond, Gregory Donnellon, Sabine Paul, and Eric Trout.