DLA is the Department of Defense’s (DOD) logistics manager for all DOD consumable items and some department repair items. Its primary business function is to provide supply support in order to sustain military operations and readiness. In addition to this primary function, which DLA refers to as either “materiel management” or “supply-chain management,” DLA performs five other major business functions: distributing materiel ordered from its inventory; purchasing fuels for DOD and the U.S. government; storing strategic materiel; marketing surplus DOD materiel for reuse and disposal; and providing numerous information services, such as item cataloging, for DOD, the United States, and selected foreign governments. DLA consists of a central command authority supported by a number of field commands that manage the agency’s six business functions. Until about 1997, DLA generally developed its systems in-house. Since then, the agency has begun to acquire systems, relying on contractors for system development and managing the acquisition of these systems. Currently, DLA is in the process of acquiring two systems: Business Systems Modernization (BSM) and Fuels Automated System (FAS). BSM is intended to modernize DLA’s materiel management business function, changing the agency from being solely a provider and manager of physical inventory to being a manager of supply chains. In this role, DLA would link customers with appropriate suppliers and track physical and financial business practices. It is planning to replace two large legacy systems, as well as several supporting programs, that are more than 30 years old and are not integrated. BSM is based on commercially available software products. DLA plans to acquire and deploy its BSM system solution through a series of four system releases/increments. First, it plans to demonstrate successful application of its new concept of doing business for selected commodities—namely, earth-moving equipment, medical/pharmaceutical supplies, and F/A-18 engine components—at the three Defense Supply Centers. If this first release is successfully demonstrated, DLA plans to expand the system solution to other commodities in three additional increments. DLA plans to invest approximately $658 million to acquire and implement BSM from fiscal years 2000 through 2005. FAS is intended to help the Defense Energy Support Center manage about $5 billion in contracts with petroleum suppliers each year. FAS is to be a multifunctional system that provides for, among other things, point-of-sale data collection inventory control, finance and accounting, procurement, and facilities management. FAS, which relies on a commercially available software package, is being fielded incrementally. Increment 1 is the base- level operational module that is currently being deployed to base-level sites worldwide. The second increment is the enterprise-level system, which is to be deployed to its direct delivery commodity business unit. DLA plans to invest $293 million in FAS from fiscal year 1995 through 2002. SEI’s SA-CMM is used to measure an organization’s capability to manage the acquisition of software. SEI’s expertise in, and model and methods for, determining software process assessment are recognized and accepted throughout the software industry. The model defines five levels of software acquisition maturity. Each level of maturity (except level 1) indicates process capability in relation to key process areas. For a maturity level to be achieved, all key process areas related to that level must be implemented effectively. The second level of process maturity, level 2 (referred to as the repeatable level), demonstrates that basic management processes are established to track performance, cost, and schedule, and the necessary discipline is in place to repeat earlier successes on similar projects. Organizations that do not effectively implement all key process areas for the repeatable level are, by default, at level 1, the initial level of maturity. Level-1 processes can be described as immature, ad hoc, and sometimes chaotic; success in software acquisition for these organizations depends on the ability and commitment of the staff involved. Figure 1 further explains the five-level software acquisition model. We evaluated DLA against six of the seven level-2 (repeatable) key process areas in the SA-CMM. We did not evaluate DLA on the seventh key process area—transition to support—because the contractors who are implementing BSM and FAS will support these systems when they are operational, rendering transition to support irrelevant for these acquisitions. We evaluated DLA against one level-3 (defined) key process area—acquisition risk management—because many software acquisition experts consider it to be one of the most important key process areas. These key process areas are described in table 1. As established by the model, each key process area contains five common features—commitment to perform, ability to perform, activities to be performed, measurement and analysis of activities, and verification of activities’ implementation. These five features collectively provide a framework for the implementation and institutionalization of the key process areas. The common feature definitions are as follows: Commitment to perform: This feature describes the actions that the organization takes to establish the process and ensure that it can endure. Key practices typically involve establishing organizational policies and sponsorship. Ability to perform: This feature describes the preconditions that must exist in the project or organization to implement the software acquisition process competently. Key practices typically include assigning responsibility and providing training. Activities to be performed: This feature describes the roles and procedures necessary to implement a key process area. Key practices typically involve establishing plans and procedures, performing the work, tracking it, and taking appropriate management actions. Measurement and analysis of activities: This feature describes the steps necessary to measure progress and analyze the measurements. Key practices typically involve defining the measurements to be taken and the analyses to be conducted to determine the status and effectiveness of the activities performed. Verification of activities’ implementation: This feature describes the steps the organization must take to ensure that project activities are performed in accordance with established processes. Key practices typically involve regular reviews by management. Each common feature consists of a number of key practices—specific actions such as developing an organizational policy for software acquisition, developing various plans for software acquisition activities, and tracking a contractor’s progress. When an organization is evaluated against the SA-CMM, comparisons of actual performance against a key practice can result in one of four possible outcomes or ratings: Strength: The key practice involved was effectively implemented. Weakness: The key practice was not effectively implemented or was not implemented. Observation: The key practice was evaluated, but cannot be characterized as a strength because (1) the project team did not provide sufficient evidence to support a strength rating or (2) the key practice was only partially performed. Not rated: The key practice is not relevant to the project. To achieve the repeatable level, DLA would have to demonstrate that the key practices related to this level were implemented effectively in the software acquisition projects being evaluated, and thus the project successes can be repeated in future projects. DLA is not at level 2 (the repeatable level of maturity) when compared with the SA-CMM—meaning that DLA does not possess an agencywide or corporate ability to effectively acquire software-intensive systems. Whereas DLA’s BSM project fully or substantially satisfied SEI’s SA-CMM requirements for the key process areas for level 2, as well as requirements for one level 3 (defined level) key process area, its FAS project did not satisfy all the criteria for any of these key process areas. A discussion of how each system compared with the SA-CMM is summarized below. BSM completely satisfied requirements for three of the level-2 key process areas, as well as for the one level-3 key process area, and substantially satisfied requirements for the remaining three level-2 key process areas that we evaluated. (See table 2 for the percentage of strengths and weakness for each area evaluated.) According to BSM officials, satisfying the criteria for the key process areas is attributable to the following factors: allocating adequate resources; following good program management practices, as defined in DOD Directive 5000; and working closely with relevant oversight groups. To address those few weaknesses that we identified, project officials told us that they have initiated corrective action. BSM satisfied all key practices in software acquisition planning, such as (1) having a written software acquisition policy, (2) having adequate resources for software acquisition planning activities, (3) developing and documenting the software acquisition strategy and plan, and (4) making and using measurements to determine the status of software acquisition planning activities. project management, including (1) designating responsibility for project management, (2) having a written policy for the management of the software project, (3) having adequate resources for the duration of the software acquisition project, and (4) tracking the risks associated with cost, schedule, resources, and the technical aspects of the project. contract tracking and oversight, including (1) designating responsibility for contract tracking and oversight, (2) including contract specialists in the project team, and (3) having a documented plan for contract tracking and oversight. acquisition risk management, such as (1) having a risk management plan, (2) having a written policy for the management of software acquisition risk, and (3) measuring and reporting on the status of acquisition risk management activities to management. BSM also satisfied all but one key practice in solicitation. Strengths included (1) designating responsibility for the software portion of the solicitation, (2) preparing cost and schedule estimates for the software products and services being acquired, and (3) having an independent review of cost and schedule estimates for the software products and services being acquired. BSM’s one weakness in this key process area was in not having a written policy for the software portion of the solicitation. This is significant because, according to the SEI, an institutional policy provides for establishing an enduring process. BSM also satisfied all but three key practices in requirements development and management.