Defense Logistics: Improvements Needed to Enhance DOD's Management Approach and Implementation of Item Unique Identification Technology
Released: 2012-05-03T00:00:00

Letter

Background

Criteria and Process for Using IUID Technology

DOD has established criteria for the types of items that are required to be marked with IUID labels.

For principal end items and contractor-marked secondary items, the criteria are as follows: 1. all items for which the government’s unit acquisition cost is $5,000 or more; 2. items for which the government’s unit acquisition cost is less than $5,000, when identified by the requiring activity as DOD serially managed, mission-essential or controlled-inventory; 3. when the government’s unit acquisition cost is less than $5,000 and the requiring activity determines that permanent identification is required; and 4. regardless of value, (a) any DOD serially managed subassembly, component, or part embedded within an item and (b) the parent item that contains the embedded subassembly, component, or part.

For secondary items in use or in inventory, the criteria are as follows: 1. all DOD serially managed items including, but not limited to: sensitive, critical safety, or pilferable items that have a unique item-level traceability requirement at any point in their life cycle; and all depot-level reparable items; and 2. any other item that the requiring activity decides requires unique item- level traceability at any point in their life cycle.

In order to use IUID technology, four processes must be completed. First, each item that qualifies for IUID marking—according to DOD’s criteria—is labeled. IUID labels often contain some human-readable information, printed as text on the label. The amount and type of human-readable information varies, but it often contains key details about the item, such as its National Stock Number, part number, or serial number. These are categories that DOD components use to identify items in their inventories. In certain cases, items do not have labels attached to them, and instead are labeled through a process known as direct part marking, in which the human- and machine-readable information are applied directly to the item. In addition, each individual label contains information about the item, encoded in a machine-readable, two-dimensional image printed on the label. Known as a data matrix, this image contains various pieces of information encoded in a two-dimensional bar code. Figure 1 shows a data plate with an IUID data matrix in the lower right-hand corner.

When combined, the pieces of information encoded in the data matrix make up a globally unique string of numbers referred to as the item’s UII number. To ensure that all items marked with IUID labels are globally unique, DOD requires that UII numbers be formatted according to international standards for syntax format. Further, UII numbers must be entered into DOD’s IUID Registry. The registry is a database intended to ensure that each UII number is unique. An item’s UII number may be entered into the registry in one of two ways: the item is marked with an IUID label and the UII number associated with that label is registered, or DOD or contractors can establish a “virtual” UII number. According to DOD guidance, these virtual UII numbers are assigned to an item that has not yet been marked with an IUID label. The guidance states that a virtual UII number may be used due to economic or logistical considerations. For example, a DOD component may virtually mark one item that is embedded in another item, so that DOD does not have to remove the embedded item solely to mark the embedded item. For legacy items already in a component’s inventory, marking and registration are the responsibility of the component. For items that are not yet in DOD’s inventory and are being delivered to a component—or for property furnished by the government to a contractor—it is the responsibility of the contractor to mark or register items. If the UII number in a data matrix is improperly formatted, it cannot be used to properly identify the item, and the data matrix must be replaced.

Second, personnel must electronically read the label’s data matrix, the two-dimensional bar code in which the item’s UII number is encoded.

There are several types of tools than can be used for this process, including hand-held scanners and web-based software that can read an image of a data matrix. Because data matrices cannot be read visually, electronically reading the matrices is the only way to access the UII data they contain. Figure 2 shows an electronic scanner being used to read a data plate with an IUID data matrix.

Third, UII data from a data matrix is passed to an IT system. According to DOD officials, there are a variety of IT systems that have a requirement to use UII data, and some of these systems currently have the capability to store UII data. Examples include the Army’s Property Book Unit Supply Enhanced, the Navy’s Configuration Data Managers Database–Open Architecture, the Marine Corps’ Joint Asset Maintenance Integrated Support System, and the Air Force’s Automated Inventory Management Tool. DOD officials have explained that some of these systems operate in “pockets” within the components, and do not share UII data across the components or DOD-wide. For instance, the Air Force’s Automated Inventory Management Tool contains UII data that is specific to an Air Force installation, and does not have the capability to share these data with other installations. DOD’s goal is for the components to share UII data across each of their individual IT systems, and DOD-wide, between components. Within DOD, this type of data sharing is characterized as “enterprisewide.” In order to accomplish enterprisewide data sharing of UII data, the components intend to use certain IT systems referred to as Enterprise Resource Planning systems. These automated systems consist of multiple, integrated functional modules that perform a variety of business-related tasks, such as general-ledger accounting, payroll, and supply chain management. DOD officials have explained that certain Enterprise Resource Planning systems will provide the capability to share UII data enterprisewide, and in this report we are focusing on DOD efforts to integrate IUID with these systems.IT system, such as an Enterprise Resource Planning system, DOD intends to store UII data and share these data within and across DOD organizations. In addition, IT systems can use software to analyze these Once UII data is uploaded into an data to improve logistics processes, such as property accountability and maintenance, in DOD’s supply chain.

Fourth, to achieve IUID technology’s potential benefits in many logistics processes, DOD personnel will have to periodically repeat the previous steps, including scanning the matrix on an item’s label; uploading the UII data into an IT system; and then storing, sharing, and analyzing the data as required by the specific logistics process. For example, in a property- accountability process that we observed, each time a weapon was checked in or out of an armory, personnel scanned the label’s data matrix; the matrix’s UII number was uploaded into an electronic property book; and software then matched the weapon’s UII number with data that identified the weapon’s owner. For many of the logistics processes in which IUID could be used, these steps would be repeated throughout an item’s life cycle.

Key Organizations Involved in IUID Implementation

It is DOD’s goal for its components to share UII data departmentwide, and the components are to use these data for unique item tracking. The Office of the Deputy Under Secretary of Defense for Logistics and Materiel Readiness—under the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics—is the focal point for implementing IUID capabilities for DOD’s supply chain materiel. As of July 2011,DOD-wide IUID implementation to the Office of the Deputy Assistant Secretary of Defense for Supply Chain Integration (ODASD). According to component officials, each component has multiple organizations carrying out IUID implementation tasks, such as creating policy and defining requirements; planning and budgeting for implementation of policy; and executing requirements. In addition, according to component officials, each component has an office that that office delegated the responsibility for programmatic lead of maintains the lead in IUID implementation policy. These respective offices are:

Army—the Office of Life Cycle Logistics Policy, in the Office of the Assistant Secretary of the Army for Acquisition, Logistics and Technology;

Navy—the Office of the Assistant Secretary of the Navy for Research, Development, and Acquisition;

Marine Corps—the Office of the Director, Logistics Plans, Policies, and Strategic Mobility, in the office of the Deputy Commandant of the Marine Corps for Installations and Logistics;

Air Force—the Directorate of Transformation, Deputy Chief of Staff for Logistics, Installations and Mission Support; and

Defense Logistics Agency—the Office of Logistics Management Standards.

Also, to facilitate intercomponent communication and collaboration, the components have established a number of working groups and other bodies to facilitate coordination on IUID implementation.

DOD’s IUID Task Force

The Joint Logistics Board determined that there were ambiguities concerning DOD’s IUID policy, requirements, and proposed value across DOD, as well as wide variation in the components’ implementation strategies, execution, and funding of IUID implementation. As a result, in 2009 the board chartered the task force, led by the Assistant Deputy Under Secretary of Defense for Maintenance Policy and Programs, and including representatives from the components and the Office of the Secretary of Defense. The task force had several goals, to include assessing the value of IUID within DOD’s supply chain, and recommending changes to policy and guidance to adequately align IUID implementation with the task force’s evaluation of IUID’s value. The task force issued a report with recommendations in June 2010 that estimated financial costs of IUID implementation, as well as financial and nonfinancial benefits. Specifically, the report stated that DOD could begin to achieve net financial benefits of IUID implementation in fiscal year 2017. In addition, the task force recommended modifying some of DOD’s IUID-marking criteria. Subsequently, the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics modified DOD’s IUID marking criteria to implement some of the task force’s recommended changes, issuing the modification in December 2010. In addition, the task force issued a revision to its initial estimates, lowering its cost DOD’s key IUID estimate. The revision was issued in March 2011.policy issuances and implementation events from fiscal year 2003 to fiscal year 2011 are summarized in appendix II.

DOD Has Taken Some Steps toward Developing a Framework for Management of IUID Implementation, but Has Not Yet Incorporated Several Key Elements of a Comprehensive Management Approach

DOD has taken some steps to improve its approach to managing and implementing IUID technology, but has yet to incorporate some key elements of best management practices into its evolving framework for management of IUID implementation. These include internal controls and analysis of return on investment. According to GAO’s previously published work, internal controls are important in helping agencies improve operational processes and implement new technological developments. Internal controls include an organizational structure that clearly defines key areas of authority; policies that enforce management directives; goals; and performance measures. In addition, GAO and DOD have identified best practices for analyzing a program’s return on investment. The practices identified by GAO include providing estimates of all potential costs and the timing of costs. DOD has identified best practices that include analyzing benefits, and making recommendations, based on relevant evaluation criteria. DOD has defined key areas of authority and responsibility for IUID implementation, and is updating policy to incorporate changes required by the implementation of IUID. However, DOD has not incorporated other key elements of best management practices into its evolving framework for management of IUID implementation. For example, DOD lacks such key information as quantitatively defined goals for marking legacy items; performance measures, such as reliable schedules for predicting when its Enterprise Resource Planning systems will have the capability to manage items using UII data; and a full estimate of IUID’s cost and benefits. Without a management framework that includes quantitatively defined goals, performance measures, and a more complete estimate of all associated costs and benefits, DOD has faced challenges in implementing IUID technology and runs the risk of not fully realizing its potential benefits, including financial benefits by fiscal year 2017.

DOD Has Taken Several Steps That Could Improve Its Management of IUID Implementation

ODASD(SCI) and the components have taken some steps that could improve DOD’s management approach for IUID implementation. According to the DASD(SCI), his office is in the process of developing a framework for managing and implementing IUID, which we reviewed. As of March 2012, this framework consisted of two elements. The first element is a set of July 2011 briefing slides titled “IUID Game Plan and Actions Underway.” The slides include a summary of actions that DOD needs to take in key areas of IUID implementation such as the marking of items, the use of IUID in business processes, and modifying IT systems to incorporate IUID. These slides also indicate that ODASD(SCI) is following some best practices of a comprehensive management framework. In the slides, DOD clearly defines the Office of the Assistant Secretary of Defense for Logistics and Materiel Readiness as the DOD organization responsible for leading IUID implementation activities. As previously discussed, that office delegated the responsibility to ODASD(SCI). In addition, the slides discuss DOD policies that enforce management directives concerning IUID implementation and that are being updated to incorporate IUID. The second element of DOD’s framework for managing and implementing IUID is a January 2012 timeline listing several planned implementation actions from fiscal years 2012 through 2017.

The components have also taken some steps toward improving their management of IUID implementation. The Army and Marine Corps are using some quantifiable goals—and certain IUID marking criteria—to track the progress of their legacy marking efforts. In addition, officials from some components told us that they had inspected some newly- acquired items to determine whether these items were sufficiently marked with IUID labels. For the items they reviewed, those inspections helped to detect problems in contractors’ marking of items. Further, the Marine Corps and Air Force are planning to integrate IUID elements into implementation schedules for their Enterprise Resource Planning systems.

DOD Has Not Fully Incorporated Certain Internal Controls into Its Framework for Management of IUID Implementation

DOD has not fully incorporated internal controls, such as quantifiable goals or metrics to assess progress, into its framework for management of IUID implementation. An agency’s establishment of internal controls is key in helping an agency meet its goals, and we have previously reported that in the absence of quantifiable targets, it is difficult for officials to assess whether goals were achieved, because comparisons cannot be made between projected performance and actual results. DOD has identified tens of millions of legacy items that meet its IUID marking criteria, but has not developed a full set of quantifiable goals or metrics to assess its progress in marking these items. The task force has stated that DOD will not achieve IUID’s potential benefits unless DOD marks a “significant” number of these legacy items, and an ODASD(SCI) official stated that DOD needs to mark a “majority” of these legacy items by fiscal year 2015. However, the task force and ODASD(SCI) have not quantitatively defined the terms “significant” or “majority,” respectively. Further, according to the task force report, the number of legacy items DOD will mark is an important factor in determining when DOD may begin to realize IUID’s projected financial benefits. Therefore, without metrics to quantify its progress in marking legacy items, it is unclear whether DOD will begin to realize these benefits by fiscal year 2017, the year in which the task force report projects these benefits may begin.

GAO-11-53 and GAO-08-883T. have adequate schedules for the integration of IUID with their Enterprise Resource Planning systems.

We have previously reported that an important element of measuring performance is the collection of data that are complete and consistent enough to document performance and support decision making. As previously discussed, DOD has established IUID marking criteria for different categories of inventory. We found that some components used these criteria to track their progress in marking legacy items; however, others did not. The DASD(SCI) stated that he has asked the components to periodically report on their progress in marking legacy items to his office. However, he stated that he has not asked the components to use DOD’s IUID marking criteria in their reporting.

DOD Has Not Completed a Full Analysis of IUID’s Return on Investment

We have previously reported that another key element of best management practices is analysis of return on investment. A complete analysis on the return on investment consists of several best practices identified in previously published GAO and DOD work. The practices identified by GAO include providing estimates on all potential costs and the timing of costs. DOD has identified best practices for analyzing benefits, and making recommendations, on the basis of relevant evaluation criteria as a best practice. In addition, GAO has reported that performing a sensitivity analysis of these estimates demonstrates the effects on the cost and schedule of an assumption that may no longer be valid. DOD began implementation of IUID in fiscal year 2003, and in fiscal year 2009 the task force estimated some costs and benefits of implementation. The task force report discusses how IUID could improve DOD’s logistical efficiency and effectiveness. In addition, the report provides a “rough order of magnitude” assessment of certain costs and financial benefits of implementation; and projects when DOD may begin to realize financial benefits. However, the components, ODASD(SCI), and the task force did not fully follow certain best practices for the estimation of costs and financial benefits. As a result, DOD does not have a full analysis of IUID’s potential return on investment.

DOD’s components could not provide complete historical and planned spending data for IUID implementation, and ODASD(SCI) has not tracked the components’ spending or budget requests for IUID. According to GAO best practices for cost estimation, historical data on the cost of a system is important for projecting a credible estimate of future costs. Although the Marine Corps and Air Force provided us with complete estimates of the amount of money they have spent in their IUID budgets, the Army and Navy provided incomplete estimates of their IUID spending, and the Defense Logistics Agency did not provide an estimate. These components were not able to provide complete estimates because they do not track IUID spending as a distinct budget category. In addition, ODASD(SCI) does not track the five components’ historical spending on IUID.

Although the components were not able to provide complete historical spending information, according to the information they provided, the components spent at least $219 million on IUID implementation from fiscal year 2004 through fiscal year 2011. Officials explained that they spent this money on a variety of IUID implementation efforts, including the acquisition of marking equipment, such as IUID label printers and scanners; the marking of legacy items; and the development of software to support marking processes. Table 1 summarizes the five components’ reported historical spending on IUID over 8 fiscal years.

GAO-09-3SP.

The components’ total reported spending in fiscal year 2011 dollars is $226.1 million.

The Army provided an estimate of the money spent on IUID implementation in its depots over this period but, as an official explained, could not provide an estimate of IUID spending outside of its depots because it does not track IUID funding as a distinct budget category. The Navy was also able to report on a portion of its IUID spending that was executed by one office within the Navy, but Navy officials stated that because the Navy has not funded IUID implementation in a centralized fashion, the Navy cannot track how much other Navy offices have spent in implementing IUID. An official from the Defense Logistics Agency explained that while the agency does spend money on IUID implementation, it does not have a distinct budget for IUID implementation, and so cannot specifically track its IUID costs.

With regard to future spending, the Marine Corps and the Air Force reported that they requested a total of $19.2 million (Marine Corps: $10.8 million; Air Force: $8.4 million) for IUID implementation in fiscal year 2012. However, the Army and Navy were unable to provide their budget requests for fiscal year 2012 because they do not budget for IUID spending as a distinct budget category. According to officials from the Defense Logistics Agency, an office within the agency submitted a request for IUID implementation spending, but this request was not included in the agency’s final fiscal year 2012 budget request.

In addition to the components not fully tracking their spending on IUID implementation, ODASD(SCI) does not track the components’ spending on IUID implementation. ODASD(SCI) explained that the office uses a set of “scorecards” to track the components’ IUID implementation efforts. In 2011, ODASD(SCI) received two sets of scorecards, the first in January and the second in November. In our review of these scorecards, we found that the components do not report information on either the amount of money they have spent on IUID implementation, or the amount of money that they plan to request for IUID implementation.

In its report, the task force estimates that DOD would need to invest $3.2 billion to realize the benefits of IUID implementation. This estimate includes several types of costs, such as the cost of marking legacy items and the labor required to perform various implementation tasks. However, the estimate does not include the following four costs:

The task force could not completely estimate the costs associated with how individual logistics processes would need to change to incorporate the use of IUID technology, because it did not have sufficient information about these changes. As a relatively new technology, IUID is not widely used in existing DOD logistics processes. That means that the components will have to modify existing processes to use IUID. DOD has made some progress in defining the type of logistics processes that would need to change to incorporate IUID. For example, according to ODASD(SCI), DOD intends to modify 10 different categories of logistics processes, such as receipt and distribution of items. DOD has made some progress in planning for such modifications. For example, the Army Materiel Command has determined that 11 of its logistics processes will incorporate IUID functionality through one of the Army’s Enterprise Resource Planning systems. However, according to Army officials, the Army has not yet defined the distinct steps at which personnel will use IUID, within its logistics processes. Further, according to DOD officials, the components have not determined the full number or type of business processes within these categories that need to be modified. In addition, in their 2011 scorecard updates, the Army, Marine Corps, and Air Force reported challenges or concerns with progress in defining how their business processes will require modification because of IUID. Without information on the number of logistics processes that will require modification and the specific steps that will need to change to accommodate the incorporation of IUID, the task force could not have completely estimated costs. Several DOD officials we spoke with agreed with this assessment, explaining that without this information, the task force was not able to completely estimate IUID implementation costs.

The task force report states that the task force did not include the cost of modifying Enterprise Resource Planning systems to share and use UII data because the functionality to use UII data is inherent in these systems. However, if DOD is to achieve benefits from IUID, Enterprise Resource Planning systems must have the ability to share and analyze UII data, and according to DOD officials the ability to perform these two functions is not always inherent in these systems. For example, the core software package for one of the Army’s Enterprise Resource Planning systems has the capability to accept UII data, but in its current version this capability is not activated. For the system to accept UII data, the capability must be activated in a future update. According to DOD officials, such modifications to Enterprise Resource Planning systems can incur costs. Navy officials explained that the task force report does not include the additional cost that would be required to modify the Navy’s systems to communicate with other components’ systems, or to analyze UII data to improve logistics processes. By not including certain costs to modify the relevant Enterprise Resource Planning systems, the task force may have left out a substantial set of costs.

The report did not include the full cost of marking newly-acquired items with IUID. Because the Defense Federal Acquisition Regulation Supplement contract clauses provide that contractors are responsible for marking or registering items, contractors are likely to build the cost of marking items into their contract pricing. Therefore, while contractor personnel are marking the items in question, this cost is borne by DOD to the extent that the contractor has built the cost of marking qualifying items into the costs of the goods or services provided. According to component officials, the components do not know how much of a newly-acquired item’s cost is attributable to contractors’ marking of that item. However, according to estimates in the task force report and provided by the Air Force, contractors’ average cost to mark a newly-acquired item is about $30 to $50. On the basis of GAO analysis of information from the DOD registry on the number and rate of contractors’ registration of newly-acquired items in January 2012, IUID marking by contractors could result in approximately $27 million to $45 million in marking costs to the components per year. This cost is not included in the task force’s estimate.

As previously discussed, an IUID label on an item may contain human-readable information, such as the item’s National Stock Number. In addition, the label has a data matrix that contains information about the labeled item. For example, the matrix contains various pieces of information that make up the globally unique string of numbers referred to as the item’s UII number. Contractors have delivered items to DOD that have labels with deficient data matrices. If a label has a deficient data matrix, DOD cannot use the label’s data matrix to track or manage items. The task force did not include the cost to the components of fixing these deficient IUID data matrices on contractor-marked items. Although component officials were not able to estimate the financial cost of fixing these data matrices, officials explained that there is productivity loss associated with fixing them. For example, according to Army and Marine Corps officials, their verification of a deficient matrix’s missing or incorrect data requires an average of 10 to 15 minutes, and sometimes requires significantly more time, when additional research into the item is needed. This investment of time, multiplied by many thousands of deficient data matrices, may result in a substantial amount of lost productivity for DOD components’ personnel.

The task force report assessed the potential benefits of IUID implementation by examining three categories of logistics processes: intensive item management, property accountability, and product life cycle management. According to the task force’s analysis, DOD’s implementation of IUID is unlikely to result in substantial financial benefit in the categories of intensive item management or property accountability. However, the report did discuss potential nonfinancial benefits of IUID implementation in these two categories. The report states that the use of IUID in intensive item management could enable strict accountability and control of DOD’s most critical assets—such as nuclear weapon–related material—across parts of DOD’s supply chain, enhancing the security and safety of such assets. Moreover, according to the report, implementing IUID into property-accountability processes on the enterprise level could enable DOD to track equipment assets throughout their life cycle. The report explains that one benefit of tracking on the enterprise level is that DOD may be able to more quickly address equipment losses. Further, according to an official from the Office of Defense Procurement and Acquisition Policy, the use of IUID in DOD’s logistics processes could lead to improved data quality that may result from automatically entering data into IT systems, as opposed to manually entering data.

While the report does not estimate substantial savings through the integration of IUID with intensive item management or property- accountability processes, the report estimates that IUID implementation could result in annual savings of $3 billion to $5 billion through the implementation of IUID in a collection of maintenance processes referred to as product life cycle management. According to our review of the report and a task force official, the key to achieving savings through product life cycle management is to track and manage individual items by a unique identifier. That approach is called serialized item management. According to DOD officials and the task force report, DOD can achieve serialized item management by using any type of unique identifier, including a traditional serial number; a UII number; or a unique identifier provided by a different type of technology, such as a radio frequency identification device or a contact memory button.

The task force estimated achieving substantial financial benefits from product life cycle management, but it used a methodology for estimating these benefits that may not be appropriate to the scale and complexity of DOD’s IUID implementation efforts. The IUID task force report states that projected savings will gradually increase as implementation of IUID spreads throughout DOD and, by fiscal years 2016 to 2017, DOD may reach a break-even point at which its annual financial savings would equal its annual spending for implementation of IUID. After fiscal year 2017, the report projects that DOD may pass the break-even point, and could begin to realize the annual savings of $3 billion to $5 billion.

To develop its estimate of cost savings through the use of serialized item management in product life cycle management, the task force used the following methodology:

Reviewed case studies of five DOD maintenance programs that use serialized item management. The task force observed that by using the maintenance programs reduced serialized item management,costs by an average of 4 to 6 percent in labor and materiel costs for maintenance, and in the cost to transport items to maintenance locations.

Next, the task force estimated maintenance costs by adding DOD’s fiscal year 2008 budget for depot- and field-level maintenance to DOD’s fiscal year 2009 budget for maintenance transportation, which together total about $83.2 billion.

Finally, by applying the 4 to 6 percent reduction to $83.2 billion in annual maintenance costs, the task force estimated that an annual savings of $3 billion to $5 billion could result from the use of serialized item management in product life cycle management maintenance processes.

However, three aspects of this methodology call into question whether it is reasonable to assume that DOD-wide use of IUID technology in maintenance processes would lead to the savings estimated by the task force.

The task force estimated DOD-wide savings on the basis of a limited number of case studies, and conclusions developed from these studies may not be applicable to the substantial complexity and size of the DOD-wide maintenance enterprise. According to the Office of the Deputy Assistant Secretary of Defense for Maintenance Policy and Programs, DOD’s maintenance operations support a wide range of weapon systems including about 280 ships, 14,000 aircraft and helicopters, 900 strategic missiles, and 30,000 combat vehicles. Our review of the Task Force report indicates the five case studies address—in total—five individual weapon systems, whereas DOD performs maintenance on hundreds of different systems. Moreover, four of the five case studies address either Air Force or Navy programs; only one addressed Army programs. Because of the limited scope of the case studies used by the task force, conclusions based on these case studies may not apply to the DOD-wide maintenance budgets that the task force used in its estimation of savings.

The case studies did not address programs that use IUID as the technology that provides a unique identifier to track items through serialized item management. Rather, the case studies addressed programs that use other means of uniquely tracking items, such as contact memory buttons. Thus, the case studies do not consider costs that may be specific to IUID technology, such as the cost to purchase scanners or software to read data matrices, or the cost to replace deficient IUID data matrices. Because of this, it may be inaccurate to assume that maintenance programs using IUID technology will achieve the same type or amount of savings claimed by the case studies of programs using other technologies.

Even when a logistics program experiences cost savings after introducing a new technology or process, it can be difficult to link the savings directly to a specific cause or technology. For example, we visited an installation that is using a combination of IUID, passive radio frequency identification, new database software, and a reorganization of warehouse space to reduce the cost of managing its supply chain. However, an installation official explained that it was not possible to determine the extent to which the cost savings were attributable to a specific change, such as the introduction of IUID. For this reason, it may have been incorrect for the task force to assume a link between estimated cost savings and the use of a specific technology such as IUID.

We have previously reported that every estimate is uncertain because of the assumptions that must be made about future projections, and because of this, cost estimators should always perform a sensitivity analysis that demonstrates the effects on the cost and schedule of an assumption that may no longer be valid.that DOD could begin to realize financial savings from IUID implementation after fiscal year 2017, and explains that its cost and benefit estimates are conditional, depending on a number of assumptions. However, the task force report does not contain a sensitivity analysis for either its cost or its benefit estimates. As a result, the task force’s report does not portray the potential effects of changing key assumptions on the report’s estimates of cost, financial benefits, or the time frames in which the report estimates DOD may realize financial benefits.

The task force report estimates There is a substantial amount of uncertainty associated with key assumptions on which the task force report’s estimates are based. For example, the task force report states that the cost to mark legacy items is one of the primary drivers of IUID implementation costs. However, as discussed in more detail later, DOD may face challenges in determining the total number of legacy items it must mark. The task force’s cost estimate does not reflect the range of costs associated with marking a population of legacy items that—according to DOD estimates—may range between about 60 million and 122 million items. In addition, the task force’s estimate of financial benefits assumes that the components’ IT systems, including their Enterprise Resource Planning systems, will have the capability to use UII data for product life cycle management in 2015. However, as discussed later in the report, the components cannot reliably predict when their Enterprise Resource Planning systems will be able to use UII data for product life cycle management. The task force’s estimate of when DOD will begin to realize these benefits does not reflect the possibility that the components’ Enterprise Resource Planning systems will not have this capability in fiscal year 2015.

Also, the task force’s estimates of financial benefits assume IUID implementation across each of DOD’s components. However, as discussed in more detail later, the Navy, Air Force, and Defense Logistics Agency are currently not carrying out key IUID implementation efforts. For example, the Navy is not systematically marking legacy items and the potential integration of IUID with its Enterprise Resource Planning system for Supply is unfunded. In addition, the Air Force is not actively integrating IUID into its Enterprise Resource Planning system. Also, the Defense Logistics Agency is not marking legacy items. The task force’s estimate of financial benefits does not consider that some benefits may not be achieved as a result of DOD’s partial implementation of IUID. Without a sensitivity analysis of its cost and benefit estimates, the task force report does not provide DOD leaders with information about how well the estimates may hold up under reasonable changes to the assumptions on which the estimates are based.

DOD and Contractors Have Made Some Progress Marking Items with IUID, but DOD Faces Challenges in Assessing Progress toward Goals and Ensuring That Items Are Sufficiently Marked

DOD components and contractors have been marking items with IUID, but due to several challenges, it is difficult for DOD to assess its progress in marking items or ensuring that contractors are sufficiently marking items. DOD components have reported marking more than 2 million legacy items, and DOD has identified tens of millions of legacy items that meet its IUID marking criteria. But, DOD does not have complete information on the total number of legacy items that its components have marked and must mark in the future. Moreover, DOD has not developed a full set of quantifiable goals to assess its progress in marking these items. Further, DOD has not set interim milestones to determine the components’ progress in marking items, and DOD’s components do not use consistent criteria to track progress in legacy item marking. With respect to newly-acquired items and pieces of government-furnished property, DOD reports that as of January 2012, more than 2,500 contractors had delivered newly-acquired items to DOD and had registered over 11.5 million such items and pieces of government- furnished property in DOD’s IUID Registry. However, DOD cannot ensure that contractors are sufficiently marking all of the items that require IUID labels, for two reasons: DOD reporting requirements do not provide assurance that appropriate marking clauses are included in all contracts, and DOD components do not have systematic processes to assess the sufficiency of IUID data matrices. As a result, DOD may be unable to ensure that contractors are marking all newly-acquired items and pieces of government-furnished property that require IUID labels, and DOD cannot know the extent to which contractors are supplying IUID data matrices that the components need to track items with IUID technology.

DOD Components Have Made Some Progress in Marking Legacy Items, but Several Challenges Make Progress toward Marking Goals Difficult to Assess

DOD has made some progress in marking legacy items. In late 2004, the Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics established the requirement for DOD components to mark legacy items with IUID labels and for the military services to develop plans to accomplish such marking. An Army depot began marking legacy items in 2005, and according to information from DOD’s IUID Registry, DOD components began registering legacy items in 2006. As of October 2011, the components reported marking more than 2 million legacy items. However, DOD does not have complete information on the total number of items to be marked and that have been marked; DOD has not quantifiably defined its marking goals according to DOD’s IUID marking criteria; DOD has not set interim milestones to determine the components’ progress in marking items; and DOD’s components do not use consistent criteria to track progress in legacy item marking.

The components provided us with estimates of the total number of items that must be marked in the future, according to DOD’s IUID marking criteria. According to the components, the total number of items in their inventories that meet DOD’s IUID marking criteria is about 122 million. In addition, the components provided us with estimates of the total number of legacy items they have marked, if estimates were available. As of October 2011, the Army, Marine Corps, and Air Force reported that they had marked a total of about 2.7 million legacy items. The Navy and Defense Logistics Agency did not report the marking of legacy items.

The Army reported that its inventory contains about 15 million items that meet DOD’s IUID marking criteria, and that it had marked about 1.2 million legacy items.

The Marine Corps reported that its inventory contains about 3.1 million items that meet DOD’s IUID marking criteria, and that it had marked about 0.3 million legacy items.

The Air Force reported that its inventory contains about 13.3 million items that meet DOD’s IUID marking criteria, and that it had marked about 1.2 million legacy items.

The Navy reported that its inventory contains about 60.6 million items that meet DOD’s IUID marking criteria. However, the Navy could not provide an estimate of the number of legacy items it had marked. According to a Navy official, there are “pockets of compliance” within the Navy, in which certain organizations had marked legacy items with IUID labels. But the Navy does not have a Navy-wide, systematic plan or approach to legacy marking; it does not track the number of legacy items being marked within these pockets; and it characterized its progress in legacy marking as “minimal.”

The Defense Logistics Agency reported that its inventory contains about 30.0 million items that meet DOD’s IUID marking criteria. However, according to the agency, it is not currently marking legacy items; does not have the capability or required technical information to mark the legacy items in its inventory; and does not plan to mark legacy items in the future. For example, the agency does not have marking equipment. In addition, agency officials explained that the agency lacks information on how to appropriately mark legacy items with IUID labels. According to the officials, the other components must provide this information to the agency, because the components manage the items that the agency stores in its inventory.

With regard to the total number of items to be marked in the future, some component officials stated that their estimates are incomplete. For example, Army officials explained that their estimate does not include certain classified items because the system they used to estimate the Army’s legacy item inventory does not interface with systems that track those classified items. Navy officials stated that their estimate does not include items that are embedded in other items—such as a circuit board inside of an aircraft—because of system limitations and the time it would have taken to include these items in the Navy’s estimate. Further, Defense Logistics Agency officials explained that their estimate does not include items that are classified by the components as serially managed. Moreover, the components’ estimates of legacy items to be marked in the future do not match the estimate of the total number of legacy items to be marked according to the task force’s report. According to the report, DOD has a total of about 60 million legacy items to be marked. However, the components report that they must mark a total of about 122 million legacy items, and the Navy alone estimates it has 60.6 million legacy items to mark in the future. Because the task force’s estimate is 49 percent smaller than the components’ estimates, DOD may not have complete information on the total number of legacy items in its inventory that meet IUID marking criteria, and that it must mark in the future.

As stated above, the components reported to us that they had marked about 2.7 million legacy items. However, information from the DOD IUID Registry indicates that about 4.9 million legacy items were registered as As previously discussed, an item’s UII may be entered of October 2011.into the registry in one of two ways. First, the item can be marked with an IUID label, and the UII associated with that label is registered. Second, DOD or contractors can establish a virtual UII, registering an item before it is eventually marked with a label. Because the registry’s estimate of legacy items registered is 45 percent larger than the components’ estimate, it is unclear how many legacy items that DOD has marked. Table 2 summarizes the components’ estimates of the total number of legacy items in their inventories that meet DOD’s IUID marking criteria; the task force’s estimate of the total number of legacy items in the components’ inventories that meet DOD’s IUID marking criteria; the components’ estimates of the total number of legacy items they have marked; and the total number of legacy items recorded in the DOD IUID Registry.

An agency’s establishment of goals is a key internal control, and we have previously reported that in the absence of quantifiable targets, it is difficult for officials to assess whether goals were achieved, because comparisons cannot be made between projected performance and actual results. In 2010, the task force recommended that DOD focus its legacy marking efforts on those items from which DOD could derive the greatest benefit. According to an ODASD(SCI) official, it is DOD’s goal to complete the marking of the “majority” of these legacy items by the end of fiscal year 2015. In addition, according to the task force’s report, DOD’s marking of a “significant” number of legacy items is one of the keys to realizing the potential financial benefits of IUID implementation. We have previously reported that, where appropriate, to more easily assess agency progress, performance goals should have quantifiable, numerical targets.DOD’s goals of marking a “majority” of legacy items, or a “significant” number of legacy items, respectively. Further, while some DOD components have quantifiable goals for marking certain legacy items, others do not. As previously discussed, DOD has established IUID marking criteria for different categories of inventory, such as certain items with a unit acquisition cost of $5,000 or greater; certain items that are serially managed; certain items that are controlled (sensitive or classified); and certain depot-level reparable items. We found that some components had set quantifiable goals using these criteria. For instance, the Marine Corps has quantifiable goals for each of the categories defined by DOD’s IUID marking criteria. The Army has quantifiable goals for each of these categories except for items that are controlled (sensitive or classified). Neither the Navy nor the Air Force have established quantifiable goals defined by DOD’s IUID marking criteria.

Once quantifiable goals have been established, we have previously reported that metrics for a program’s main efforts—such as interim milestones and schedules—give decision makers the information needed to assess progress and estimate realistic completion dates. As previously discussed, we have noted that quantifiable target—such as interim milestones—can assist organizations in tracking progress toward their goals. For example, quantifiable interim milestones could assist DOD in evaluating whether the components are marking legacy items at a rate that will allow DOD to meet its fiscal year 2015 goal of marking a “majority” or “significant” number of legacy items in its inventory.

As of January 2012, ODASD(SCI), the task force, and the components had not fully developed quantifiable interim milestones to track progress toward DOD’s goals for marking legacy items. Neither ODASD(SCI) nor the task force had set interim milestones for the number of items that should be marked in fiscal year 2015, DOD’s target date for marking a “majority” of legacy items and a “significant” number of legacy items. Further, while some components have set interim milestones for tracking their progress in marking certain legacy items, others have not. For instance, the Air Force has established interim milestones for the marking but it is still working of some legacy items, such as class VII equipment,on developing interim milestones for others, such as class II items. The Marine Corps has established a goal of completing the marking of about 34 percent of its legacy items by December 2012. However, it has established no interim milestones. Similarly, neither the Navy nor the Defense Logistics Agency has developed interim milestones.

In January 2012, ODASD(SCI) provided us with an IUID timeline containing targets that are not quantified. The IUID timeline indicates that ODASD(SCI) expects the components to be marking legacy items at least through fiscal year 2017. In addition, ODASD(SCI)’s IUID timeline lays out fiscal year 2012 targets for the components to determine the number of items they need to mark, and fiscal year 2013 targets for the components to report on their legacy marking progress to ODASD(SCI). According to ODASD(SCI) officials, the office intends for the components to use the Federal Logistics Information System to eventually provide quantifiable reports on their legacy marking efforts.DASD(SCI) stated that he has asked the components to develop quantifiable interim milestones for legacy marking. The components’ use of the system may provide them with a means to report progress in regard to these milestones. However, ODASD(SCI) officials stated that the system does not yet have this capability and they do not yet have an estimate for when the components could begin their reporting. Until ODASD(SCI) and the components begin to use such milestones to assess DOD’s progress in marking legacy items, it is difficult to know whether DOD’s current number of marked legacy items represents what DOD intended to achieve, almost 7 years after DOD established the requirement to mark legacy items.

GAO/GGD-96-118. milestones, and the tracking of progress with categories defined by DOD’s IUID marking criteria is summarized in table 3.

While legacy items are marked by DOD, newly-acquired items and government-furnished property must be marked by DOD’s contractors. DOD has made some progress in ensuring that these types of items are marked by contractors. According to the DOD IUID Registry, more than 2,500 contractors had registered over 11.5 million newly-acquired items in the registry as of January 2012. DOD plans to eventually finish marking its legacy items, but contractors will continue to mark items that are acquired by DOD, or provided by DOD to contractors, and meet its IUID marking criteria. According to January 2012 data from the DOD IUID Registry, the number of newly-acquired items and government-furnished property already exceeds the number of legacy items. If current marking trends continue, the ratio of these items to legacy items will continue to increase, and newly-acquired items and government-furnished property will continue to make up the majority of DOD’s inventory of IUID-labeled items. For this reason, the future success of DOD’s IUID implementation efforts depends on having contractors sufficiently mark newly-acquired items and government-furnished property with IUID labels. While DOD has made some progress, it cannot currently ensure that contractors are sufficiently marking all of the items that require IUID labels, for two reasons: reporting requirements do not provide assurance that appropriate IUID-marking contract clauses are included, and DOD’s inspection efforts are not systematic. Without adequate reporting requirements regarding the components’ insertion of IUID clauses into applicable contracts, DOD cannot know the extent to which it is requiring contractors to mark all items that should have IUID labels. And, without sufficient inspection of IUID data matrices, DOD cannot know the extent to which contractors are supplying deficient data matrices.

In order for DOD to use IUID technology to track contractor-marked items that qualify for IUID marking, DOD and its contractors must take certain steps to help ensure that qualified items are marked, and that the IUID marks are usable. These steps are as follows: in contracts for qualified items, the components must ensure that the contracts contain appropriate contract clauses and that those clauses are correctly completed. These clauses require the contractor to mark or register qualified items; the contractor must then mark items with an IUID label with a data matrix that can be read electronically or establish a virtual UII in some cases; and the data matrix must contain the necessary information, organized with the proper data elements and syntax, which must be registered in DOD’s IUID Registry.

Defense Federal Acquisition Regulation Supplement section 252.211-7003. label and whether that label’s data matrix is formatted according to DOD- wide standards. Officials from several components told us that they had inspected some newly-acquired items to determine whether these items were sufficiently marked with IUID labels. For the items they reviewed, those inspections helped to detect problems in contractors’ marking of items. Also, since 2009, the Defense Contract Management Agency has had a surveillance program in place to inspect newly-acquired items and assess contractors’ compliance with the acquired-items clause.

A DOD memorandum requires that the components report to the Office of Defense Procurement and Acquisition Policy on only a portion of contracts that should include the clauses related to IUID. According to the memorandum, components are required to report on, among other things, whether the acquired-items clause is present in contracts for newly-acquired items. However, they are not required to report on whether the government-furnished property clause is present in contracts involving government-furnished property. Several components are thus reporting only on whether they are including the acquired-items clause in contracts. While the Air Force reports on whether it is including both the acquired-items clause and the government-property clause in contracts, the Army, Navy, Marine Corps, and Defense Logistics Agency report only on whether they are including the acquired-items clause. While certain components are reviewing contracts for items that cost more than $5,000 or are serially managed, none of the components are reviewing contracts for items that meet other DOD IUID marking criteria, such as those for controlled (sensitive or classified) items. Consequently, DOD does not know the full extent to which the components are complying with the requirements to include IUID-related clauses in contracts. Without this information, DOD may be unable to ensure that contractors are marking all newly-acquired items and pieces of government-furnished property that require IUID labels.

According to DOD officials, it is a good practice for either a DOD component or the Defense Contract Management Agency to inspect items to ensure that contractors have marked items with IUID labels, and that the labels’ data matrices are not deficient. Inspection of newly- acquired items is important because contractors have been delivering IUID labels with deficient data matrices that cannot be used by DOD. However, neither the components nor the Defense Contract Management Agency have been systematically inspecting the data matrices in IUID labels applied to items by contractors. According to both the Marine Corps and the Air Force, more than 10 percent of the newly-acquired items’ data matrices they examined after receipt of the items from the contractor have been deficient. For example, one Marine Corps installation reported that from January 2010 through October 2011, there were 8 months in which more than 10 percent of newly-acquired items provided by contractors were marked with deficient data matrices. Problems include having matrices in which the syntax of the UII data was incorrect or missing key elements; matrices that could not be electronically read; and matrices that contained a UII number that had not been registered in DOD’s IUID Registry.

Although the Marine Corps and Air Force have assessed a portion of the data matrices on newly-acquired items in their inventories, these components are not systematically assessing whether contractors are sufficiently marking these items. For example, the Marine Corps estimates that it has assessed the sufficiency of contractors’ marking for about 79 percent of all newly-acquired items’ data matrices in its inventory. Officials from both the Marine Corps and Air Force explained, however, that neither has developed a systematic approach for inspecting these items’ data matrices. According to Army officials, the Army also lacks a systematic approach to inspecting these items’ data matrices. For example, the two depots identified by the Army as furthest along in IUID implementation have not established a policy or set procedures for assessing the sufficiency of the data matrices of newly-acquired items in their inventories. Defense Logistics Agency officials explained that although its personnel do perform various types of inspection and acceptance procedures on items delivered to its sites, personnel are not inspecting items’ data matrices. In addition, the Navy does not have a policy or plans in place to systematically assess newly-acquired items’ data matrices.

According to the Defense Contract Management Agency’s information memorandum that describes procedures for inspecting contractors’ data matrices, the agency’s inspectors are to verify the readability of these matrices if a scanner for reading matrices is available at the inspection site. However, as previously discussed, the only way to assess the functionality of a data matrix is to use a tool that can electronically read the matrix. Because its memorandum does not require inspectors to electronically read data matrices in all cases, the Defense Contract Management Agency cannot ensure that the items it inspects have an IUID label with a data matrix that can be properly read; that the data matrix contains the necessary UII data, organized with the proper syntax; and that the item’s UII number is registered in DOD’s IUID Registry. Furthermore, officials from the Defense Contract Management Agency said their inspectors rely on contractors to provide the technology to electronically read data matrices. If contractors do not provide this technology, inspectors at those manufacturing sites cannot electronically read and verify the sufficiency of data matrices.

The Office of Defense Procurement and Acquisition Policy and the components have taken steps to address some of these challenges. For example, in February 2011, the office issued a DOD standard operating procedure for assessing the sufficiency of data matrices. Also, through the Product Quality Deficiency Report process, Marine Corps item managers are beginning to work with contractors to address the contractors’ delivery of deficient data matrices. In addition, in the first quarter of fiscal year 2012, the Air Force began to track the number of deficient data matrices it is discovering as it assesses the sufficiency of newly-acquired items’ data matrices. However, unless all of the components and the Defense Contract Management Agency are systematically assessing whether contractors are sufficiently marking newly-acquired items, DOD cannot know the full extent to which contractors are supplying deficient data matrices.

DOD Has Made Some Progress in Preparing to Share UII Data Enterprisewide, but Faces Challenges

DOD has made some progress in developing a capability to share UII data enterprisewide and integrating IUID functionality with its Enterprise Resource Planning systems. For example, DOD is in the process of revising key guidance on using IUID technology and UII data across DOD, and three components are temporarily storing UII data until they are ready to use these data in their Enterprise Resource Planning systems. However, DOD faces several challenges in sharing UII data enterprisewide, and it is unlikely that it will meet a fiscal year 2015 goal to use UII data for the management of items in enterprise information systems. Further, DOD cannot reliably predict when it will meet this goal, because ODASD(SCI) and the components have not fully scheduled for integrating IUID functionality with the IT systems through which the components plan to achieve this capability, their Enterprise Resource Planning systems.

To Achieve Key Benefits, DOD Plans to Use IUID Technology in Enterprisewide Logistics Processes by the End of Fiscal Year 2015

It is DOD’s goal for its components to share UII data departmentwide, and the components are to use these data for unique item tracking. According to a DOD instruction,Acquisition Policy is to ensure unique IUID identifiers are established to enable items to be tracked and traced throughout their life cycle in acquisition and logistics business processes and systems, in an integrated approach across DOD. Further, a 2011 IUID implementation schedule from ODASD(SCI) states that certain DOD item management processes are to be using UII data to manage items by the end of fiscal year 2015. Specifically, according to the schedule, by fiscal year 2015, two categories of logistics processes—intensive item management and product life cycle management—are to use IUID technology. the Director for Defense Procurement and As previously discussed, the task force report assessed the potential benefits of using IUID technology in these logistics processes. In both cases, the report explains that sharing UII data across DOD is key to realizing the full benefits of these processes. Regarding intensive item management, the report states it is clear that DOD requires an enterprisewide approach to managing critical items and the largest benefits of managing items intensively would be achieved by using UII data across the enterprise.

With regard to product life cycle management, the task force report estimates that DOD could achieve substantial financial benefits through the use of UII data in this process. In our previous discussion, we explained that the task force report’s methodology for estimating these financial benefits may not be appropriate. However, if DOD is to achieve potential benefits of product life cycle management, the report explains that benefits would come primarily through analysis of UII data, and that DOD should expect to see the full benefit of this analysis as its Enterprise Resource Planning systems begin sharing and using these data. As previously mentioned, the report states that DOD could begin to achieve the net financial benefits of IUID implementation in fiscal year 2017. However, in order to do so, the report assumes that DOD will have the capability to share and use UII data, enterprisewide, by fiscal year 2015.

DOD and Its Components Have Made Some Progress in Integrating IUID with Their Enterprise Resource Planning Systems

Since DOD began its IUID technology implementation efforts in fiscal year 2004, it has made some progress in preparing to share UII data enterprisewide through its Enterprise Resource Planning systems, in two main areas. First, DOD is in the process of modifying its supply chain management policy and guidance to incorporate use of IUID technology and UII data across DOD. Second, three of the components have developed IT systems to temporarily store data from IUID-labeled items until these data can be uploaded into Enterprise Resource Planning systems.

In January 2012, ODASD(SCI) provided us with sections from a draft revision to the regulation that establishes DOD’s supply chain management processes and procedures, including sections pertaining to IUID. The draft sections we reviewed define standards and procedures for using IUID in DOD Enterprise Resource Planning systems. If implemented, the revisions would likely help DOD move forward in its integration of IUID technology and UII data with its IT systems—including its Enterprise Resource Planning systems—in two ways. First, the draft revisions would establish standards for acceptable electronic scanners, which should help ensure interoperability across DOD organizations that are scanning and uploading UII data from IUID labels’ data matrices. Second, the draft revisions would require DOD organizations to update their UII data-sharing capabilities by adopting a system to share UII data, such as the Defense Logistics Management Standards; according to DOD officials, this system is replacing an older one that is unable to share UII data.for that reason essential to IUID implementation.

Because their Enterprise Resource Planning systems are not currently capable of accepting or storing UII data at a componentwide level, the Army, Marine Corps, and Air Force have developed IT systems to temporarily store UII data generated by the labeling of both legacy and newly-acquired items. However, these systems have limited capabilities to manage or use UII data. For example, these temporary systems are not capable of sharing UII data within or between components. As of January 2012, the Air Force’s temporary system was limited to use on individual computer workstations, and could not send or receive UII data from other Air Force or DOD computers. In addition, the Marine Corps used its temporary system to provide us with information on its inventory of IUID-labeled legacy items, but the system was not designed to perform more complex tasks such as analyzing UII data in support of product life cycle management processes.

DOD Faces Several Challenges in Integrating IUID and Is Unlikely to Meet Its Fiscal Year 2015 Goal

DOD faces three challenges in sharing UII data enterprisewide and integrating IUID functionality with its Enterprise Resource Planning systems. First, ODASD(SCI) and the components have not fully defined the requirements for using UII data across DOD, or within the components’ Enterprise Resource Planning systems. Second, as of April 2012, the Air Force and the Navy were not actively integrating IUID with their Enterprise Resource Planning systems. And third, ODASD(SCI) and the components have not fully scheduled for integration of IUID functionality with their Enterprise Resource Planning systems. As a result, DOD is unlikely to meet its fiscal year 2015 goal to use UII data in intensive item management and product life cycle management.

Officials from the Army, Marine Corps, Air Force, and Defense Logistics Agency said that their components had not yet fully defined the component-specific UII requirements for their respective Enterprise Resource Planning systems. A Marine Corps official stated that, as of January 2012, requirements for how the Marine Corps system will interface with scanners were in draft form. Officials from the Defense Logistics Agency explained that because the agency manages items on the basis of the requirements of the other components, it could not finalize the business rules for using UII data in its system until the other components had determined their requirements. According to DOD officials, it is unclear when the requirements or related business rules will be fully defined and, until they are defined, the components cannot complete their integration of IUID technology with their IT systems, including their Enterprise Resource Planning systems.

As of April 2012, the Air Force and the Navy were not actively integrating IUID with their Enterprise Resource Planning systems. According to Air Force officials, because of cost overruns and delays in the development of its Enterprise Resource Planning system, the Office of the Secretary of Defense and the Air Force are planning to evaluate alternatives to the system. Because this system is central to the Air Force’s IUID implementation efforts and the Air Force does not know when a decision will be made, officials stated that they cannot estimate when—or whether—the system will be ready to share and use UII data. According to Air Force officials, the Air Force has a data network that provides the capability to share UII data within the Air Force, between certain IT systems. However, they stated that this data network does not currently have the capability to share UII data with other components, across DOD. Further, the officials stated that the Air Force’s plan is to eventually share UII data enterprisewide through its Enterprise Resource Planning system. In October 2011, senior Navy officials stated that the Navy had no plans to integrate IUID with its Enterprise Resource Planning system for Supply, and will not be ready to share or use UII across its systems—or with other components’ systems—by the end of fiscal year 2015. Further, they explained that the Navy was not actively integrating IUID with its Enterprise Resource Planning system for Supply because strict budget conditions compelled Navy leadership to allocate funds to programs the Navy considered to be of higher priority than IUID implementation. As of April 2012, the Navy stated that proposed integration efforts remained unfunded.

Because neither ODASD(SCI) nor the components have complete integrated master schedules for the integration of IUID functionality with their Enterprise Resource Planning systems, DOD cannot reliably predict when it will be able to use these systems to meet its fiscal year 2015 goal to use UII data in intensive item management and product life cycle management. A key internal control is the use of performance measures, and we have previously reported that such metrics for a program’s main efforts—including interim milestones and schedules—give decision makers the information needed to assess progress and estimate realistic completion dates. In addition, we have reported that a reliable schedule—such as an integrated master schedule—is crucial to estimating the overall timeline and cost of IT programs, including Enterprise Resource Planning systems. An integrated master schedule is the time-phased schedule DOD and other agencies use for assessing technical performance. It contains the detailed tasks or work packages necessary to ensure program execution. Further, we have reported that without fully integrating the distinct activities that make up an IT program with such a schedule, an organization will not be able to measure its progress toward completion and cannot be held accountable for results.

Although the Army has integrated master schedules for its two Enterprise Resource Planning systems, these schedules lack key elements, such as distinct activities for IUID integration and detailed processes for transmission of UII data across the systems. The Air Force has an integrated master schedule for its Enterprise Resource Planning system, and this schedule has distinct activities for IUID integration. However, as previously discussed, the Office of the Secretary of Defense and the Air Force are in the process of evaluating whether to modify, or cancel and replace, the Air Force’s Enterprise Resource Planning system. According to Air Force officials, the system’s current schedule will need to be revised once the future of the system has been determined. The Marine Corps has an integrated master schedule for its Enterprise Resource Planning system, and Marine Corps officials have stated that they plan to amend the schedule to include distinct IUID activities. However, as of January 2012, it did not contain them. As discussed previously, the Navy is not actively integrating IUID with its Enterprise Resource Planning system for Supply and does not have an integrated master schedule for integrating IUID with its Enterprise Resource Planning system. ODASD(SCI) has produced an IUID timeline that contains general targets for the fielding of IUID-capable Enterprise Resource Planning systems. However, an ODASD(SCI) official stated that it does not have an integrated master schedule to coordinate or track the progress of the components’ efforts to integrate IUID with their Enterprise Resource Planning systems. Officials from ODASD(SCI) and several components have reported that they are unsure of when the components’ Enterprise Resource Planning systems will be able to share UII data within their networks, a key capability for both intensive item management and product life cycle management.

Given the challenges ODASD(SCI) and the components face in sharing UII data enterprisewide and integrating IUID with their Enterprise Resource Planning systems, DOD likely will face difficulties in meeting its IUID integration goals. Without fully defined requirements; resolving the challenges posed by the Air Force and the Navy not actively integrating IUID with their Enterprise Resource Planning systems; and an integrated master schedule that includes IUID integration at the component level and at the DOD-wide level, DOD cannot reliably predict whether it will meet its goal to use these systems to manage items through intensive item management and product life cycle management by the end of fiscal year 2015, or predict when these systems will have this capability.

Conclusions

Using UII data could enable DOD to improve accountability and management of equipment and materiel, and increase efficiencies in maintenance, which could potentially result in cost savings in some cases. DOD is in the process of developing a framework for managing and implementing IUID technology, but could benefit from fully implementing best management practices that would enable the department to better determine the costs and benefits of IUID implementation, and progress toward goals. DOD components have reported marking more than 2 million legacy items, and DOD has made some progress in ensuring that its contractors are marking newly- acquired items with IUID. As of January 2012, though, DOD’s implementation of IUID technology faces several substantial challenges. For example, while the components report marking more than 2 million items, DOD does not have quantifiable goals or interim milestones that it can use to assess progress in achieving its fiscal year 2015 goal of marking a “majority” of legacy items, or the task force’s goal of marking a “significant” number of legacy items. With regard to items that must be marked by contractors, in the absence of policies and procedures that establish a systematic process for assessing the sufficiency of contractor- supplied data matrices, DOD is unable to determine the extent to which contractors are sufficiently marking items. This limits DOD’s ability to ensure that it can track those items. Also, DOD has not fully developed the schedules needed to integrate IUID with existing IT systems, so that DOD can share UII data enterprisewide. This impedes its successful integration of IUID technology with these systems by the end of fiscal year 2015, its stated goal, and prevents the department from determining when it might achieve this integration. At a time when the nation faces fiscal challenges, and defense budgets are becoming tighter, DOD leaders’ lack of key information on IUID implementation could hinder sound program management and decision making.

Recommendations for Executive Action

We recommend that the Secretary of Defense direct the Under Secretary of Defense for Acquisition, Technology, and Logistics to complete its implementation and management framework for IUID by incorporating key elements of a comprehensive management approach, such as a complete analysis of the return on investment, quantitatively-defined goals, and metrics for measuring progress. To do so, we recommend that the Secretary of Defense direct the following organizations to take six actions:

The Under Secretary of Defense for Acquisition, Technology, and Logistics to update the IUID task force report’s estimates of costs and benefits by incorporating key elements of a sound investment analysis including a more complete estimate of all associated costs, an appropriate methodology for estimating benefits, and a sensitivity analysis of these estimates.

The Under Secretary of Defense for Acquisition, Technology, and Logistics, in coordination with the components, develop quantitatively- defined goals for the number of legacy items that may allow DOD to achieve the Task Force’s estimate of IUID’s potential benefits, by marking a “significant” number of these legacy items, or meet ODASD(SCI)’s goal that DOD needs to mark a “majority” of these legacy items by fiscal year 2015.

The Under Secretary of Defense for Acquisition, Technology, and Logistics, in coordination with the components, establish quantifiable interim milestones for marking legacy items that allow DOD to track progress toward its goals.

The Under Secretary of Defense for Acquisition, Technology, and Logistics, in coordination with the components, track progress using a consistent set of criteria, such as DOD’s IUID marking criteria.

The components and the Defense Contract Management Agency develop policies and procedures that provide for systematic assessment of the sufficiency of contractor-marked items’ data matrices.

The Under Secretary of Defense for Acquisition, Technology, and Logistics require the components to examine and report to the Office of Defense Procurement and Acquisition Policy on all types of contracts that should include the acquired-items and government- property clauses.

In addition, to enable DOD to successfully share UII data enterprisewide and integrate IUID functionality with its Enterprise Resource Planning systems, we recommend that the Secretary of Defense direct the Under Secretary of Defense for Acquisition, Technology, and Logistics to coordinate with the military services and the Defense Logistics Agency to take the following two actions:

Define the requirements for using UII data across DOD and within the components’ Enterprise Resource Planning systems.

Develop or revise integrated master schedules for the integration of IUID technology with the components’ individual Enterprise Resource Planning systems—and between these systems—across DOD. These schedules should fully integrate distinct IUID activities.

We also recommend that the Secretary of Defense direct the Secretary of the Navy to develop a plan to share UII data enterprisewide.

Agency Comments and Our Evaluation

In commenting on a draft of this report, DOD concurred with eight recommendations and partially concurred with one recommendation. DOD’s comments are reprinted in appendix III. DOD also provided technical comments, which we considered and incorporated where appropriate.

DOD concurred with our recommendation to develop quantitatively- defined goals for the number of legacy items that may allow DOD to achieve the task force’s estimate of IUID’s potential benefits. DOD stated that an IUID working group will identify the target population of items that qualify for IUID marking in a list of the items’ National Stock Numbers and DOD will track progress in its marking of individual items on this list according to component IUID implementation plans that are due to be submitted to the Assistant Secretary of Defense for Logistics and Materiel Readiness by September 2012.

DOD concurred with our recommendation to establish quantifiable interim milestones for marking legacy items that allow DOD to track progress toward its goals. DOD stated that its IUID working group will establish interim milestones to track the progress of marking legacy assets as part of the development of component IUID implementation plans to be submitted to the Assistant Secretary of Defense for Logistics and Materiel Readiness by September 2012.

DOD concurred with our recommendation to track progress using a consistent set of criteria. DOD stated that progress will be tracked using a consistent set of criteria, once developed by the IUID working group.

DOD concurred with our recommendation to develop policies and procedures that provide for systematic assessment of the sufficiency of contractor-marked items’ data matrices. DOD stated that the Defense Contract Management Agency has risk-based assessment policies and procedures in place. According to DOD, these include a review of contracts to determine whether they contain an IUID requirement; surveillance of a contractor’s IUID marking; and an IUID checklist that requires agency personnel to examine an item’s data matrix. DOD explained that agency personnel assess the sufficiency of a data matrix by electronically reading it with a scanner supplied by a contractor or through a statement of quality from contractors that the agency has determined have adequate quality control. We believe that DOD’s concurrence with our recommendation may lead to the components improving their capability to systematically assess these matrices, and that the agency’s policies and procedures may assist its inspectors in doing the same. However, our review of the policies and procedures provided by the agency indicate that it does not require inspectors to assess the sufficiency of data matrices in all cases. For example, if a contractor does not provide evidence that it has marked items with sufficient data matrices, and no IUID scanner is available on site, neither the agency’s 2009 information memorandum describing procedures for inspecting contractors’ data matrices, nor its IUID checklist, provide an alternative method for inspectors to assess the sufficiency of items’ matrices. Because of this, we continue to believe that the agency cannot ensure that the items it inspects have IUID labels with sufficient data matrices, and that it should continue to develop policies and procedures that provide for systematic assessment of the sufficiency of contractor- marked items’ data matrices.

DOD concurred with our recommendation for the components to examine and report to the Office of Defense Procurement and Acquisition Policy on all types of contracts that should include the acquired-items and government-property clauses. DOD stated that the components will provide contract evaluation reports—for items meeting any of DOD’s IUID criteria, as well as pieces of government-furnished property that meet these criteria—and report on compliance with the requirements to include the appropriate Defense Federal Acquisition Regulation Supplement IUID clauses in contracts.

DOD concurred with our recommendation to define the requirements for using UII data across DOD and within the components’ Enterprise Resource Planning systems. DOD stated that the IUID working group will define DOD-wide IUID functional requirements.

DOD concurred with our recommendation to develop or revise integrated master schedules for the integration of IUID technology with the components’ individual Enterprise Resource Planning systems—and between these systems—across the department. DOD stated that the components have been tasked to submit revised IUID implementation plans to the Assistant Secretary of Defense for Logistics and Materiel Readiness by September 2012. We believe that such plans could assist DOD in improving its management approach for the implementation of IUID. However, as previously discussed, an integrated master schedule has specific characteristics that make it distinct from an implementation plan. Specifically, an integrated master schedule is a time-phased schedule that DOD and other agencies use for assessing technical performance. It contains the detailed tasks or work packages necessary to ensure program execution. Further, we have reported that without fully integrating the distinct activities that make up an IT program with such a schedule, an organization will not be able to measure its progress toward completion and cannot be held accountable for results. Because of this, we continue to believe that integrated master schedules should be developed or revised for the integration of IUID technology with the components’ Enterprise Resource Planning Systems.

DOD concurred with our recommendation that the Secretary of the Navy develop a plan to share UII data enterprisewide. DOD stated that the Navy—participating in the IUID working group—will develop IUID requirements as part of the working group’s definition of DOD-wide IUID functional requirements. Regarding the specific requirement for IUID functionality in the Navy’s Enterprise Resource Planning system—Navy Enterprise Resource Planning system for Supply—DOD stated that the Assistant Secretary of Defense for Logistics and Materiel Readiness will continue to work with the Chief of Naval Operations (Deputy Chief of Naval Operations ) to develop a plan to include IUID requirements in this system.

DOD partially concurred with our recommendation to update the IUID task force report’s estimates of costs and benefits by incorporating key elements of a sound investment analysis, including a more complete estimate of all associated costs, an appropriate methodology for estimating benefits, and a sensitivity analysis of these estimates. DOD stated that the benefits to DOD of implementing IUID marking are to improve asset accountability, tracking, and the life cycle management of targeted items. Further, DOD stated that it will continue to identify costs of implementing IUID as IUID is implemented across DOD. As previously discussed, a best practice for analyzing a program’s return on investment is the estimation of all potential costs, and DOD efforts to continue to identify costs of IUID implementation may be a positive step in this direction. According to DOD, another best practice for analyzing a program’s return on investment is analyzing benefits, and making recommendations, based on relevant evaluation criteria. DOD has estimated that IUID implementation could cost $3.2 billion, and the components report that they have already spent at least $219 million on implementation efforts. Moreover, DOD has estimated that implementing IUID technology could save $3 billion to $5 billion per year. As previously discussed, DOD may have used a methodology for estimating these benefits that may not be appropriate to the scale and complexity of DOD’s IUID implementation efforts. For example, the task force estimated DOD- wide savings on the basis of a limited number of case studies, these case studies did not address programs that use IUID as the technology that provides a unique identifier to track items through serialized item management, and even when a logistics program experiences cost savings after introducing a new technology or process, it can be difficult to link the savings directly to a specific cause or technology such as IUID. Given IUID’s potential costs and that DOD’s methodology for estimating IUID’s potential financial benefits may not be appropriate, we continue to believe that an estimate of both IUID’s costs and benefits, based on an appropriate methodology, and a sensitivity analysis of these estimates, would provide DOD leaders with key information to better enable sound program management and determine whether continued spending on IUID is likely to result in a significant return on investment.

We are sending copies of this report to the Secretaries of Defense, the Army, the Navy, and the Air Force; the Commandant of the Marine Corps; and the Directors of the Defense Logistics Agency and the Defense Contract Management Agency. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-5257 or merrittz@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Scope and Methodology

To determine the extent to which the Department of Defense (DOD) has a comprehensive management approach for its implementation of item unique identification (IUID), we reviewed previously published DOD and GAO work to identify best practices; the IUID implementation framework documentation provided by the Office of the Deputy Assistant Secretary of Defense for Supply Chain Integration (ODASD); the DOD IUID task force’s analysis of the potential costs and benefits of IUID implementation; and the components’ estimates of historical spending and fiscal year 2012 budget requests for IUID implementation.

To determine the extent to which DOD components have marked legacy items with IUID, we reviewed DOD criteria for the types of items to be marked with IUID labels; reviewed DOD’s plans for marking legacy items; and gathered data on the number and type of legacy items in components’ inventories, and how many had been marked as of October 2011. To determine the extent to which DOD has taken steps to ensure that items are sufficiently marked by contractors with IUID, we reviewed the Defense Federal Acquisition Regulation Supplement clauses that require contractors to mark items with labels—or establish virtual unique item identifiers (UII)—and register the items, and we gathered data from the components on the quality of contractors’ IUID data matrices.

To determine the extent to which DOD has integrated IUID with its enterprise information systems, we reviewed our previously published work on best practices for the planning of large-scale information- reviewed DOD-wide and component-level policy on technology efforts;the use of UII data in its information technology systems, which include DOD Enterprise Resource Planning systems; reviewed DOD-wide and component-level integrated master schedules for the integration of IUID technology with these systems, if available; and reviewed other types of existing schedules and system planning documents.

In addition, we visited selected sites to observe key IUID activities. To select the sites we used a nongeneralizable, judgmental sample based on a number of criteria, including DOD component and the type of IUID activity performed at the site. For all our objectives, we interviewed officials knowledgeable about DOD’s IUID implementation efforts, including officials from ODASD(SCI), as well as other officials from the Office of the Secretary of Defense, the components, and the Defense Contract Management Agency.

We assessed the reliability of all computer-generated data provided by DOD for each of our objectives by reviewing existing information about the data and the systems that produced the data and by interviewing agency officials knowledgeable about the data to determine the steps taken to ensure the accuracy and completeness of the data. In the course of our assessment, we reviewed estimates provided by the components on the number of legacy items marked and to be marked; deficient data matrices; and historical and requested IUID spending.

Each of the components provided estimates on either the number of legacy items marked, or to be marked; several of the components provided estimates of both. On the basis of our review of the sources and methodology used by the Army, Marine Corps, and Air Force to produce estimates of the number of legacy items they have marked, we determined that these data are sufficiently reliable for the purposes of reporting the components’ best estimates of the size of this population of items. The Navy was not able to estimate the number of legacy items it has marked; the Defense Logistics Agency reported that it has not marked legacy items. Based on our review of the sources and methodology used by the Marine Corps and Air Force to produce estimates of the number of legacy items they must mark in the future, we determined that these data are sufficiently reliable for the purposes of reporting on the components’ best estimates of the size of this population of items. As previously discussed, the Army, Navy, and Defense Logistics Agency explained that their estimates on the number of legacy items that must be marked in the future are not complete. Although not complete, we determined that the data on legacy items represent the components’ best estimates, and are sufficiently reliable for the purposes of reporting on the general size of the population of legacy items they must mark in the future.

The Army, Navy, and Defense Logistics Agency did not provide data on deficient data matrices; the Marine Corps and Air Force did provide these data. We reviewed the data sources and methodology used by the Marine Corps and Air Force to produce data on the number of deficient data matrices they have discovered through their review of a portion of data matrices on newly-acquired items in their inventories. We determined that these data are sufficiently reliable for the purposes of reporting on the percentage of data matrices that these components classified as deficient, out of the portion of data matrices in their inventories that they have assessed for sufficiency.

The Defense Logistics Agency did not provide estimates on either historical or requested IUID spending; the Army and Navy provided estimates on historical IUID spending; and the Marine Corps and Air Force provided estimates of both historical and requested IUID spending. We discussed with component officials the sources and methodology they used to produce the data on their historical IUID spending and fiscal year 2012 budget requests for IUID spending. The Marine Corps and Air Force provided data on both their historical and future spending. On the basis of our review of their sources and methodology for producing these data, we determined that the spending data provided by the Marine Corps and Air Force are sufficiently reliable for the purposes of reporting on these components’ historical spending and fiscal year 2012 IUID budget requests. As previously discussed, Army and Navy officials explained that their estimates on historical IUID spending are not complete. Although not complete, we determined that these data represent the best estimates of the Army and Navy on their historical IUID spending, and are sufficiently reliable for the purposes of reporting on the historical spending data that are available.

Appendix II: Key IUID Policy Issuances and Implementation Events, Fiscal Year 2003 to Fiscal Year 2011

Appendix III: Comments from the Department of Defense

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Kimberly Seay, Assistant Director; Emily Biskup; Cindy Brown Barnes; Cynthia Grant; Neelaxi Lakhmani; Jason Lee; Alberto Leff; John Martin; Charles Perdue; Carol Petersen; Karen Richey; Darby Smith; Chris Turner; Cheryl Weissman; and Michael Willems made key contributions to this report.

Related GAO Products

Defense Logistics: DOD Needs to Take Additional Actions to Address Challenges in Supply Chain Management. GAO-11-569. Washington, D.C.: July 28, 2011.

High-Risk Series: An Update. GAO-11-278. Washington, D.C.: February 16, 2011.

DOD’s 2010 Comprehensive Inventory Management Improvement Plan Addressed Statutory Requirements, but Faces Implementation Challenges. GAO-11-240R. Washington, D.C.: January 7, 2011.

DOD Business Transformation: Improved Management Oversight of Business System Modernization Efforts Needed. GAO-11-53. Washington, D.C.: October 7, 2010.

DOD’s High-Risk Areas: Observations on DOD’s Progress and Challenges in Strategic Planning for Supply Chain Management. GAO-10-929T. Washington, D.C.: July 27, 2010.

Department of Defense: Additional Actions Needed to Improve Financial Management of Military Equipment. GAO-10-695. Washington, D.C.: July 26, 2010.

DOD’s High-Risk Areas: Actions Needed to Reduce Vulnerabilities and Improve Business Outcomes. GAO-09-460T. Washington, D.C.: March 12, 2009.

Defense Logistics: Lack of Key Information May Impede DOD’s Ability to Improve Supply Chain Management. GAO-09-150. Washington, D.C.: January 12, 2009.

DOD’s High-Risk Areas: Efforts to Improve Supply Chain Can Be Enhanced by Linkage to Outcomes, Progress in Transforming Business Operations, and Reexamination of Logistics Governance and Strategy. GAO-07-1064T. Washington, D.C.: July 10, 2007.

Defense Logistics: Efforts to Improve Distribution and Supply Support for Joint Military Operations Could Benefit from a Coordinated Management Approach. GAO-07-807. Washington, D.C.: June 29, 2007.

DOD’s High-Risk Areas: Progress Made Implementing Supply Chain Management Recommendations, but Full Extent of Improvement Unknown. GAO-07-234. Washington, D.C.: January 17, 2007.

DOD’s High-Risk Areas: Challenges Remain to Achieving and Demonstrating Progress in Supply Chain Management. GAO-06-983T. Washington, D.C.: July 25, 2006.