Defense Acquisition: DOD Should Clarify Requirements for Assessing and Documenting Technical-Data Needs
Published: 2011-05-11T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Some of the Department of Defense's (DOD) weapon systems remain in the inventory for decades. Therefore, decisions that program officials make during the acquisition process to acquire or not acquire rights to technical data, which may cost $1 billion, can have far-reaching implications for DOD's ability to sustain and competitively procure parts and services for those systems. DOD needs access to technical data to control costs, maintain flexibility in acquisition and sustainment, and maintain and operate systems. In response to a congressional request, GAO reviewed the extent to which: (1) DOD has updated its acquisition and procurement policies to reflect a 2007 law and 2006 GAO recommendations; (2) selected acquisition programs adhered to requirements to document technical-data needs; and (3) DOD took actions to improve technical-data decisions by program managers. GAO interviewed DOD officials, reviewed acquisition strategies and acquisition plans from 12 programs, and compared those documents to relevant DOD policies.
section_title: What GAO Found
paragraphs: DOD updated its acquisition and procurement policies to require that acquisition program managers document their long-term technical-data needs in a manner that reflects a 2007 law and GAO's 2006 recommendations. Together these policies require documentation of: (1) an assessment of technical-data requirements, (2) the merits of a "priced-contract option" that enables DOD to obtain additional technical data that it did not acquire in its initial contract, (3) the contractor's responsibility to verify its assertions of limits to DOD's ability to use the technical data, and (4) the potential for changes in the system's sustainment plan. According to DOD officials, these policy updates do not require changes to the way program managers assess technical-data needs. Sampled acquisition programs partially addressed the four updated technical-data-documentation requirements. Ten of the 12 programs GAO reviewed addressed at least 1 of the 4 requirements in their acquisition strategies and acquisition plans; however, none of the programs addressed all 4 of the requirements. Specifically, 9 of the 12 strategies documented an assessment of their technical-data requirements. For example, the strategy for a Navy communications system stated that the program planned to obtain technical data and associated rights to sustain the system over its life cycle and allow for competitive procurement of future systems. In contrast, 3 of the 12 strategies documented the contractor's responsibility to verify its assertions of limits to DOD's ability to use the technical data. Each of the three strategies noted that the program planned to include a clause in its contracts that identifies the contractor's responsibilities. DOD has issued guides--that are voluntary for the program managers to use--to improve technical-data decision-making. These guides may help program managers with decisions and documentation on technical data. However, DOD technical-data policies remain unclear. Effective internal controls help organizations implement their directives. GAO found that, because DOD has not issued clarifications to its policy, DOD policies that require documentation of long-term technical-data needs are unclear. As a result, acquisition strategies have not always documented required information on technical data--a point the Office of the Under Secretary of Defense for Acquisition, Technology and Logistics recently emphasized. Because of the ambiguity in the policies, DOD's ability to implement effective internal control over those policies is limited. Moreover, DOD recently added a requirement that program managers conduct a business-case analysis for systems' long-term technical-data needs. However, DOD has not issued policy or other internal controls that describe how to conduct this analysis. GAO has previously reported that the military services inconsistently completed similar business-case analyses because DOD had not issued instructions on how to conduct them. Without instructions that describe how to conduct the business-case analysis, senior acquisition decision makers may not receive the information they need to decide whether to approve programs at major milestones in the acquisition process.
section_title: What GAO Recommends
paragraphs: GAO recommends that DOD (1) update policies to clarify its technical-data documentation requirements and (2) instruct program managers on the elements to include and the information to report for technical-data business-case analyses. DOD concurred with GAO's recommendations.

section_title: Letter
paragraphs: The Department of Defense (DOD) needs access to technical data related  to its weapon systems in order to control costs and maintain flexibility in  the acquisition and sustainment of those weapon systems. Technical  data—recorded information used to produce, support, maintain, or  operate a system—can enable the government to complete maintenance  work in house, as well as to competitively award contracts for the  acquisition and sustainment of a weapon system. Because many systems  remain in DOD’s inventory for decades, decisions that officials make  during the acquisition process to acquire or not acquire rights to technical  data can have far-reaching implications for DOD’s ability to sustain the  systems and competitively procure parts and services. Weapon systems  are costly to sustain in part because they often incorporate technologically  complex subsystems and components and need expensive spare parts and  logistics support to meet required readiness levels. According to DOD, at  least 70 percent of a weapon system’s life-cycle costs are incurred to  operate and support a weapon system after it has been acquired, with the  percentage depending on how long a system remains in the inventory.
Since 2002, we have issued several reports that address technical data.  For example, we reported in 2006 that a number of fielded Army and Air  Force systems encountered limitations in sustainment options because the  military services lacked needed technical-data rights. In the 2006 report,  we recommended that DOD require program managers to assess long-term  technical-data needs and establish corresponding acquisition strategies  that provide for the technical-data rights needed to sustain weapon  systems over their life cycles. More recently, we reported in 2010 that the  government’s lack of access to proprietary technical data, among other  things, limits—or even precludes the possibility of—competition for DOD  weapons programs.
Congress has also highlighted the importance of technical data in the  defense acquisition process. For example, section 802 of the National  Defense Authorization Act for Fiscal Year 2007 includes a requirement that  the Secretary of Defense direct program managers for major weapon  systems to assess their systems’ long-term technical-data needs.  The act  also requires that the Secretary direct program managers to develop  corresponding acquisition strategies that provide for the technical-data  needs to sustain their systems throughout their life cycle. Similarly,  Congress passed the Weapon System Acquisition Reform Act of 2009,  which required in part that the Secretary of Defense is to ensure the  acquisition strategy for each major defense-acquisition program includes  measures to ensure competition, or the option of competition, in contracts  for the program throughout its life cycle. The act cited the acquisition of  complete technical-data packages as one option for promoting  competition.
In response to your request, this report addresses the extent to which (1)  DOD has updated its acquisition and procurement policies to reflect  certain technical-data-related provisions of the 2007 National Defense  Authorization Act and GAO’s 2006 recommendations aimed at assessments  of long-term technical-data needs; (2) selected defense acquisition  programs have adhered to the updated requirements in DOD policy to  document their systems’ long-term technical-data needs; and (3) DOD has  taken actions to improve decision making by program managers on the  long-term technical- data needs for systems in its acquisition process.
To evaluate the extent to which DOD updated its acquisition and  procurement policies to reflect certain technical-data-related provisions of  the 2007 National Defense Authorization Act and GAO’s 2006  recommendations, we analyzed the act and our prior recommendations.  We identified and evaluated changes the department made to its  acquisition and procurement policies since the 2007 act. We used  information from our evaluation of these policies in the analyses we  conducted for each of our objectives. We also reviewed follow-up records  maintained by DOD and GAO that document actions the department had  taken in response to audit recommendations. To evaluate the extent to  which selected defense-acquisition programs have adhered to the  requirements in DOD policy to document their system’s long-term  technical-data needs, we selected a non-generalizable sample of 12  programs out of about 50 programs subject to the requirements outlined in  DOD acquisition and procurement policies. Our sample included Army,  Navy, and Air Force programs in the two highest-value acquisition  categories that reached major milestones in the defense acquisition  process from September 2007 to August 2010. In addition, findings from  our sample are not generalizable to all DOD acquisition programs,  although the variety of circumstances that the programs in our sample  face illustrates important aspects of documenting a system’s long-term  technical-data needs. We obtained key acquisition documents—the  acquisition strategy and acquisition plan—from the programs in our  sample that reflect the technical-data needs of the system. Two team  members concurrently conducted independent analyses of the same  acquisition documents for each program comparing the documents against  relevant criteria from DOD policy. We then compared the two sets of  observations and reconciled any differences with the assistance of a third  analyst, when necessary. We provided our preliminary observations of the  documents to officials in each program and considered additional  information they provided when our observations indicated that the  program documents had not addressed one or more of the requirements.  To evaluate the extent to which DOD has taken actions to improve  technical-data decision making, we reviewed documentation including the  November 2010 Under Secretary of Defense for Acquisition, Technology  and Logistics memorandum, Implementation Directive for Better Buying  Power–Obtaining Greater Efficiency and Productivity in Defense  Spending, as well as guidelines on technical data that the Office of the  Under Secretary of Defense for Acquisition, Technology and Logistics,  which is part of the Office of the Secretary of Defense (OSD), and each of  the military departments recently issued. For each of our objectives, we  also interviewed officials in a variety of relevant organizations including  the OSD, the acquisition headquarters office in each of the military  departments, and the 12 acquisition programs in our sample. During these  interviews, we obtained perspectives from officials and documentation  such as acquisition strategies and acquisition plans from each of the  programs in our sample. We also assessed the reliability of all of the data  that we discuss in this report by reviewing relevant documentation and  interviewing knowledgeable officials. We found the data sufficiently  reliable for the purposes of this report.
We conducted this performance audit from May 2010 to May 2011 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives. We discuss our scope and methodology in  more detail in appendix I.
subsections: section_title: Background
paragraphs: DOD program managers obtain technical data and technical-data rights to  enable the department to acquire and sustain weapon systems at the  lowest cost, to provide flexibility in future acquisition and sustainment of  systems and subsystems, and to maintain those systems. DOD may obtain  different levels of rights to technical data including unlimited rights,  government-purpose rights, and limited rights. If DOD obtains unlimited  rights to technical data, it may provide the data to anyone for any reason.  However, if DOD obtains government-purpose rights, it may provide the  data to third-party contractors only for activities in which the U.S.  government is a party, including competitive reprocurement, but not  including commercial purposes. Further, if DOD obtains limited rights, it  may only use the data internally and may provide the data to third parties  in a limited number of circumstances (e.g., emergency repair and  overhaul.) Moreover, DOD and contractor maintenance personnel need  technical data and technical-data rights in order to maintain, repair, and  upgrade weapon systems throughout the life cycle of the systems.
subsections: section_title: The Process to Acquire Technical Data
paragraphs: The process that DOD program officials follow to acquire technical data  and technical-data rights for systems includes four general phases with  multiple steps in each phase. In this report, we evaluated aspects of the  first phase of this process (see fig. 1).
Progrm offici pecify  technicl d requirement  in olicittionssued to  contrctor. Contrctor’  proposa assert ny  retriction on DOD’ right  to technicl d needed to  prodce  tem. Progrm  offici review nd evuate  proposa, identify reas of  disagreement, nd my  chllenge contrctor’  assertion.
When contrctor prodce  the tem, they massert  ome dditionl retriction  to technicl d right,  which DOD my chllenge.  Contrctorrk ll d  they deliver to DOD with the  pproprite level of right,  nd DOD review nd  evuate thee mrk for  contency with DOD  policie nd greement in  the contrct.
DOD my relize if it has  quired the needed d  nd right when it susin  it tem. DOD us  nd right to mintin,  repir, nd olicit for  susinment contrct for it  tem. DOD my  chllenge d rightrking within 3 ye of  contrct completion. DOD  mo exercie option  for dditionl right nd  d tht it did not initilly  quire if thi option i  provided for in the contrct.
Requirements, strategies, and plans phase: Program officials assess  the long-term technical data and technical-data rights requirements for  their system and then document those requirements in an acquisition  strategy and an acquisition plan for their system. To assess a system’s  technical-data requirements, program officials determine which  components DOD will need technical data for and the level of rights to  seek for those data. Program officials consider several factors in their  assessment, such as the government’s cost for the rights to the data,  sustainment plans, re-procurement needs, and contractors’ economic  interest. Once program officials complete their assessment, they record  the technical-data requirements in a data-management strategy that is  included in the acquisition strategy, a document that is required by DOD  Instruction 5000.02. They also include similar documentation in the  acquisition plan, which is required by the DFARS. The acquisition strategy  describes the overall approach for managing and planning for the program,  while the acquisition plan describes the program’s contracting approach.  The program manager then submits these documents to senior department  officials to review and approve at certain major milestones in the defense  acquisition process.
Contracting phase: Program officials specify the approved technical-data  requirements in solicitations they issue to contractors. These solicitations  describe the capability requirements for a system that the government  intends to acquire. Contractors then submit proposals to DOD in which  they describe the system that they would build to provide the required  capability. In the proposals, contractors also discuss technical-data issues.  For example, if a contractor desires to assert restrictions on DOD’s ability  to use any of the technical data needed to manufacture or sustain the  system, the contractor asserts those restrictions in its proposal. Program  officials then review and evaluate the contractors’ proposals using criteria  included in the solicitation. Officials evaluate any asserted restrictions on  DOD’s use of technical data to identify areas of disagreement that the  department should resolve through negotiations or other procedures in  accordance with applicable law. DOD officials then award a contract.
Performance and delivery phase: During this phase, the selected  contractor begins producing the system and may assert additional  restrictions to technical-data rights in certain circumstances. For example,  the contractor may assert new restrictions if the department modifies its  system requirements or if the contractor inadvertently omitted a  restriction during the contracting phase. DOD officials may also challenge  these additional asserted restrictions. Contractors mark all technical data  they deliver to the government with a level of rights (e.g., government  purpose or limited rights). In addition, program officials review these  markings to ensure that the contractor has identified them in a manner  that is consistent with DOD policies and the agreement in the contract.
Post-performance and sustainment phase: In this phase, the  contractor has delivered a system to DOD. DOD officials may realize  during post-performance and sustainment whether they have acquired the  necessary technical data and technical-data rights during the sustainment  phase. When sustaining systems, DOD personnel may use technical data  for critical functions including maintaining and repairing systems. Any  new technical data and technical-data rights that would be needed for any  support contracts during sustainment phase would need to be acquired.  Program officials also may challenge the level of rights that the contractor  asserted for any delivered technical data that is used to produce the  system for up to 3 years after final payment under the contract or three  years after delivery of the data, whichever is later. Program officials may  also exercise options to obtain additional rights and data that the  department did not acquire during the performance and delivery phase if  DOD and the contractor had included a provision in the contract called a  “priced-contract option.”
section_title: Prior GAO and Defense Audit Agencies’ Work on Technical Data
paragraphs: For nearly a decade, we and the military-service audit agencies have  conducted reviews that included information on DOD’s acquisition of  technical data and technical-data rights for systems in the acquisition  process. In February 2002, we reported that DOD officials expressed  concern that they did not have affordable technical data to develop  additional or new sources of repair and maintenance to ensure a  competitive market. Subsequently, we reported in August 2004 that DOD  program managers often opt to spend limited acquisition dollars on  increased weapon system capability rather than on acquiring the rights to  the technical data—thus limiting their flexibility to perform maintenance  work in house or to support alternate source development should  contractual arrangements fail. We subsequently reported in July 2006 that  the Army and the Air Force encountered limitations in their sustainment  options for some fielded-weapon systems because they lacked technical- data rights. More recently, we reported in 2010 that the government’s  lack of access to proprietary technical data, among other things, limits—or  even precludes the possibility of—competition for DOD weapons  programs.
Additionally, the Air Force and Army audit agencies have reported on  issues related to the acquisition of technical data and technical-data rights.  For example, in May 2009, the Air Force Audit Agency reported that Air  Force program officials had not effectively implemented OSD and Air  Force initiatives to improve the management and acquisition of technical - data rights and had not satisfied technical-data assessment requirements.  Similarly, the Army Audit Agency reported in July 2009 that (1) Army  policies on technical-data assessments and documentation were not  incorporated into Army regulations and (2) the Army acquisition  workforce had not received training on assessing and managing technical  data and technical-data rights requirements and as a result did not  consistently address technical data and technical-data rights  requirements. We provide more detail in appendix II about the  recommendations in these audit agency reports and the services’  responses.
section_title: DOD Created Requirements to Document Long-Term Technical-Data Needs
paragraphs: DOD updated its acquisition and procurement policies, in a manner that  reflects a 2007 legislative provision and our 2006 recommendations, to  require that acquisition program managers document their long-term  technical-data needs. According to DOD officials, these policy updates do  not change the requirements program managers must follow that to decide  what technical data or technical-data rights to acquire for their systems.
Section 802 of the 2007 National Defense Authorization Act required the  Secretary of Defense to direct program managers for major weapon  systems—and subsystems of major weapon systems—to assess the long- term technical-data needs of their systems and establish strategies  providing for the technical-data rights needed to sustain the systems over  their life cycles. The 2007 act required, among other things, that the  strategies developed in accordance with the section address:  the merits of a priced contract option for the future delivery of  technical data that were not acquired upon initial contract award, and  the potential for changes in the sustainment plan over the life cycle of  the system.
We had previously recommended that DOD establish these requirements  for program managers in our July 2006 report. We recommended these  actions after finding that a lack of technical-data rights limited the  flexibilities of the Army and Air Force to make changes to sustainment  plans for some fielded weapon systems. We also found that delaying action  in acquiring technical-data rights can make these data cost-prohibitive or  difficult to obtain later in a weapon system’s life cycle.
DOD took a series of actions to change its acquisition and procurement  policies in a manner that reflects the language of the 2007 act and our 2006  recommendations. As a result of these actions, program managers are now  required to record their long-term technical-data needs in two key  acquisition program documents: the acquisition strategy and acquisition  plan. Initially, OSD issued a memorandum in July 2007 requiring program  managers for systems in the two highest-value acquisition categories  (ACAT I and II) to assess the long-term technical-data needs for their  systems and document a corresponding strategy for technical data in each  program’s acquisition strategy. DOD later included this policy change in  the December 2008 update of its acquisition policy, DOD Instruction  5000.02. In a separate action, DOD issued an interim rule in September  2007 amending the DFARS. This rule also requires program managers to  assess the long-term technical-data needs for their systems and document  a corresponding strategy in each program’s acquisition plan. DOD finalized  the interim rule in December 2009.  Together these policy changes  required that strategies and plans for major acquisition programs:  1.  assess the data required to design, manufacture, and sustain the  system as well as to support re-competition for production,  sustainment, or upgrade;  2.  address the merits of including a priced contract option for future  delivery of data not initially acquired;  3.  consider the contractor’s responsibility to verify any assertion of  restricted use and release of data; and  4.  address the potential for changes in the sustainment plan over the life  cycle of the weapon system or subsystem.
OSD officials told us that these policy updates do not change the  requirements program managers must follow to decide what technical  data or technical-data rights to acquire for their systems. They also told us  that the only new requirement was that program managers include  documentation of their system’s long-term technical-data needs in  acquisition strategies and acquisition plans. Moreover, OSD and each  military department have issued guides for program managers that  elaborate on the requirements in DOD policy assessing long-term  technical-data needs and the updated requirement to document those  needs in acquisition strategies and acquisition plans. We discuss these  guides in more detail later in this report.
section_title: Selected Programs Partially Addressed the Updated Requirements to Document Long-Term Technical-Data Needs
paragraphs: The documentation we reviewed for 12 acquisition programs partially  addressed the revised DOD policies on long-term technical-data needs. We  evaluated these programs’ acquisition strategies and acquisition plans  against four criteria identified in the revised technical-data policies  (described earlier in more detail). These policies require programs to  document (1) an assessment of technical-data requirements, (2) the merits  of a priced-contract option, (3) the contractor’s responsibility to verify  assertions of limited data rights, and (4) the potential for changes in the  system’s sustainment plan. We examined program acquisition strategies  for the first three requirements. We reviewed program acquisition plans  for the fourth requirement because the requirement was not included in  the revised acquisition policy that governs acquisition strategies but was  included in the procurement-policy update, which governs acquisition  plans.
As a part of our review, we did not consider the amount or level of quality  of the information that the acquisition strategies and acquisition plans  included in response to each requirement because DOD’s policies did not  specify the minimum levels or types of information that program officials  are required to include to satisfy each of the four requirements. Programs  in our sample included varying amounts of information in response to  each requirement they addressed. For example, one acquisition strategy  contained a 95-page appendix on technical-data management while  another contained three paragraphs focusing on technical data. If a  strategy or plan included any discussion of a requirement, we determined  that the strategy or plan addressed that requirement, regardless of the  level of detail.
Figure 2 summarizes the results of our analysis and shows that 10 of the 12  programs that we evaluated addressed at least one of the 4 requirements in  their documentation, and 4 addressed as many as 3 requirements.  However, none of the programs addressed all four of the requirements in  its documentation, and two did not address any of the requirements.
Assessments of technical-data requirements: Nine of the 12  acquisition strategies documented an assessment of the data required to  design, manufacture, and sustain the system as well as support re- competition for production, sustainment, or upgrade of the system, for  example:    The Integrated Air and Missile Defense strategy included an appendix  that, among other things, stated that the program office would require  delivery of sufficient data to completely describe and define the  functional and physical characteristics of the system for  manufacturing, and it also provided a list of required types of data.
The strategy for the Navy Multiband Terminal stated that the program  manager had “assessed the long-term technical-data needs” of the  system and “established acquisition strategies that provide for  technical data” and “associated license rights needed to sustain [the  systems] over their life cycle and allow for competitive procurement of  future terminals.”
The three strategies that did not address the requirement did not  identify any required data.
Merits of a priced-contract option: Four of the 12 acquisition strategies  discussed the merits of a priced contract option—an option to obtain  additional data and rights that the program did not acquire during the  contracting phase, for example:    The Small Diameter Bomb II strategy stated that the contract “will  contain a priced contract option…for a one-time delivery of a  technical-data package” that would consist of data “that describes the  design, support, test, and maintenance” of the system, and the models,  simulation and analysis used to predict its performance.
The strategy for the Joint High Speed Vessel stated that due to “the  non-developmental nature of the program, a priced   option…was not considered a cost-effective use of government funds.”
The eight other strategies did not discuss the merits of a priced  contract option for technical data.
Contractor’s responsibility to verify data assertions: Three of the 12  acquisition strategies referred to the contractor’s responsibility to verify  any assertion that the contractor made to restrict the government’s use  and release of any technical data. Each of the three strategies noted that  the program planned to include a clause in its contracts that identifies the  contractor’s responsibility to provide sufficient information to the  government’s contracting officers to enable them to evaluate the  contractor’s assertions. While nine strategies did not discuss the  contractor’s responsibility to verify assertions of restricted use and release  of technical data or mention the contract clause, a number of these  strategies discussed the program office’s efforts or responsibility to verify  contractor assertions of restricted use and release of data. For example,  the B-1 Bomber Radar Reliability and Maintainability strategy discussed  the program office’s efforts to verify the contractor’s assertion of  restricted use and release of data.
Potential for sustainment changes: Four acquisition plans addressed  the potential for changes in the system’s sustainment plan over its life  cycle, and the acquisition plans for two other programs were not subject  to this requirement, for example:    The Joint High Speed Vessel acquisition plan stated that the “potential  for changes in the sustainment plan is small.”
Two of the 12 programs in our sample were not subject to this  requirement. The requirement did not apply to the Joint Battle  Command-Platform and Navy Multiband Terminal because both  programs developed acquisition plans prior to the September 2007  procurement policy change on technical data and neither was required  to update its plan. Addressing the potential for changes in the system’s  sustainment plan over its life cycle is required for acquisition plans  developed or updated after DOD’s 2007 revision to its procurement  policy.
The six acquisition plans that did not address this requirement did not  discuss the potential for future changes in the sustainment plan as they  relate to technical-data needs.
Later in the report, we note that (1) a cause for the partially addressed  documentation is ambiguity in DOD’s revised policies and (2) this  ambiguity results in limits to department decision makers’ ability to  exercise effective internal control in their reviews of acquisition  documentation, which may result in delays in the acquisition process.  Because these issues are related to a similar ambiguity in another  technical-data policy, we provide a more detailed discussion of the causes  and effects for both types of problematic outcomes later in this report. In  the next section of our report, we describe OSD and military department  guides that discuss additional voluntary steps the program managers may  take for conducting and documenting assessments of long-term technical- data needs. These guides may result in acquisition documentation that is  more responsive to DOD’s revised policies. However, most of the guides  we describe were issued after most of the acquisition documentation we  reviewed was approved.
section_title: DOD Issued Guides to Improve Technical- Data Decision Making, but Technical-Data Policy Requirements Remain Unclear
subsections: section_title: DOD and Military Departments Issued Guides to Improve Program Managers’ Technical-Data-Related Decisions
paragraphs: OSD and each military department have issued several guides for program  managers that elaborate on the requirements in DOD policy for conducting  and documenting assessments of long-term technical-data needs. From  December 2009 through December 2010, DOD and the military  departments issued guides covering voluntary actions that program  managers might take to improve their decisions related to technical data.  While officials in DOD and the military departments told us that program  officials have found the various DOD-wide and military department- specific guides useful, program managers are not required to follow any of  the recommendations contained in the guides.
In December 2009, OSD updated the Web-based Defense Acquisition  Guidebook to elaborate on the new requirements for program managers to  document the long-term technical-data needs for their systems. The DOD- wide guidebook now includes topics that OSD recommends that program  managers discuss in their acquisition strategy documenting the system’s  long-term technical-data needs. For example, the guidebook recommends  that for data acquired to support competition, the program manager  document the (1) logic applied to select the technical data and technical- data rights, (2) alternative solutions considered, and (3) criteria used to  decide what, if any, data to procure.
Subsequent to the changes in the DOD-wide guidebook, the military  departments provided their own additional guides. The Air Force Program  Management and Acquisition Excellence Office in December 2010 issued  an update to a guide for program managers that includes recommended  steps to follow when determining a system’s long-term technical-data  needs and documenting those needs in a data-management strategy. For  example, the guide suggests that program managers consider whether Air  Force depot officials agree that the technical data and technical-data rights  that the program intends to acquire for the system are sufficient to enable  depot-level maintenance. Later, in October 2010, the Air Force’s Product  Data Acquisition Team launched a technical-data-focused Web site that  includes some of the same information contained in the earlier Air Force  guide and additional information. For example, the Web site asks program  managers if the technical-data rights that program managers intend to  acquire enable the Air Force to support competition for contracts for  spare parts, equipment to upgrade to a system, and logistics support.
The Army’s Product Data and Engineering Working Group in August 2010  published a 68-page guide that describes steps it recommends program  officials take to assess a system’s long-term technical-data needs and  document those needs in a data-management strategy. The Army’s guide  contains a work sheet that provides program managers with a systematic  approach to assess their technical-data needs. For example, for each  component of a system, the worksheet prompts program managers to  consider the (1) level of rights required, (2) expected levels of rights the  Army will acquire in negotiations with a manufacturer, (3) any gaps  between the requirements and expected negotiated outcomes, (4) plans to  close any gaps, and (5) risks associated with those plans.
The Navy in June 2010 published a set of guidelines that it recommends  program managers follow when they determine their systems’ technical- data rights. The Naval Open Architecture Enterprise Team included these  guidelines in an appendix to a contracting guidebook. Like the Air Force  and Army resources, the appendix lists questions that the team  recommends program managers consider when conducting a technical- data rights assessment. For example, the appendix asks whether the  government will obtain government-purpose rights at a minimum for a  system and asks for a justification for agreeing to more restrictive rights  than government purpose rights.
In addition to these department-level guides, some subordinate commands  within military departments have issued guidance on technical-data  assessments. For example, Air Force Materiel Command issued a  handbook on technical-data rights in May 2010, while the Air Force’s  Space and Missile Systems Center issued the third edition of a similar  guide in January 2011. By issuing their own guidance, these subordinate  commands are able to focus on issues of technical data particular to the  command in question.
section_title: Required Policies on Technical-Data Assessments Remain Unclear
paragraphs: While OSD and each of the military departments took actions to help  program managers prepare technical-data assessments, DOD has not  clarified ambiguities in the required technical-data policies to ensure their  full implementation. Specifically, DOD has not clarified how program  offices should address the requirement for documenting technical-data  assessments, and has not clarified a recent requirement to conduct a  business-case analysis on technical-data needs. Without internal controls  such as clear instructions on how to respond to these policies, DOD and  the military departments risk incomplete and inconsistent actions and  documentation in response to the technical-data requirements. According  to standards for internal control, implementing effective internal controls  is a key factor that helps organizations ensure that management’s  directives are carried out. Examples of internal control actions that  management can take include issuing policies or instructions that enforce  management’s directives.
subsections: section_title: Revised Technical-Data Policy Requirements Are Unclear
paragraphs: We found that the revisions to DOD’s acquisition and procurement  policies, which require acquisition program managers to document their  long-term technical-data needs, are unclear. For example, the revised DOD  Instruction 5000.02 requires program managers to document an  assessment of long-term technical-data requirements for their systems.  However, the policy does not clearly state the level of detail program  managers are required to document, or the extent to which they should  document their reasoning for acquiring or not acquiring technical data and  technical-data rights. Likewise, the DFARS requires programs to address  in program documentation the potential for changes in the sustainment  plan over the system’s life. However, the policy does not make clear what  information DOD expects to be provided in documentation of possible  future changes to a system’s sustainment plan (for example, underlying  assumptions), and how this information should relate to the technical-data  discussion. Our previously discussed evaluation of 12 acquisition  strategies and plans—most of which were approved before OSD and the  military departments issued their voluntary guides—showed that program  managers may not fully understand how to respond to these revised  policies. As we noted, we found that eight of the 12 acquisition strategies  and plans we reviewed addressed no more than two of the four  requirements (see fig. 2).
OSD had not issued an update to the DOD Instruction 5000.02 or the  DFARS as of April 2011 to clarify what programs specifically need to do to  address the assessments of technical data. OSD officials acknowledged to  us that the policies could be rewritten for greater clarity, and they pointed  out ambiguities in some of the requirements. For example, they told us  that the assessment of technical-data requirement is unclear. The officials  told us that if they had the opportunity, they would clarify the requirement  to state that program managers (1) assess the data that are needed to re- compete for production, sustainment, or upgrade, and (2) determine what,  if any, of that technical data the program requires.
Ambiguity in the revised policies results in limits to department decision  makers’ ability to exercise effective internal control in their reviews of  acquisition documentation. Without clear policies on documenting long- term technical-data needs, program managers may not understand how to  respond and, as a result, may continue to submit incomplete acquisition  documentation. Without complete documentation, senior level department  decision makers are limited in their ability to carry out their internal  control responsibilities to ensure that programs are aligned with  department policies and priorities. An August 2010 memorandum from the  Office of the Under Secretary of Defense for Acquisition, Technology and  Logistics called attention to this limitation stating that recent acquisition  strategies often did not include sufficient detail on topics including  technical-data requirements. The memorandum stated that future  acquisition strategies submitted that did not provide all of the required  information would be delayed. Delays in the acquisition process can, in  turn, hinder DOD’s ability to provide needed materiel to the warfighter.
section_title: OSD Requires a Business-Case Analysis for Technical-Data Decisions, but Has Not Issued Instructions on How to Conduct the Analysis
paragraphs: OSD recently added a requirement that program managers conduct a  business-case analysis as part of their assessment to determine the long- term technical-data needs for their systems; however, DOD has not issued  policy or other internal controls that describe how to conduct this  analysis. In November 2010, the Under Secretary of Defense for  Acquisition, Technology and Logistics issued a memorandum that requires  program officials to take a number of actions to improve efficiency and  productivity in defense spending. Among other things, the memorandum  requires program managers for all acquisition programs to (1) conduct a  business-case analysis that outlines the technical-data rights the  government will pursue to ensure competition and (2) include the results  of this analysis in acquisition strategies at Milestone B.
According to OSD officials, a business-case analysis would require  program managers to determine whether the benefits of acquiring  technical data are worth the costs of acquiring them. Prior to this  memorandum, a formal cost benefit analysis was not required for  technical-data decisions. As of January 2011, DOD officials told us that no  acquisition program had yet completed this analysis because no program  had reached Milestone B since the Under Secretary issued the  memorandum. Therefore, we could not evaluate an analysis conducted in  response to this new requirement.
Since establishing the requirement in its November 2010 memorandum,  OSD had not issued policy or other internal controls, as of April 2011, that  describe how to conduct the business-case analysis or what information to  report in the acquisition strategy. The Under Secretary of Defense for  Acquisition, Technology and Logistics stated in the memorandum that the  department would take additional actions in support of the memorandum.  However, OSD officials told us that they have not decided whether to issue  additional clarifying policy to instruct program managers on how to  conduct the analysis or what information about the results of the analysis  they should include in acquisition strategies.
section_title: Previous GAO Review Found Business-Case Analyses Were Inconsistently Completed
paragraphs: We have previously reported that the military services inconsistently  completed similar business-case analyses when DOD had not issued  instructions on how to conduct them. In 2008, we found that DOD had  not issued a policy instructing program managers on the elements to  include in the documentation of the analyses that program managers  conducted for decisions on performance based logistics arrangements—a  DOD approach to providing support to weapon systems. As a result,  program staff conducted business-case analyses that were inconsistent  and missing one or more elements recommended by a DOD instruction on  economic analyses. We found that DOD officials implemented the  performance based logistics arrangements for the sample of programs we  reviewed without the benefit of sound and consistent analyses. Among  other things, we recommended that DOD clearly define specific criteria for  these analyses in DOD policy. DOD partially agreed with our  recommendation. To address our recommendation, in April 2011, the  Principal Deputy Assistant Secretary of Defense for Logistics and Materiel  Readiness issued the Product Support Business-Case Analysis  Guidebook.
Because OSD has not issued policy instructing program managers on how  to conduct and document the analyses, program managers may conduct  incomplete or inconsistent analyses and report inconsistently on  important elements of the analyses and findings. Similar to the situations  we described in our 2008 report, program managers may not include key  required elements of business-case analyses, such as assumptions, feasible  alternatives, and costs and benefits that support their technical-data  decisions. In addition, because OSD has not issued policy instructing  program managers on how to report on the results of these analyses,  program managers may not provide the information that senior leaders in  DOD and the military departments need in order to decide whether to  approve the acquisition programs at major milestones in the acquisition  process. Technical-data decisions can be costly, with some prime  contractors quoting a price in excess of $1 billion for technical-data  packages. Thus, decision makers need sufficient details to conduct their  reviews and make fully informed decisions. The November 2010  memorandum demonstrates that this negative effect already exists for  technical-data-related requirements.
section_title: Conclusions
paragraphs: DOD has taken meaningful actions that could lead to an increased focus  on technical data in defense acquisition—actions that may help DOD  improve effectiveness and cost efficiency when acquiring and sustaining  its weapon systems. DOD has reflected congressionally mandated and  GAO-recommended changes in updated policies to emphasize the  importance of discussing and documenting assessments of technical data  and data rights in acquisition documentation, but program officials could  benefit from additional clarifications to these policies. If DOD does not  clarify the level and type of detail required in these updated policies,  program managers may continue to inconsistently include the needed  information. Furthermore, senior department officials may delay  approving these acquisition strategies at major milestone reviews. Delays  at major acquisition milestones could postpone the department’s effort to  provide needed materiel to the warfighter.
Moreover, DOD has required that program managers conduct a business- case analysis to weigh the costs of access to technical data for DOD’s  systems against the benefits of acquiring these data. This recently required  step may add rigor to decisions to acquire technical data that program  managers make early in the process. However, in the absence of DOD- wide instructions to program managers on how to conduct these analyses,  program officials may conduct analyses that exclude key elements and  therefore do not support optimal decision making for rights to technical  data that can cost $1 billion or more. Delaying issuing implementing  instructions to program managers for the business-case analysis could  slow DOD’s and the military departments’ efforts to answer the Under  Secretary of Defense’s call to take a more aggressive approach to finding  efficiencies and reducing DOD’s spending where possible in order to  better afford its future weapon systems.
section_title: Recommendations for Executive Action
paragraphs: To establish effective internal controls over technical-data policies that  improve DOD’s ability to efficiently and cost-effectively acquire and  sustain weapon systems over their life cycles, we recommend that the  Secretary of Defense direct the Under Secretary of Defense for  Acquisition, Technology and Logistics to take the following two actions:  Issue updates to the acquisition and procurement policies that clarify  requirements for documenting long-term technical-data requirements in  program acquisition strategies and acquisition plans. Among other  things, DOD should clarify the level and type of detail required for  acquiring technical data and technical-data rights that should be  included in acquisition strategies and acquisition plans  Issue instructions for program managers to use when conducting  business-case analyses that are part of the process for determining the  levels and types of technical data and technical-data rights needed to  sustain DOD’s systems. The instructions should identify the elements  to be included in the analyses and the types of information to be  documented in reports on the analyses.
section_title: Agency Comments
paragraphs: In written comments on a draft of this report, DOD concurred with our  two recommendations. The department’s written comments are reprinted  in their entirety in appendix III. DOD also provided technical comments  that we have incorporated into this report where applicable.
In response to our recommendation that DOD issue updates to the  acquisition and procurement policies that clarify requirements for  documenting long-term technical-data needs in program acquisition  strategies and acquisition plans, DOD stated that it planned to issue a  clarification this calendar year.
In response to our recommendation that DOD issue instructions for  program managers to use when conducting business-case analyses for  technical-data decisions, the department stated that it planned to issue  guidance this year related to this recommendation.
As we agreed with your office, unless you publicly announce its contents  earlier, we plan no further distribution of the report until 30 days from the  report’s date.  At that time, we will send copies of this report to the  Secretary of Defense; the Secretaries of the Army, the Navy, and the Air  Force; and the Under Secretary of Defense for Acquisition, Technology  and Logistics. In addition, the report will be available at no charge on the  GAO Web site at http://www.gao.gov.
section_title: Appendix I: Scope and Methodology
paragraphs: To evaluate the extent to which the Department of Defense (DOD)  updated its acquisition and procurement policies to reflect certain  technical-data-related provisions of the National Defense Authorization  Act for Fiscal Year 2007 and GAO’s 2006 recommendations, we reviewed  the law, our recommendations, and a variety of documents related to the  context of the act and recommendations. We reviewed DOD and military  department regulations governing technical-data acquisition and technical- data-related reports issued by GAO and DOD. We compared changes that  the department made to its acquisition and procurement policies to  respond to the law and our recommendations. Specifically, we analyzed  the following policies: (1) a memorandum issued by the Office of the  Under Secretary of Defense for Acquisition, Technology and Logistics  (OSD), Data Management and Technical Data Rights (July 19, 2007); (2)  DOD Instruction 5000.02, Operation of the Defense Acquisition System  enclosure 12(9) (Dec. 8, 2008); and (3) the Defense Federal Acquisition  Regulation Supplement (DFARS) 207.106 (S-70). We also used information  from this evaluation of the policies and their requirements in the analyses  we conducted for our other objectives. To obtain DOD’s perspective on  changes to these policies as well as information for all three of our  objectives, we interviewed officials in a variety of organizations including  OSD, acquisition headquarters for each military department, selected  program executive offices, and the acquisition programs in our sample.  Table 1 lists the organizations we contacted to conduct interviews and  obtain documents related to the acquisition of technical data. We also  reviewed information in databases in the DOD Office of the Inspector  General and GAO that record actions DOD took to implement our  recommendations. To evaluate actions DOD and the military departments  took to implement additional legislative provisions and audit  recommendations related to technical data (that we describe in app. II),  we evaluated the requirements in the relevant legislation or the actions  called for in the relevant recommendations. We then obtained and  analyzed key documentation, such as updates DOD made to the DFARS to  implement section 202 of the Weapon Systems Acquisition Reform Act of  2009.
To evaluate the extent to which selected defense acquisition programs  adhered to the updated requirements in DOD policy to document their  systems’ long-term technical-data needs, we selected a non-generalizable  sample of 12 acquisition programs from a population of about 50  programs. To draw the sample, we asked the three military departments to  identify all acquisition programs at the two highest-value acquisition  categories (ACAT I or II) that had reached the first three acquisition  milestones—A - Material Solution Analysis, B - Technology Development,  and C - Engineering and Manufacturing Development—between  September 2007 and August 2010. We chose September 2007 because this  was the first point at which both sets of requirements for documenting  long-term technical-data needs were in effect, and we chose August 2010  because we selected our sample at that time. Because too few Marine  Corps programs reached one of these milestones during this period, we  excluded this service from our evaluation. To draw the sample, we  selected four programs from each department, balancing the ACAT levels  and milestones. We then selected those programs that had most recently,  at the time we drew our sample, reached their respective milestones. We  used this approach because DOD updated its policies in 2007, and we  wanted to allow as much time as possible for the military departments to  develop methods to respond to the requirements in DOD’s updated  policies. Although findings from this sample are not generalizable to all  DOD acquisition programs, the variety of circumstances that programs in  our sample face can illustrate important aspects of documenting a  system’s long-term technical-data needs. Our sample includes a variety of  acquisition programs, including new systems (e.g., Joint High Speed  Vessel), modifications to existing systems (e.g., C-130 Avionics  Modernization Program), and systems that were primarily software (e.g.,  Joint Battle Command-Platform). After completing our sample selection,  we analyzed the content of each program’s acquisition strategy and  acquisition plan, which are required to document the program’s long-term  technical-data needs. To conduct these analyses, we compared each  program’s acquisition strategy and acquisition plan against certain criteria  from the July 2007 memorandum from the Under Secretary of Defense for  Acquisition, Technology and Logistics, DOD Instruction 5000.02, and the  DFARS. We could not compare the acquisition strategies and plans in our  sample to the voluntary guides that OSD and the military departments  issued in 2009 and 2010 because the guides were issued after the majority  of programs in our sample had completed their acquisition milestone  documentation. Two team members concurrently conducted independent  analyses of the same documentation. We then compared the two sets of  observations and reconciled any differences with the assistance of a third  analyst, when necessary. We also provided our preliminary observations of  each strategy to officials in each program and considered additional  information they provided when our observations indicated that the  program had not addressed one or more of the requirements.
To evaluate the extent to which DOD has taken actions to improve  decision making by program managers on the long-term technical-data  needs for systems in the acquisition process, we identified recent steps the  department has taken. We interviewed officials in a variety of offices  including OSD and the acquisition headquarters offices for each military  department. We interviewed the officials responsible for implementing any  steps DOD took, and we obtained and evaluated supporting  documentation (e.g., the Defense Acquisition Guidebook and guides issued  by each military department). The officials we interviewed represent  organizations such as the Army’s Product Data and Engineering Working  Group and the Air Force’s Product Data Acquisition Team.
For each objective, we assessed the reliability of the data we analyzed by  reviewing existing documentation related to the data sources and  interviewing knowledgeable agency officials about the data that we used.  We found the data sufficiently reliable for the purposes of this report.
We conducted this performance audit from May 2010 to May 2011 in  accordance with generally accepted government auditing standards. Those  standards require that we plan and perform the audit to obtain sufficient,  appropriate evidence to provide a reasonable basis for our findings and  conclusions based on our audit objectives. We believe that the evidence  obtained provides a reasonable basis for our findings and conclusions  based on our audit objectives.
section_title: Appendix II: DOD’s Implementation of Technical- Data Requirements Cited in Recent Legislation and DOD Audit Agencies’ Report Recommendations
paragraphs: The information in this appendix supplements the information we  provided elsewhere in this report. The legal requirements, audit  recommendations, and the Department of Defense (DOD) and military  department implementation actions in this appendix are more narrowly  focused than those reviewed earlier in this report. Together, the two sets  of mandated actions, recommendations, and response actions provide  additional information about technical-data-related requirements.
To evaluate the actions DOD has taken to implement the technical-data  requirements of the Duncan Hunter National Defense Authorization Act  for Fiscal Year 2009 and report recommendations by the Army and Air  Force Audit Agencies, we identified and evaluated the requirements and  recommendations. We then interviewed relevant DOD and Air Force and  Army officials and obtained key documentation such as updates DOD  made to the Defense Federal Acquisition Regulation Supplement to  implement section 202 of the Weapon Systems Acquisition Reform Act of  2009.
subsections: section_title: Technical-Data Requirements in the Duncan Hunter National Defense Authorization Act for Fiscal Year 2009 and DOD’s Response
paragraphs: Section 822 of the Duncan Hunter National Defense Authorization Act for  Fiscal Year 2009 requires DOD to take two technical-data-related actions  in the acquisition process. Table 2 lists these two legislative requirements  and DOD’s response to each.
section_title: Recommendations on Technical Data by Army and Air Force Audit Agencies’ Reports, and the Services’ Responses
paragraphs: The Army Audit Agency and the Air Force Audit Agency recently issued  reports that address the acquisition of technical data. Table 3 lists these  reports, their recommendations, and the military departments’ responses  to the recommendations.
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Staff Acknowledgments
paragraphs: Key contributors to this report were Carleen Bennett, Assistant Director;  Larry Bridges; Simon Hirschfeld; Amber Keyser; James P. Klein; Katherine  Lenane; Richard Powelson; Michael Silver; and Ryan Starks.
section_title: Related GAO Products
paragraphs: Federal Contracting: Opportunities Exist to Increase Competition and  Assess Reasons When Only One Offer Is Received. GAO-10-833.  Washington, D.C.: July 26, 2010.
Defense Management: DOD Needs Better Information and Guidance to  More Effectively Manage and Reduce Operating and Support Costs of  Major Weapon Systems. GAO-10-717. Washington, D.C.: July 20, 2010.
Defense Logistics: Improved Analysis and Cost Data Needed to Evaluate  the Cost-effectiveness of Performance Based Logistics. GAO-09-41.  Washington, D.C.: December 19, 2008.
Weapons Acquisition: DOD Should Strengthen Policies for Assessing  Technical Data Needs to Support Weapon Systems. GAO-06-839.  Washington, D.C.: July 14, 2006.
Defense Management: Opportunities to Enhance the Implementation of  Performance-Based Logistics. GAO-04-715. Washington, D.C.: August 16,  2004.
Defense Logistics: Opportunities to Improve the Army’s and the Navy’s  Decision-making Process for Weapons Systems Support. GAO-02-306.  Washington, D.C.: February 28, 2002.