Defense Acquisitions: Goals and Associated Metrics Needed to Assess Progress in Improving Service Acquisition
Released: 2013-06-27T00:00:00


Highlights: section_title: Why GAO Did This Study
paragraphs: In fiscal year 2012, DOD obligated more than $186 billion for contracted services, making it the federal government’s largest buyer of services. GAO’s prior work found that DOD’s use of contracted services has been the result of thousands of individual decisions, not strategic planning across the department.

Over the years, Congress has legislated a number of requirements to improve DOD’s service acquisitions. For example, Congress required DOD to implement a service acquisition management structure, approval process, and policies. Congress also directed DOD to develop a plan to implement the Defense Science Board’s recommendations for improving service acquisition.

The National Defense Authorization Act for Fiscal Year 2012 mandated that GAO report on DOD’s actions to improve service acquisition and management. GAO examined (1) the actions DOD has taken to respond to legislative requirements and (2) how DOD determines the effects of its actions to improve service acquisition. GAO reviewed documentation and interviewed DOD officials on the actions taken in response to the legislative requirements. GAO also assessed whether DOD addressed key factors, including establishing goals and metrics, to help it determine if it has improved service acquisition.

section_title: What GAO Found
paragraphs: Over the last decade, the Department of Defense (DOD) has taken several actions to address legislative requirements to improve the acquisition and management of services. In 2001, as amended in 2006, Congress required DOD to implement a management structure for the acquisition of services. In response, DOD implemented such a structure and service acquisition review and approval process. Recently, DOD also established new positions within its management structure, including senior managers within the office of the Under Secretary of Defense for Acquisition, Technology, and Logistics (USD(AT&L)) and the military departments, to oversee and coordinate service acquisition. With a management structure and review process in place, USD(AT&L) is focusing on efforts to improve the process for how requirements for individual service acquisitions are developed and enhancing training to respond to several legislative directives. USD(AT&L) also created its Acquisition of Services Functional Integrated Product Team, in part, to determine how to address legislative requirements to provide training for personnel acquiring services. USD(AT&L) did not develop a plan to implement the Defense Science Board recommendations to improve service acquisition but identified 23 different actions, including its Better Buying Power Initiative, it has planned or taken that officials regard as addressing what the plan was to include. For example, USD(AT&L) is updating its guidance on using incentives to improve contractor performance, which addresses one of the elements that was to be in the plan.

While DOD has taken a number of actions that address legislative requirements, DOD is not yet positioned to determine what effects these actions have had on improving service acquisition. Specifically, USD(AT&L) has not identified specific goals and associated metrics that would enable it to assess progress toward achieving those goals. USD(AT&L) has identified improving service acquisition as a priority but has not defined a desired end state for its actions or the measurable characteristics that would embody achieving such a goal. It is challenged in defining a desired end state for its actions, in part, because it has not determined the current status of service acquisition in terms of the volume, type, location, and trends. DOD is taking steps to improve its contract and financial systems to obtain such data, but these efforts will not be complete until at least 2014. Further, DOD has not established departmentwide metrics to assess its progress in improving service acquisition but has acknowledged the need to do so, which officials described as challenging. Nevertheless, despite the challenges in doing so, it is not impossible. For example, DOD has agreed to set goals for the amount of spending managed through strategically sourced acquisitions, link strategic sourcing to its Better Buying Power Initiative, and establish metrics, such as utilization rates, to track progress toward these goals. However, DOD is not fully leveraging the command-level assessments, feedback from the military departments, and other ongoing efforts it relies on to gauge the effects of its actions to improve service acquisition. By using its budget and spending data and leveraging these efforts, DOD could develop baseline data and identify trends over time, enabling it to develop measurable goals and gain more insight into whether its actions are improving service acquisition. Until then, DOD will continue to be in a position where it does not know whether its actions are sufficient to achieve desired outcomes.

section_title: What GAO Recommends
paragraphs: GAO recommends that DOD establish baseline data, specific goals for improving service acquisition, and associated metrics to assess its progress. DOD concurred with the three recommendations.

section_title: Letter
paragraphs: Since fiscal year 2000, the Department of Defense’s (DOD) annual  obligations for contracted services have more than doubled when  adjusted for inflation. In fiscal year 2012, DOD reported $186 billion in  obligations for contracted services, making the department the federal  government’s largest buyer of contracted services. DOD buys a wide  range of services, including consulting, administrative, medical, and  information technology, to support its missions. Within DOD, responsibility  for acquiring these services is spread among buying activities within  individual military commands, weapon system program offices, and  functional units on military bases. We previously found the increased use  of contracted services has been the result of thousands of individual  decisions, not strategic planning across the department.

Congress has passed a number of measures over the years to improve  DOD’s acquisition and management of contracted services. In 2001,  Congress required the Secretary of Defense to establish a management  structure for the acquisition of services, under section 2330, title 10,  United States Code. As part of the management structure, the Secretary  was to designate officials within the military departments and defense  agencies to be responsible for managing service acquisition within their  respective departments or agencies. The Secretary was also directed to  establish a process for approving individual service acquisitions in  advance of contract award. Congress subsequently amended 10 U.S.C.  § 2330 in 2006 to require, among other things, that the Under Secretary  of Defense for Acquisition, Technology, and Logistics (USD(AT&L))  develop and maintain policies, procedures, and best practices guidelines  for acquisition planning, requirements development, and other aspects  involved with the procurement of contracted services.

More recently, in section 802 of the National Defense Authorization Act  (NDAA) for Fiscal Year 2010, Congress required USD(AT&L) to have the  Defense Science Board (DSB) independently assess improvements to  DOD’s acquisition and oversight of services. The resulting March 2011  DSB report contained multiple recommendations to improve DOD’s  approach to contracting for services. The DSB’s recommendations  focused on creating new policies and processes, strengthening  management and oversight, designating roles and leadership  responsibilities, and improving the skills and capabilities of personnel  involved in services contracting. Subsequently, as part of section 807 of  the NDAA for Fiscal Year 2012, Congress directed USD(AT&L) to  develop a plan for implementing the recommendations of the DSB to  include, to the extent USD(AT&L) deemed appropriate, eight elements,  most of which align with the DSB’s recommendations.elements include incentives for high contractor performance, guidance on  the use of appropriate contract types, and training of services acquisition  personnel.

Section 807 of the NDAA for Fiscal Year 2012 also mandated that we  report on DOD’s actions to improve service acquisition and  management. This report addresses (1) DOD’s actions to address legislative requirements in 10 U.S.C. § 2330 and section 807 and (2) the  extent to which DOD is able to determine whether its actions have  resulted in improvements to service acquisition and management.

To describe how DOD has addressed legislative requirements, we  reviewed policies, guidance, memorandums, and additional information  documenting the actions that USD(AT&L) and the Departments of the Air  Force, Army, and Navy identified as taking in response to 10 U.S.C.  § 2330 and section 807 of the NDAA for Fiscal Year 2012. We  interviewed USD(AT&L) and military department officials regarding how  DOD is implementing these actions. We also met with Defense  Acquisition University (DAU) officials to obtain information on training,  guidance, and other efforts to address legislative requirements for service  acquisition training or career development.

To evaluate the extent to which DOD is able to determine whether its  actions have resulted in improvements to service acquisition and  management, we interviewed USD(AT&L) and senior military department  officials responsible for overseeing service acquisition and management.  We reviewed supporting documentation on the approaches they have  used to gain insight into the effectiveness of their actions, including  USD(AT&L) and the military departments’ reviews of individual service  acquisitions. We then assessed whether these approaches reflected key  factors needed to improve service acquisition based on findings in our  November 2006 report to determine if improvement efforts are achieving  their intended results. In the 2006 report, we identified several key  factors, which we consider to still be relevant, that leading commercial  firms reported must be in place to significantly improve service  acquisition. These factors include (1) leadership commitment, (2) a  desired end state with goals for the future, and (3) metrics that define  specified outcomes. We also found that critical to establishing a desired  end state with goals for the future is the ability to determine where service  acquisition is today in terms of specific and aggregate knowledge on the  current volume, type, location, and trends.

We conducted this performance audit from September 2012 to June 2013  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Our prior work has found that DOD’s approach to managing service  acquisition has tended to be reactive and has not fully addressed key  factors for success at either the strategic or transactional level. The  strategic level is where the enterprise sets the direction or vision for what  it needs, captures knowledge to enable more informed management  decisions, ensures enterprisewide goals and objectives are achieved,  determines how to go about meeting those needs, and assesses the  resources it has to achieve desired outcomes. The strategic level also  sets the context for the transactional level, where the focus is on making  sound decisions on individual acquisitions.

Congress has required USD(AT&L) to take a number of steps to improve  service acquisition. Specifically in 10 U.S.C. § 2330, enacted in 2001 and  amended in 2006, Congress required USD(AT&L) and the military  departments to establish a management structure for the acquisition of  services. Since 2003, we have evaluated DOD’s implementation of  10 U.S.C. § 2330 and efforts to establish the management structure and  service acquisition approval process twice. First, in September 2003, we  concluded that DOD’s approach to managing service acquisition did not  provide a departmentwide assessment of how spending for services  could be more effective. We therefore recommended that DOD give  greater attention to promoting a strategic orientation by setting  performance goals for improvements and ensuring accountability for  results. DOD concurred in principle with our recommendation and agreed  that additional actions could strengthen the management structure and  acquisition approval process but also identified challenges for doing so  based on its organizational size, complexity, and the acquisition  environment.

Subsequently, in November 2006, we found continued weaknesses  associated with DOD’s management of service acquisitions at the  strategic and transactional level. Specifically, we found that DOD’s approach to managing service acquisition tended to be reactive and that  the department had not developed a means for evaluating whether  ongoing and planned efforts were achieving intended results. DOD had  not developed a strategic vision and lacked sustained commitment to  managing service acquisition risks and fostering more efficient outcomes.  DOD also had not developed metrics to assess whether any changes to  improve service acquisition actually achieved the expected outcomes. As  a result, DOD was not in a position to determine whether investments in  services were achieving their desired outcomes. Moreover, the results of  individual acquisitions were generally not used to inform or adjust the  strategic direction. We recommended that, among other actions, DOD  take steps to understand how and where service acquisition dollars are  currently and will be spent, in part, to assist in adopting a proactive  approach to managing service acquisition. We also recommended that  DOD take steps to provide a capability to determine whether service  acquisitions are meeting cost, schedule, and performance objectives. At  that time, DOD concurred with our recommendations. USD(AT&L),  however, acknowledged in 2010 that DOD still needed a cohesive,  integrated strategy for acquiring services. DOD contract management has  remained on our High Risk List, in part, because DOD has not developed  such a strategy and continues to lack reliable services spending data to  inform decision making.

While Congress has required USD(AT&L) to take steps to improve  service acquisition, USD(AT&L) has taken actions on its own initiative as  well. For example, USD(AT&L) established its Better Buying Power  Initiative in a September 2010 memorandum to provide guidance for  obtaining greater efficiency and productivity in defense spending. In its  memorandum, USD(AT&L) emphasized that DOD must prepare to  continue supporting the warfighter through the acquisition of products and  services in potentially fiscally constrained times. In its own words,  USD(AT&L) noted that DOD must “do more without more.” USD(AT&L)  organized the Better Buying Power Initiative around five major areas,  including an area focused on improving tradecraft in service acquisition.  This area identified actions to improve service acquisition, such as  categorizing acquisitions by portfolio groups and assigning new managers  to coordinate these groups. USD(AT&L) issued another memorandum in  April 2013 to update the Better Buying Power Initiative. This  memorandum identifies seven areas USD(AT&L) is pursuing to increase  efficiency and productivity in defense spending. One area is to improve  service acquisition and the memorandum identifies a number of related  actions, such as increasing small business participation in service  acquisitions and improving how DOD conducts services-related market  research.

section_title: DOD Has Taken Actions to Address Legislative Requirements to Improve Service Acquisition
paragraphs: Over the last decade, DOD has taken actions to address legislative  requirements to improve the acquisition and management of services.  Senior officials we spoke with across the military departments credit  USD(AT&L)’s leadership and commitment as the driving force behind  many of the actions taken to improve service acquisition. A number of  these actions were intended to strengthen DOD’s management structure  and approach to reviewing service acquisitions, as required by 10 U.S.C.  § 2330. For example, both USD(AT&L) and the military departments  established new senior management positions to improve oversight and  coordination of service acquisition. With this management structure and  review process in place, USD(AT&L) is focusing on efforts to improve the  process for how requirements for individual service acquisitions are  developed and training to respond to legislative direction. USD(AT&L)  also created a senior-level team to identify and determine the training  needs for DOD personnel responsible for developing service acquisition  requirements. USD(AT&L) did not develop a specific implementation plan  as required by section 807, but officials identified a number of actions that  they regard as addressing the eight elements specified.
subsections: section_title: DOD Has Strengthened Its Management Structure and Service Acquisition Review Process
paragraphs: Since 2002, DOD has increased its management attention on high dollar  value service acquisitions by instituting new policies and review  processes. In response to the initial requirements to establish a  management structure for the acquisition of services, USD(AT&L) issued  a guidance memorandum in May 2002. This memorandum required that  service acquisitions be reviewed and approved based on dollar  thresholds and that the acquisition strategy—addressing things such as  the requirements to be satisfied and any potential risks—be approved  prior to initiating any action to commit the government to the strategy.  Under this policy, USD(AT&L) was responsible for reviewing and  approving all proposed service acquisitions with an estimated value of  $2 billion or more. Following the 2006 amendment to 10 U.S.C. § 2330,  USD(AT&L) issued a revised memorandum in October of that year. Under  the revised policy, which remains in effect, USD(AT&L) lowered the  threshold for its review to service acquisitions valued at over $1 billion.  The military departments have developed internal policies for reviewing  and approving service acquisitions below USD(AT&L)’s threshold.  Further, USD(AT&L) required that acquisition strategies be reviewed  before contract award and that these and other acquisition planning  documents include a top-level discussion of the source selection process  as well as noting any waivers and deviations. USD(AT&L) and military  department officials informed us that while these reviews are conducted,  they have not tracked the total number of service acquisitions reviewed to  date.

In 2008, USD(AT&L) incorporated these requirements into DOD  Instruction 5000.02, which is part of DOD’s overarching policy governing  the operation of the defense acquisition system. This instruction  currently requires that senior officials across DOD consider a number of  factors when reviewing a service acquisition, including  the source of the requirement, the previous approach to satisfying the requirement, the total cost of the acquisition, the competition strategy, and the source selection planning.

USD(AT&L) expects to issue a stand-alone instruction in 2014 for service  acquisition policy to replace Enclosure 9 of DOD Instruction 5000.02.  Additionally, in a February 2009 memorandum, USD(AT&L) refined its  guidance on conducting service acquisition strategy reviews. Specifically,  USD(AT&L)’s memorandum identified criteria that service acquisitions  must adhere to and that reviewers are to assess, such as use of  appropriate contract type, maximization of competition, and inclusion of  objective criteria to measure contractor performance.

DOD also established new senior-level management positions, in part, to  address legislative requirements, although some roles and responsibilities  are still being defined. For example, the 2006 amendment to 10 U.S.C.  § 2330 required that USD(AT&L) and the military departments establish  commodity managers to coordinate procurement of key categories of  services. In 2010 and 2012, USD(AT&L) revised how it organized its  contracted services under nine key categories. These categories of  services, referred to as portfolio groups, are (1) research and  development, (2) knowledge based, (3) logistics management,  (4) electronic and communication, (5) equipment related, (6) medical,  (7) facility related, (8) construction, and (9) transportation. In 2011, the  military departments began establishing commodity manager positions to  improve coordination and assist requiring activities with their procurement  of services within these portfolio groups. By July 1, 2013, USD(AT&L)  expects to establish similar positions responsible for supporting the DOD- wide procurement of services, but their authorities and responsibilities are  not yet fully defined. Additionally, as part of its Better Buying Power  Initiative, USD(AT&L) assigned the Principal Deputy Under Secretary of  Defense for Acquisition, Technology, and Logistics as DOD’s senior  manager for service acquisition, responsible for policy, training, and  oversight across DOD. Table 1 summarizes the established positions and  accompanying responsibilities in descending order of their hierarchy  within DOD.

While these positions have a role in reviewing, approving, or coordinating  individual service acquisitions, senior USD(AT&L) and military department  officials explained that they do not have responsibility or authority for  making departmentwide decisions, such as determining current or future  resources allocated to contracted services. These officials explained that  the military departments’ commands and requiring activities are  responsible for determining their requirements and how best to meet  them, as well as requesting and allocating budgetary resources. For  example, while USD(AT&L) officials and the military department senior  services managers are responsible for reviewing service acquisitions to  determine whether the planned acquisition strategy clearly defines the  military department’s requirement, they do not determine what contracted  services are needed or whether an alternative acquisition approach could  better meet their need. USD(AT&L) officials and the military department  senior services managers stated they do not have insight into each  requiring activity’s specific needs and are not positioned to validate those  needs.

For additional details on the actions that USD(AT&L) and the military  departments have taken to address the specific requirements of  10 U.S.C. § 2330, see appendix I.

section_title: DOD Continues to Focus on Improving the Requirements Development Process and Training for Individual Service Acquisitions
paragraphs: USD(AT&L) has planned and implemented actions to improve DOD’s  process for developing requirements for individual service acquisitions, as  required by the 2006 amendment to 10 U.S.C. § 2330. USD(AT&L)  officials noted that it has collaborated with DAU officials to develop new  tools and training to help DOD personnel develop better acquisitions. For  example,

USD(AT&L) collaborated with DAU to create the Acquisition  Requirements Roadmap Tool (ARRT) in 2012. The ARRT is an online  resource designed to help personnel write performance-based  requirements and create several pre-award documents, including  performance work statements and quality assurance surveillance  plans. The ARRT guides users through a series of questions to  develop the pre-award documents using a standardized template  tailored to the specific requirement for services. Although using the  ARRT is not required across DOD, DAU officials told us they have  integrated its use into other DAU training, such as the Performance  Requirements for Service Acquisitions course. DAU officials did not  have data on the effectiveness of the ARRT but noted that feedback  has been positive. For example, they have heard that performance  work statements are better reflecting requirements as a result of  personnel using the tool.

In 2009, DAU introduced its Services Acquisition Workshop (SAW) to  provide training and guidance on developing service acquisition  requirements. The SAW is a 4-day workshop tailored to proposed  service acquisitions. Upon request from commands or requiring  activities, DAU officials travel to the requestor and convene the  multifunctional team responsible for an acquisition, including general  counsel, individuals associated with the acquisition requirements,  contracting personnel, and oversight personnel. This team is then to  develop the language that will be used to articulate the service  requirement using the ARRT. By the end of the 4 days, the command  is to have drafts of its performance work statement, quality assurance  surveillance plan, and performance requirement summary. A key  aspect of the workshop DAU officials identified is that it brings  together the key personnel responsible for the acquisition to discuss  the service requirements and how they will know if a contractor has  met those requirements. From fiscal years 2009 through 2012, DAU  conducted 78 SAWs. In 2012, USD(AT&L) mandated use of the SAW  for service acquisitions valued at $1 billion and above and is  encouraging its use for acquisitions valued at $100 million or more.  USD(AT&L) has directed the Director of Defense Procurement and  Acquisition Policy (DPAP) and the senior services managers to  assess the effectiveness of the SAW and develop lessons learned  and best practices by October 1, 2013.

In addition to implementing the ARRT and the SAW, USD(AT&L)  established the Acquisition of Services Functional Integrated Product  Team (Services FIPT) in August 2012, in part, to address training  requirements in 10 U.S.C. § 2330. According to its charter, the Services  FIPT is comprised of the Director of DPAP, DAU officials, and other  officials responsible for acquisition career management within the DOD.  The Services FIPT is to provide input toward the development and  dissemination of training products and practical tools to assist personnel  responsible for acquiring services. In addition, the Services FIPT is to  explore the feasibility of certification standards and career development  for all personnel who acquire services, including personnel within and  outside of the defense acquisition workforce. USD(AT&L) officials  explained that non-acquisition personnel are most often involved in the  requirements development portion of the acquisition process but may not  be trained on how DOD buys services. In 2011, we found that non- acquisition personnel with acquisition-related responsibilities represented  more than half of the 430 personnel involved in the 29 services contracts  we reviewed. While we found that non-acquisition personnel received  some acquisition training, this training was largely related to contract  oversight as opposed to requirements development. According to its  charter, one of the Services FIPT’s first tasks will be to identify DOD’s  non-acquisition personnel involved in service acquisitions and determine  how best to train them.

The Services FIPT, however, has made little progress to date, and has  met once since it was established. USD(AT&L) officials could not provide  a time line for when the Services FIPT may fully address the training  requirements in 10 U.S.C. § 2330. The officials explained that they expect  the team to make more progress in 2013 when the Principal Deputy  Under Secretary for Acquisition, Technology, and Logistics assumes  leadership of the Services FIPT.

section_title: USD(AT&L) Did Not Develop a Plan to Meet the Requirement of Section 807 but Has Taken Actions to Address Each Element in the Law
paragraphs: Section 807 of the NDAA for Fiscal Year 2012 required USD(AT&L) to  develop a plan by June 28, 2012, for implementing the recommendations  of the DSB to include, to the extent USD(AT&L) deemed appropriate, the  following eight elements:  1.  incentives to services contractors for high performance at low cost,  2.  communication between the government and the services contracting  industry while developing requirements for services contracts,  3.  guidance for defense acquisition personnel on the use of appropriate  4.  formal certification and training requirements for services acquisition  5.  recruiting and training of services acquisition personnel,  6.  policies and guidance on career development for services acquisition  7.  ensuring the military departments dedicate portfolio-specific  8.  ensuring DOD conducts realistic exercises and training that account  for services contracting during contingency operations.

USD(AT&L) officials told us they did not develop a specific plan to  address the section 807 requirement. They explained, however, that the  April 2013 Better Buying Power Initiative memorandum addresses seven  of the eight elements and that they have addressed the last element  through a separate effort. In reviewing the April 2013 memorandum, we  also found that it reflects actions to address all of the elements except the  one pertaining to training and exercises during contingency operations.  USD(AT&L) also identified 23 different actions it has taken or plans to  take that officials regard as addressing all of the elements the plan was to  include, some of which pre-date the April 2013 Better Buying Power  Initiative memorandum. For example,  In January 2012, USD(AT&L) issued guidance to improve how  DOD communicates with the vendor community.

In April 2013, USD(AT&L) directed that new guidance be  developed to help acquisition personnel select the appropriate  contract type and contractor performance incentives in DOD’s  service acquisitions.

DOD plans to conduct a joint mission rehearsal exercise in 2014  that will include training for services contracting during  contingency operations.

See appendix II for a more detailed description of the actions USD(AT&L)  took to address the section 807 elements.

section_title: DOD Has Not Fully Addressed Key Factors to Determine Whether Actions Are Improving Service Acquisition
paragraphs: While DOD has taken a number of actions to address legislative  requirements, DOD is not yet positioned to determine what effects its  actions have had on improving service acquisition. Specifically,  USD(AT&L) has not yet fully addressed two key factors—a desired end  state for the future with specific goals and associated metrics that would  enable it to assess progress toward achieving those goals and determine  whether service acquisition is improving. USD(AT&L) is challenged in  addressing these key factors, in part, because it has limited insight into  the current status of service acquisition in terms of the volume, type,  location, and trends. While they have not established metrics to assess  departmentwide progress, USD(AT&L) officials rely on reviews of  individual service acquisitions, command level assessments, and  feedback from the military departments as means to gauge whether  DOD’s efforts are contributing to better service acquisitions. DOD has not  established aggregated results or trends which could be used to provide a  departmentwide perspective on the effects of its actions.
subsections: section_title: DOD Does Not Have the Information Needed to Define a Desired End State for Its Improvement Efforts
paragraphs: USD(AT&L) and military department leadership have demonstrated a  commitment to improving service acquisition, but USD(AT&L) officials  stated that they have not defined the desired end state or specific goals  its actions were intended to achieve. In our November 2006 report, we  found, based on assessments of leading commercial firms, that  identifying and communicating a defined end state or specific goals can  significantly improve service acquisition. This work also found that being  able to define a desired end state or what goals are to be achieved at a  specified time necessitates knowledge of the current volume, type,  location, and trends of service acquisitions.

USD(AT&L) and the military department senior services managers  acknowledge that they are challenged in defining the desired end state, in  part, because limitations within DOD’s contracting and financial data  systems hinder their insight into where service acquisition is today.  USD(AT&L) and military department officials explained that DOD’s  primary source of information on contracts, the Federal Procurement Data  System-Next Generation (FPDS-NG), has a number of data limitations,  including that it only reflects the predominant service purchased on a service does not reveal any services embedded in a contract for goods, does not fully identify the location of the requiring activity  contracting for the service.

Additionally, DOD’s financial systems do not provide detailed information  on DOD’s budget and actual spending on specific types of contracted  services and are not linked to the data maintained in FPDS-NG.  According to USD(AT&L) officials and the senior services managers,  collectively, the limitations of both FPDS-NG and DOD’s financial systems  create challenges in identifying the current volume, type, location, and  any potential trends in service acquisition. For example, USD(AT&L)  stated that DOD wants to more strategically manage its nine portfolio  groups of contracted services but does not have adequate insight into  what services DOD currently buys within these portfolio groups.

To improve insight into DOD’s contracted services, USD(AT&L) is linking  DOD’s contract and financial data systems and increasing the level of  detail these systems provide. For example, DOD is updating its financial  systems to provide data on each service purchased under a contract.  USD(AT&L) officials stated that improving and linking data within its  contract and financial systems will enable DOD to determine what it  budgeted for a particular service, what it actually spent for that service,  and which organizations bought the service. Officials, however, do not  expect to have this capability until at least 2014. USD(AT&L) officials  noted that this effort could help provide better insight into future budget  requirements for services. USD(AT&L) officials also stated that they are  exploring how to use Electronic Document Access—a DOD online  document access system for acquisition related information—to provide  them with better insight into the different types of services DOD buys  under each of its contracts. USD(AT&L) identified that, collectively, these  efforts will help them to improve the management of its nine portfolio  groups of contracted services, thereby enabling the department to better leverage its buying power, provide insight into the marketplace and buying behaviors, and  identify opportunities for cost savings.

In its April 2013 Better Buying Power Initiative memorandum, USD(AT&L)  also identified that by managing service acquisition by portfolio group, the  senior services managers should be able to work with requiring activities  to forecast future services requirements. While the military departments  have taken some steps to forecast or track future contracted services  requirements, these efforts are too new to determine their utility in  identifying what services DOD plans to buy. For example, in 2012, the  Army senior services manager requested that Army commands provide  an estimate for contracted services valued over $10 million to be  purchased over the next five fiscal years in an effort to identify any  potential cost savings. Air Force officials also track information on service  acquisitions that they expect will be awarded over the next three years to  aid in planning acquisition strategy reviews. The Navy is developing its  own approach to forecast future contracted services requirements, which  officials stated will be implemented in 2013. While it is too early to assess  the effects of these forecast or tracking efforts, they have the potential to  help the military departments better understand what services will be  purchased and facilitate DOD in identifying its desired end state for  service acquisition.

section_title: USD(AT&L) Has Not Established Metrics to Determine its Progress in Improving Service Acquisition
paragraphs: USD(AT&L) has not established departmentwide metrics to assess the  effects of its actions to improve service acquisition. Our prior work found  that metrics linked to specified outcomes are another key factor to  (1) evaluating and understanding performance levels, (2) identifying  critical processes that require attention, (3) documenting results over  time, and (4) reporting information to senior officials for decision making  purposes. In lieu of such metrics, USD(AT&L) and military department  officials stated that they rely on results from reviews of individual service  acquisitions, command level assessments, and feedback from the military  departments to gauge whether the department’s actions to improve  services acquisitions, such as those required by Congress or established  under DOD’s Better Buying Power Initiative, are having a positive effect.

USD(AT&L) officials have acknowledged the need to establish  departmentwide metrics but explained that developing such metrics has  proven challenging. They further indicated that metrics used by leading  commercial companies, which often focus on reducing spending for  services to improve a company’s financial position, may not be  appropriate for DOD. USD(AT&L) officials noted that DOD’s budget is  based on an assessment of its missions and the resources needed to  achieve its objective. These officials noted that while DOD is continuously  looking for ways to improve its efficiency, it is difficult to set goals and  measure actual reductions in spending as any savings or cost avoidances  will generally be invested in other unfunded or high priority activities.  Further, USD(AT&L) officials noted that since DOD’s budget is  appropriated by Congress rather than derived from the sale of goods and  services, changes in its resources are often outside its direct control.

While developing goals and metrics is challenging, it is not impossible.  DOD has acknowledged the need to establish departmentwide metrics.  For example, our recent work on strategic sourcing—a process that  moves an organization away from numerous individual acquisitions to a  broader, aggregate approach—found that federal agencies, including  DOD, could expand the use of this approach. Strategic sourcing  enables federal agencies to lower costs and maximize the value of  services they buy, which is consistent with DOD’s Better Buying Power  Initiative. We found that some agencies, including DOD, did not address  the categories that represented their highest spending, the majority of  which exceeded $1 billion and were for services. To improve its strategic  sourcing efforts at DOD, we recommended, among other things, that  DOD set goals for the amount of spending managed through strategically  sourced acquisitions, link strategic sourcing to its Better Buying Power  Initiative, and establish metrics, such as utilization rates, to track progress  toward these goals. DOD concurred with the recommendations and  stated it would establish goals and metrics by September 2013.

In the absence of departmentwide metrics, USD(AT&L) officials and  senior services managers identified several ongoing efforts they rely on to  gauge the effects of their actions to improve service acquisition. For  example, USD(AT&L) and the military departments conduct pre- and  post-award independent management reviews, or peer reviews, to ensure  individual service acquisitions are conducted in accordance with  applicable laws, regulations, and policies. USD(AT&L) and military  department officials stated that through these peer reviews, they can  determine if individual service acquisitions have resulted in the intended  outcomes. For example, during the post-award phase, reviewers are to  assess whether cost, schedule, and performance measures associated  with individual service acquisitions are being achieved. We have  previously found, however, that cost or schedule performance measures  may not be as effective for service acquisitions as they are for product or  weapon system acquisitions. Further, while peer reviews provide DOD  with insight into the performance of a single service acquisition, DOD  does not have information on how many post-award peer reviews have  been completed by the military departments and has not aggregated the  results or identified trends from all of DOD’s peer reviews.

Additionally, the Air Force and the Navy are conducting assessments at  the command level to evaluate organizations that buy and manage  service acquisitions. These assessments are intended to identify  performance levels, needed improvements, and best practices. For  example, the Air Force implemented health assessments to review a  command’s timeliness of contract awards, creation and use of  standardized templates, implementation of internal and external  recommendations and new policy requirements, and quality of  communication. According to officials, the Air Force first implemented its  health assessments in approximately 2009 to rate or score each of its  commands in a number of different performance areas, such as program  management and fiscal responsibility. Air Force officials reported,  however, that they have not established baselines or identified any  quantifiable trends from these health assessments. That said, Air Force  officials told us that these assessments have contributed to improvements  in the service acquisition process. For example, in a 2011 health  assessment, the Air Force found that one program office reduced the use  of bridge contracts—a potentially undesirable contract that spans the time  between an expiring contract and a new award—by 50 percent from fiscal  year 2010 to 2011.2012. During this assessment, the Navy identified a requirements  development tool created and used within a command that was  potentially a best practice and is being considered for Navy-wide use. The  Army’s senior services manager is in the process of determining how to  assess the health of the Army’s service acquisition organizations and  expects to implement an approach in 2013.

The Navy completed its first health assessment in  USD(AT&L) officials also plan to assess the health of service acquisition  across the military departments, potentially down to the program office  level, using a number of indicators of risks, referred to as tripwires.  Tripwires are established thresholds for measurable risk or performance  indicators related to the acquisition of goods or services that, when  triggered, could result in further review. USD(AT&L) officials stated that  tripwires are still under development but could include thresholds for the  number of days FPDS-NG data was input past deadlines or the number  of contract modifications within 30 days of contract award. USD(AT&L)  officials explained that tripwires alone are not sufficient to assess service  acquisition performance, but tripwires could provide insight into what may  or may not be going well and provide trend data over time.

Further, USD(AT&L) annually reviews the military departments and other  DOD components to understand the effects of its actions and policies  related to improving service acquisitions and solicit recommendations for  changes. For example, in 2012, USD(AT&L) inquired about the actions  that have been taken to comply with various defense acquisition  regulations or policies, such as the Better Buying Power Initiative. The  Army’s and Navy’s responses noted that actions to improve competition  led to an 11 and 12 percent increase, respectively, in the rate of effective  competition—situations where more than one offer is received in  response to a competitive solicitation—for service contracts from fiscal  year 2010 through 2012. In response to an open-ended question on  recommendations for improvements, each military department suggested  that USD(AT&L) take additional actions to increase departmentwide  coordination on service acquisitions. Specifically, the Army and the Air  Force recommended departmentwide service acquisition management  meetings to coordinate on issues such as emerging regulations,  directives, and policies to improve service acquisitions. In response,  USD(AT&L) officials told us that the Director of DPAP meets with the  military departments’ senior services managers regularly.

DOD’s ongoing efforts to gauge the effects of their actions to improve  service acquisition also offer opportunities for DOD to develop baseline  data, establish goals, and identify departmentwide metrics to measure  progress. For example, by analyzing and aggregating the results of its  health assessments, each military department could establish baselines  against which to assess individual commands and over time, identify  trends to determine if its commands are improving how they acquire  services. Similarly, in coordination with the military departments,  USD(AT&L) could use its tripwire approach to determine what percent of  DOD’s service acquisition strategies are not approved or require changes  before approval. DOD could then use such information to help identify  reasons for why certain service acquisitions are not approved and  determine appropriate corrective actions. DOD could further develop  metrics associated with actions outlined in the Better Buying Power  Initiative. For example, using its established services portfolio groups,  DOD could develop baseline data on the degree of effective competition  for services within each group. Depending on the results of that analysis,  DOD could determine whether it would be appropriate to establish  effective competition goals and metrics for each portfolio group or specific  types of services within each group.

section_title: Conclusions
paragraphs: In light of the billions of dollars DOD spends each year on services and  the constrained fiscal environment, it is critical for DOD to identify how it  can best utilize its financial resources and acquire services more  efficiently and effectively. DOD leadership has demonstrated a  commitment to improving service acquisition and management and has  taken a number of actions to address legislative requirements. For  example, USD(AT&L) and the military departments have focused more  management attention on improving service acquisitions through new  policies and guidance, reviews of high-dollar service acquisitions, and  new tools and training for personnel who acquire services. Further, DOD  recently designated the Principal Deputy Under Secretary of Defense for  Acquisition, Technology, and Logistics as the department’s senior  manager for service acquisition and has established similar positions,  including senior services managers, within each of the military  departments. In some cases, however, DOD remains in the process of  defining the duties and responsibilities of these positions. When taken  collectively, DOD has taken action to address the requirements of  10 U.S.C. § 2330 and section 807 of the NDAA for Fiscal Year 2012.

DOD, however, does not know whether or how these actions, individually  or collectively, have resulted in improvements to service acquisition. This  is due, in part, to the fact that DOD continues to have limited knowledge  and baseline data on the current state of service acquisition. To address  this shortfall, DOD expects to obtain better service acquisition data by  improving and linking data within its contract and financial systems, but  this effort will not be complete until at least 2014. Having baseline budget  and spending data can provide a foundation for measuring progress, but  other factors such as articulating its desired end state and developing  specific and measurable goals are also important for assessing progress.  While developing specific goals and departmentwide metrics is  challenging, it is not impossible. For example, DOD concurred with the  need to set goals for the amount of spending managed through  strategically sourced acquisitions, link strategic sourcing to its Better  Buying Power Initiative, and establish metrics, such as utilization rates, to  track progress toward these goals. However, DOD is currently missing  opportunities to fully leverage its command-level assessments, feedback  from the military departments, and other ongoing efforts it relies on to  gauge the effects of its actions to improve service acquisition. Each of  these efforts has merit and value in their own regard. Nevertheless, until  DOD utilizes them to develop baseline data, goals, and associated  metrics, similar to what it has committed to do for its strategic sourcing  efforts, DOD will continue to be in a position where it does not know  whether its actions are sufficient to achieve desired outcomes.

section_title: Recommendations for Executive Action
paragraphs: To better position DOD to determine whether its actions have improved  service acquisition, we recommend that the Principal Deputy Under  Secretary of Defense for Acquisition, Technology, and Logistics, in  consultation with the military departments’ senior services managers, take  the following three actions:  identify baseline data on the status of service acquisition, in part,  by using budget and spending data and leveraging its ongoing  efforts to gauge the effects of its actions to improve service  acquisition, develop specific goals associated with their actions to improve establish metrics to assess progress in meeting these goals.

section_title: Agency Comments and Our Evalutation
paragraphs: DOD provided us with written comments on a draft on this report, which  are reprinted in appendix III. DOD concurred with the three  recommendations, noting that they are consistent with DOD’s ongoing  Better Buying Power Initiative. DOD also stated that as it improves its  management of service acquisition, it should be able to measure  performance, track productivity trends, and establish consistent best  practices across the department. We agree that DOD has the opportunity  to leverage its ongoing efforts as it works to implement our  recommendations. By incorporating our recommendations into those  efforts, DOD will be better positioned to determine whether its actions are  improving service acquisition. DOD also provided technical comments,  which were incorporated as appropriate.

We are sending copies of this report to the Secretary of Defense; the  Secretaries of the Army, Air Force, and the Navy; the Principal Deputy  Under Secretary of Defense for Acquisition, Technology, and Logistics;  and interested congressional committees. This report will also be  available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841 or by e-mail at dinapolit@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. Staff who made key contributions  to this report are listed in appendix IV.

section_title: Appendix I: Department of Defense Actions to Implement Requirements in 10 U.S.C. § 2330
paragraphs: In 2001, Congress required the Secretary of Defense to implement a  management structure for the acquisition of services under section 2330,  title 10, United States Code (U.S.C.). This provision requires, among  other things, the Department of Defense (DOD) to develop a process for  approving individual service acquisitions based on dollar thresholds and  other criteria to ensure that DOD acquires services by means that are in  the government’s best interest and managed in compliance with  applicable statutory requirements. Under DOD’s initial May 2002  guidance for implementing the required management structure and  service acquisition approval process, the Under Secretary of Defense for  Acquisition, Technology, and Logistics (USD(AT&L)) was to review all  proposed service acquisitions with an estimated value of $2 billion or  more. The military departments and other defense components were to  review service acquisitions below that threshold. The military  departments each subsequently developed their own service acquisition  approval processes that had several elements in common. Chief among  these elements was the requirement that acquisition strategies be  reviewed and approved by senior officials before contracts are awarded.  Acquisition strategies to be reviewed were to include, among other things,  information on contract requirements, anticipated risks, and business  arrangements. Once acquisition strategies were approved, DOD  contracting offices may continue the acquisition process, including  soliciting bids for proposed work and awarding contracts.

In January 2006, Congress amended 10 U.S.C. § 2330 to include  additional requirements for DOD’s management of the acquisition of  services. The amendment requires, among other things, that the senior  officials responsible for management of acquisition of contract services  assign responsibility for the review and approval of procurements based  on estimated value of the acquisition. Senior officials within DOD are  identified as USD(AT&L) and the service acquisition executives of the  military departments. In response to these requirements, USD(AT&L)  issued an October 2006 memorandum to update its 2002 acquisition of  services policy. The revised policy identifies categories of service  acquisitions, based on dollar thresholds and related roles and  responsibilities within USD(AT&L) and the military departments. The  policy requires all proposed service acquisitions with a value estimated at  more than $1 billion be referred to USD(AT&L) and formally reviewed at  the discretion of USD(AT&L). Acquisitions with a value estimated under  that threshold are subject to military department acquisition approval  reviews. USD(AT&L)’s 2006 acquisition of services policy was  incorporated into Enclosure 9 of DOD’s 5000.02 acquisition instruction.  In 2010, USD(AT&L) required that each of the military departments  establish senior managers to be responsible for the governance in  planning, execution, strategic sourcing, and management of service  contracts. Additionally, these senior managers are to review service  acquisitions valued at $10 million but less than $250 million. USD(AT&L)  expects to issue a stand-alone instruction in 2014 for service acquisition  policy to replace Enclosure 9 of DOD Instruction 5000.02. See table 2 for  a summary of service acquisition review thresholds and approval  authorities.

The 2006 amendments to 10 U.S.C. § 2330 require DOD to take a  number of other actions. For example, DOD is to develop service  acquisition policies, guidance, and best practices; appoint full-time  commodity managers for key categories of services; and ensure  competitive procedures and performance-based contracting be used to  the maximum extent practicable. In table 3, we summarize the actions  that DOD took in response to the requirements in 10 U.S.C. § 2330. To  do so, we collected USD(AT&L) and each military department’s self- reported information using a data collection template; corroborated  reported actions with related documentation when available; and  conducted interviews with knowledgeable agency officials to clarify  responses. We did not evaluate the appropriateness or sufficiency of any  actions taken or planned by DOD.

section_title: Appendix II: Under Secretary of Defense for Acquisition, Technology, and Logistics Actions to Address Elements in Section 807 of the National Defense Authorization Act for Fiscal Year 2012
paragraphs: Section 802 of the National Defense Authorization Act (NDAA) for Fiscal  Year 2010 required the Under Secretary of Defense for Acquisition,  Technology, and Logistics (USD(AT&L)) to direct the Defense Science  Board (DSB) to independently assess improvements to the Department of  Defense’s (DOD) acquisition and oversight of services. The resulting  March 2011 DSB report, “Improvements to Services Contracting,”  contained 20 recommendations aimed at improving DOD’s contracting for  services. These recommendations focused on developing new policies  and processes to strengthen management and oversight of services  contracting, designating roles and leadership responsibilities, and  strengthening the skills and capabilities of personnel involved in services  contracting, including those in contingency environments. Subsequently,  section 807 of the NDAA for Fiscal Year 2012 required USD(AT&L) to  develop a plan, by June 28, 2012, to implement the DSB  recommendations. The plan was to address, to the extent USD(AT&L)  deemed appropriate, eight different elements most of which align with the  DSB recommendations.

USD(AT&L) officials told us they did not develop a specific plan to  address the section 807 requirement, but that the April 2013 Better  Buying Power Initiative memorandum addresses seven of the eight  elements. In reviewing the memorandum, we also found that it reflects  actions to address all of the elements except the one pertaining to training  and exercises during contingency operations. USD(AT&L) also identified  23 different actions it has taken or plans to take that officials regard as  addressing all of the elements the plan was to include, a number which  pre-date the April 2013 Better Buying Power Initiative memorandum.

Table 4 provides a summary of the actions USD(AT&L) reported as  addressing each of the eight section 807 elements. To determine if  USD(AT&L) has taken or planned actions to address the elements in  section 807, we collected USD(AT&L)’s self-reported information using a  data collection template, corroborated reported actions with related  documentation when available, and conducted interviews with  knowledgeable USD(AT&L), military department, and Defense Acquisition  University officials to clarify responses. We did not evaluate the  appropriateness or sufficiency of any actions taken or planned by  USD(AT&L).

section_title: Appendix IV: GAO Contact and Staff Acknowledgements
subsections: section_title: GAO Contact
subsections: section_title: Staff Acknowledgments
paragraphs: In addition to the contact name above, the following staff members made  key contributions to this report: Johana R. Ayers; Helena Brink, Burns  Chamberlain Eckert, Danielle Greene, Kristine Hassinger; Justin Jaynes;  and Roxanna Sun.


Question-summary annotations: questions: question: What has been DOD's stance toward service acquisition and management in the last decade?
answer: Over the last decade, the Department of Defense (DOD) has taken several actions to address legislative requirements to improve the acquisition and management of services.
child_questions: question: What legislative requirements did DOD face?
answer: In 2001, as amended in 2006, Congress required DOD to implement a management structure for the acquisition of services.
child_questions: question: How did DOD respond to these requirements?
answer: In response, DOD implemented such a structure and service acquisition review and approval process. Recently, DOD also established new positions within its management structure, including senior managers within the office of the Under Secretary of Defense for Acquisition, Technology, and Logistics (USD(AT&L)) and the military departments, to oversee and coordinate service acquisition.

question: How is USD(AT&L) addressing these requirements?
answer: With a management structure and review process in place, USD(AT&L) is focusing on efforts to improve the process for how requirements for individual service acquisitions are developed and enhancing training to respond to several legislative directives. USD(AT&L) also created its Acquisition of Services Functional Integrated Product Team, in part, to determine how to address legislative requirements to provide training for personnel acquiring services.
child_questions: question: To what extent did USD(AT&L) address the Defense Science Board recommendations?
answer: USD(AT&L) did not develop a plan to implement the Defense Science Board recommendations to improve service acquisition but identified 23 different actions, including its Better Buying Power Initiative, it has planned or taken that officials regard as addressing what the plan was to include. For example, USD(AT&L) is updating its guidance on using incentives to improve contractor performance, which addresses one of the elements that was to be in the plan.

questions: question: To what extent has DOD addressed legislative requirements regarding service acquisition?
answer: While DOD has taken a number of actions that address legislative requirements, DOD is not yet positioned to determine what effects these actions have had on improving service acquisition. Specifically, USD(AT&L) has not identified specific goals and associated metrics that would enable it to assess progress toward achieving those goals.

question: How successfully has USD(AT&L) improved service acquisition?
answer: USD(AT&L) has identified improving service acquisition as a priority but has not defined a desired end state for its actions or the measurable characteristics that would embody achieving such a goal. It is challenged in defining a desired end state for its actions, in part, because it has not determined the current status of service acquisition in terms of the volume, type, location, and trends. It is challenged in defining a desired end state for its actions, in part, because it has not determined the current status of service acquisition in terms of the volume, type, location, and trends.

question: What actions has DOD taken to improve its contract and financial systems?
answer: DOD is taking steps to improve its contract and financial systems to obtain such data, but these efforts will not be complete until at least 2014. However, DOD is not fully leveraging the command-level assessments, feedback from the military departments, and other ongoing efforts it relies on to gauge the effects of its actions to improve service acquisition. Nevertheless, despite the challenges in doing so, it is not impossible.
child_questions: question: What is one example of progress that has been made?
answer: For example, DOD has agreed to set goals for the amount of spending managed through strategically sourced acquisitions, link strategic sourcing to its Better Buying Power Initiative, and establish metrics, such as utilization rates, to track progress toward these goals.

question: What are some areas of improvement that remain to be undertaken?
answer: However, DOD is not fully leveraging the command-level assessments, feedback from the military departments, and other ongoing efforts it relies on to gauge the effects of its actions to improve service acquisition.

question: What would be the benefit, for DOD, of greater measurement and assessment?
answer: By using its budget and spending data and leveraging these efforts, DOD could develop baseline data and identify trends over time, enabling it to develop measurable goals and gain more insight into whether its actions are improving service acquisition. Until then, DOD will continue to be in a position where it does not know whether its actions are sufficient to achieve desired outcomes.

questions: question: Why did GAO report on DOD's efforts to improve service management and acquisition?
answer: The National Defense Authorization Act for Fiscal Year 2012 mandated that GAO report on DOD’s actions to improve service acquisition and management.
child_questions: question: What did GAO examine?
answer: GAO examined (1) the actions DOD has taken to respond to legislative requirements and (2) how DOD determines the effects of its actions to improve service acquisition.
child_questions: question: What data did GAO collect for their report?
answer: GAO reviewed documentation and interviewed DOD officials on the actions taken in response to the legislative requirements.

question: How did GAO assess the DOD's use of best practices?
answer: GAO also assessed whether DOD addressed key factors, including establishing goals and metrics, to help it determine if it has improved service acquisition.