Defense Acquisitions: Further Actions Needed to Address Weaknesses in DOD's Management of Professional and Management Support Contracts
Published: 2009-11-20T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In fiscal year 2008, the Department of Defense (DOD) obligated $200 billion on services contracts, including $42 billion for professional and management services. The Government Accountability Office (GAO) previously identified weaknesses in DOD's management and oversight of services contracts, contributing to DOD contract management being on GAO's high-risk list. For selected professional and management support contracts, GAO was asked to examine (1) the extent to which DOD considered the risks of contractors closely supporting inherently governmental functions at key decision points, (2) how DOD implemented performance-based practices, (3) the extent to which DOD designated trained surveillance personnel, and (4) whether a new review process may improve DOD's management of such contracts. GAO reviewed federal regulations, agency policies and guidance, and analyzed seven acquisitions approved from 2004 to 2007 and 64 related task orders for services.
section_title: What GAO Found
paragraphs: DOD policies do not require assessments of the risks associated with contractors closely supporting inherently governmental functions as part of its management reviews of acquisition strategies nor when task orders are issued for professional and management services. Such risks include the potential loss of government control over and accountability for mission-related policy and program decisions. Though all seven acquisitions and more than 75 percent of the task orders GAO reviewed provided for such services, GAO found no evidence that these risks were among those considered in the documentation reviewed. DOD guidance issued after these acquisitions were approved requires that consideration be given to using civilian personnel rather than contractors when the activities closely support inherently governmental functions. This guidance, however, does not require DOD personnel to consider and document risks posed when contractors perform these activities. Further, DOD personnel were unaware of the need to provide enhanced oversight when contracting for such services. DOD faces challenges in defining requirements and outcome-based measures when acquiring professional and management services. DOD personnel generally expressed task order requirements in terms of a broad range of activities that contractors may perform, but used standards and measures that were not always well-suited to assess outcomes. DOD made more use of objective measures to assess cost and schedule performance, but generally relied on subjective measures to assess the quality of the contractors' work. For example, DOD often measured quality based on the number of complaints lodged against the contractor, which provided little detail into how desired outcomes were achieved. DOD also missed opportunities to include objective measures that may have been better suited to assess outcomes. DOD officials stated that developing outcome-based, objective measures is challenging, but noted that initiatives are under way to better utilize such approaches. DOD has made progress in ensuring that trained surveillance personnel are assigned to monitor contract performance. Surveillance personnel were assigned to all 64 of the task orders GAO reviewed, and all but 3 had received required training. GAO identified, however, 3 instances of surveillance personnel who were not assigned before the contractor began work on a task order and 20 instances of personnel who did not receive training prior to beginning surveillance duties. In September 2008, DOD implemented a new peer review process that is tasked to address, among other issues, contractors closely supporting inherently governmental functions, the use of performance-based practices, and contractor surveillance. As of October 2009, four pre-award reviews and one post-award review of professional and management support contracts have been conducted and it is too early to tell whether such reviews will encourage DOD personnel to address these issues across the range of DOD's services contracts.

section_title: Letter
paragraphs: The Department of Defense (DOD) is the federal government’s largest  purchaser of contractor-provided services, obligating more than $200  billion on services contracts in fiscal year 2008. DOD obligated $42 billion  (21 percent) of its 2008 obligations for services on professional,  administrative, and management support. Examples of such services  include budget and program management and contracting support. Often  these services are procured through performance-based acquisition  approaches, which encourage the private sector to develop innovative  solutions to meet DOD’s needs.
DOD’s reliance on contracted services to support its missions makes  effective management and oversight of these contracts essential. Our work  over the past several years, however, has identified weaknesses in DOD’s  management and oversight. For example, in 2005, we found that DOD’s  surveillance of service contractors needed improvement as staff  responsible for assessing contractor performance were not always  appointed or trained. In 2006, we found that DOD’s approach to managing  services acquisitions did not put the department in a position to determine  whether its investments in services were achieving the desired outcomes.
These shortfalls stemmed, in part, from the limitations of the management  reviews DOD established to review and approve high-dollar services  acquisitions. Additionally, our recent work has shown that reliance on  contractors, including professional and management support contractors,  to support core missions can place the government at risk of transferring  government responsibilities to contractors. Collectively, such issues  continue to place DOD at high risk of fraud, waste, abuse, and  mismanagement and result in DOD contract management being on our  high-risk list.
In response to your interest in whether DOD’s policies and actions to  improve the management of services contracts addressed issues affecting  professional and management support contracts, we examined (1) the  extent to which DOD considered the risks associated with contractors  closely supporting inherently governmental functions at key acquisition  decision points; (2) how DOD was implementing performance-based  acquisition practices, such as identifying requirements in terms of  expected and measurable outcomes; (3) the extent to which DOD  designated trained surveillance personnel; and (4) whether recent actions  to implement a peer review process may improve DOD’s management and  use of such contracts.
To assess the extent to which DOD’s management reviews and policies  addressed the risks associated with contractors closely supporting  inherently governmental functions at key acquisition points, we reviewed  DOD’s May 2002 and October 2006 guidance that established its processes  for reviewing and approving proposed acquisitions to obtain contractor  support. To assess how these risks were addressed during reviews of  specific services acquisitions, we reviewed a DOD-provided list of 102  services acquisitions approved by the Air Force, Army, or Navy from fiscal  years 2004 through 2007. Using contract data from the Federal  Procurement Data System–Next Generation, we determined that 32 of  these acquisitions, with almost $15 billion in total combined obligations  from fiscal years 2004 through 2008, included contracts for the purchase of  professional, administrative, and management support. From these 32  acquisitions, we selected 7 for our review based on the percentage of total  obligations that went towards professional, administrative, and  management support services; the specific types of services acquired; and  the military department responsible for the acquisition. We also  determined that the military departments obligated over $4.3 billion on  1,874 task orders from these 7 acquisitions from fiscal years 2004 through  2008. We randomly selected and reviewed 64 task orders that had  obligations of $500,000 or more and were issued during fiscal years 2004  through 2007 from these acquisitions to assess how such risks were  addressed prior to DOD’s decision to award a contract or issue a task  order for professional and management support contracts. We did not  review acquisitions approved after fiscal year 2007 since our analysis  indicated that it was often a year or more from the time that the  acquisition strategy was approved to the time when task orders were  actually issued. We reviewed the acquisition strategy, base contract, task  order, statements of work, and other documentation supporting the need  to acquire contract support and any risk assessments prepared. We also  interviewed officials who managed these acquisitions to obtain  information on why these services were purchased, the risks that were  considered, and any steps taken for enhanced contractor oversight.
We assessed the reliability of the Federal Procurement Data System–Next  Generation to identify acquisitions and to select task orders that were  within the scope of our review by verifying (1) the contract and task order  identification numbers, (2) the contract award date, (3) that the task  orders associated with the acquisitions were for professional and  management support services, and (4) that the task orders had obligations  exceeding $500,000. On the basis of this assessment, we determined that  the data were sufficiently reliable for the purposes of this review.
To assess how DOD was implementing performance-based acquisitions  practices on professional and management support contracts, we  reviewed Federal Acquisition Regulation (FAR) requirements and DOD  guidance on developing performance work statements and specifically on  how to identify contract requirements in terms of expected and  measurable outcomes. We then reviewed performance work statements  for the 64 task orders to assess whether contract requirements were  measurable and outcome based. We interviewed contracting and program  officials who managed these task orders to identify how requirements and  performance measures were developed and analyzed documentation to  determine how contractor performance was measured.
To assess the extent to which DOD designated trained surveillance  personnel on professional and management support contracts, we  reviewed the Defense Federal Acquisition Regulation Supplement  (DFARS) and DOD policies and procedures to identify the department’s  surveillance and training requirements. We analyzed surveillance  personnel appointment and training documentation associated with each  of the 64 task orders to determine whether these requirements were met.
To identify how recent actions to implement a new peer review process  may improve DOD’s management and use of professional and  management support contracts, we reviewed relevant legislation and the  department’s implementation policy and guidance to identify the scope  and content of these reviews. We also interviewed officials from the Under  Secretary of Defense for Acquisition, Technology, and Logistics’  (USD(AT&L)) Office of Defense Procurement and Acquisition Policy  (DPAP) and the military departments and reviewed memoranda  summarizing the findings and recommendations of reviews performed on  professional and management support services contracts. Additional  information on our scope and methodology may be found in appendix I.
We conducted this performance audit from July 2008 through November  2009 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
subsections: section_title: Background
subsections: section_title: DOD Obligations and Workforce Trends
paragraphs: From fiscal years 2001 through 2008, DOD’s reported obligations on  contracts for services, when measured in real terms, more than doubled— from roughly $92 billion to slightly over $200 billion. These obligations  accounted for over half of the department’s total contract obligations in  fiscal year 2008. Over that same time period, DOD’s obligations on  professional, administrative, and management support contracts nearly  tripled from $14.2 billion to $42 billion. These services represented about  15 percent of DOD’s total obligations on services contracts in 2001 and     21 percent in 2008. As we have reported in the past, this increased use of  contractor-provided services has been the result of thousands of individual  decisions, not the result of strategic, comprehensive planning for the  whole department in which the volume and composition of contracted  services have been measured outcomes. We also noted that the absence  of well-defined requirements, sound contracting arrangements, or effective  management and oversight has contributed to schedule delays, cost  overruns, and unmet expectations.
Despite substantial increases in spending on both goods and services from  fiscal year 2001 through 2008, DOD’s acquisition workforce has declined  by 2.6 percent (see table 1). Without an adequate workforce to manage  DOD’s billion-dollar acquisitions, there is an increased risk of poor  acquisition outcomes and vulnerability to fraud, waste, and abuse. We  reported in March 2009 that DOD lacked critical, departmentwide  information needed to ensure that its acquisition workforce was sufficient  to meet its national security mission. We found, for example, that DOD  did not collect or track information on contractor personnel, despite the  fact that those personnel providing professional and management support  services make up a key segment of the total acquisition workforce.
Additionally, DOD lacked complete information on the reasons personnel  are contracted, thus limiting its ability to determine whether decisions to  augment the in-house acquisition workforce with contractors were  appropriate.
section_title: Risks of Contractors Closely Supporting Inherently Governmental Functions
paragraphs: Federal agencies acquire basic services, such as custodial and  landscaping, to more complex professional and management support  services, which may closely support the performance of inherently  governmental functions. Tasks that require discretion in applying  government authority or value judgments in making decisions for the  government are defined by the FAR as inherently governmental functions;  as such they are required to be performed by government employees, not  private contractors. The FAR provides 20 examples of such work  functions, including determining agency policy or federal program budget  request priorities; directing and controlling federal employees; and  awarding, administering or terminating federal contracts.
The FAR also provides examples of functions that, while not inherently  governmental, approach the category due to the nature of the function, the  manner in which a contractor performs the task, or methods used by the  government to administer performance under a contract (see app. II).  Services that closely support inherently governmental functions include  professional and management support services, such as those that involve  or relate to supporting budget preparation; program planning; acquisition  planning; technical evaluation for contract proposals or source selections;  and development of statements of work.
The decision to turn to contractors can, in some cases, create risks that  the government needs to consider and manage. Of key concern is the loss  of government control over and accountability for mission-related policy  and program decisions when contractors provide services that closely  support inherently governmental functions. The closer contractor services  come to supporting inherently governmental functions, the greater the risk  of their influencing the government’s control over and accountability for  decisions that may be based, in part, on contractor work. This may result  in decisions that are not in the best interest of the government and may  increase vulnerability to waste, fraud, and abuse. Given this risk, the FAR  and Office of Federal Procurement Policy (OFPP) guidance state that a  greater scrutiny and an enhanced degree of management oversight is  required when contracting for functions that closely support the  performance of inherently governmental functions. Additionally, the  Ronald W. Reagan National Defense Authorization Act for Fiscal Year 2005  required, among other things, that prior to entering into a contract for  performance of acquisition functions that closely support inherently  governmental functions, DOD must ensure that its personnel cannot  reasonably be made available to perform these activities, that appropriate  DOD personnel supervise contractor performance and perform all  inherently governmental functions, and that DOD address any potential  organizational conflict of interest of the contractor when performing these  tasks. Table 2 provides examples of contracted services and their relative  risk of influencing government decision making.
Our previous work has identified that the use of contractors for services  that closely support inherently governmental functions introduces other  risks due to a potential loss of government control over program  decisions. Such concerns include an increased potential for conflicts of  interest, both organizational and personal; the potential for improper use  of personal services contracts, which the FAR generally prohibits because  of the employer-employee relationship they create between the  government and contractor personnel; and the potential additional cost to  the government of hiring contractors rather than government personnel.
section_title: DOD Management and Peer Reviews
paragraphs: The National Defense Authorization Act for Fiscal Year 2002 required the  Secretary of Defense to establish a review process for the approval of  DOD services acquisitions. USD(AT&L) issued guidance in May 2002  establishing its management reviews, intending to ensure that DOD’s  services acquisitions are based on clear, performance-based requirements  with measurable outcomes and that acquisitions are planned and  administered to achieve intended results. Under this initial guidance  USD(AT&L) was to review all proposed services acquisitions with an  estimated value of $2 billion or more, and military department and other  defense component officials were to review those below that threshold.  The military departments each subsequently developed their own  management review processes for acquisitions that contained several of  the same elements. Chief among these was the requirement that written  acquisition strategies must be reviewed and approved by senior officials  before contracts may be awarded. The content of these strategies  included, among other things, information on contract requirements, risks,  and business arrangements. Once the acquisition strategies are approved,  DOD contracting offices may continue the acquisition process, including  soliciting bids for proposed work and subsequently awarding contracts.
Based on further requirements in the National Defense Authorization Act  for Fiscal Year 2006, in October 2006, USD(AT&L) issued a memorandum  updating DOD’s acquisition of services policy. Under the updated policy,  all proposed services acquisitions with a value estimated at more than $1  billion or designated as “special interest” should be referred to  USD(AT&L) for review. The dollar threshold for military department  reviews was also lowered. While the substance of the management  reviews remained largely unchanged from the 2002 policy, it did  incorporate a few additional, specific, acquisition strategy requirements  concerning inclusion of any required waivers and top-level discussion of  source selection processes.
In 2006, we reported that although DOD had established formal  management reviews for the approval of services acquisitions, issues with  services contracts at the strategic and transactional level remained. We  reported that DOD’s approach to managing services acquisitions tended to  be reactive and that it had not developed a means for gauging whether  ongoing and planned efforts were achieving intended results. At the  transactional level, DOD focused primarily on elements associated with  awarding contracts, with much less attention paid to requirements or the  assessment of the actual delivery of contracted services. Moreover, the  results of individual acquisitions were generally not used to inform or  adjust strategic direction. We recommended that, among other actions,  DOD take steps to determine areas of specific risk that were inherent in  acquiring services and that should be managed with greater attention.  DOD agreed with this recommendation and has identified actions under  way to address our concerns.
In response to a requirement in Section 808 of the National Defense  Authorization Act for Fiscal Year 2008, USD(AT&L) established a  multiphased, pre- and post-contract award independent management  review, or peer review, process for services acquisitions. In December  2008, DOD Instruction 5000.02, Operation of the Defense Acquisition  System, was revised and incorporated these peer reviews as well as the  management reviews and dollar thresholds established in the October  2006 acquisition of services policy.
section_title: DOD Policies Do Not Require an Assessment of Risk of Contractors Closely Supporting Inherently Governmental Functions at Key Acquisition Decision Points
paragraphs: DOD and the military departments are to assess a number of risks when  developing an acquisition strategy for services, but DOD policy does not  require an assessment of risks associated with contractors closely  supporting inherently governmental functions at two key decision  points—when approving acquisition strategies or issuing task orders. All 7  of the proposed acquisitions for professional, administrative, and  management services and more than 75 percent of the 64 related task  orders we reviewed required the contractor to provide services that  closely supported inherently governmental functions. A DOD instruction  issued after the approval of the acquisition strategies we reviewed requires  that consideration be given to using civilian personnel rather than  contractors, specifically when the activities to be performed cannot be  separated or distinguished from inherently governmental functions.  However, once the decision to rely on contractors is made, DOD personnel  are not required to identify and document risks posed when contractors  are given responsibility for closely supporting inherently governmental  functions or take steps to mitigate those risks.
subsections: section_title: Risks Associated with Contractors Closely Supporting Inherently Governmental Functions Are Not among Those Assessed in Acquisition Strategies Submitted for Management Reviews
paragraphs: DPAP representatives we met with noted that the acquisition strategy is a  “big picture” document that examines whether the proposed methods for  acquiring services are sound and how the acquisition compares with  previous acquisitions. The strategy is developed, in part, with the  assistance of both program office representatives, who identify the  requirements, and contracting personnel, who will develop the basic  contracts and subsequently issue task orders. Agency personnel are  required to document in the acquisition strategy their assessment of  current and potential technical, cost, schedule, and performance risks; the  level of those risks; and a mitigation plan. As part of the acquisition  strategy, this assessment is subsequently reviewed by senior DOD officials  during management reviews. Documentation for all seven acquisition  strategies we reviewed included a discussion of these risks as well as  methods to mitigate their impact, as required in policy.
DOD policy, however, does not require the military departments to  consider and document in their acquisition strategies the extent to which  the proposed services will be used to closely support inherently  governmental functions. As a result, none of the acquisition strategies or  related risk assessment documentation reviewed under DOD’s  management reviews that we analyzed identified such concerns or any  related mitigation strategies. The acquisition strategies and supporting  documentation we reviewed included broad descriptions of the services to  be provided over the course of the acquisition, which included acquisition,  contracting, and policy development support services. Each of these  services are identified in the FAR as examples of those that closely  support inherently governmental functions.
DOD and military department officials we spoke with indicated that, to  some degree, DOD’s management review is not well suited to assess the  risks of contractors closely supporting inherently governmental functions.  These officials noted that the acquisition strategies generally described  requirements in broad terms and that the timing of the review—often  months before DOD actually solicits bids from the contractors or awards  contracts and issues task orders for specific services—makes it  challenging to know what specific risks might be encountered or whether  a mitigation strategy is warranted. These officials indicated that identifying  such risks would be more appropriate during the planning for the  subsequent award of contracts or issuance of task orders, when the  program offices have more clearly defined their specific needs and  requirements.
In most cases, however, the military departments had prior knowledge  about their expected use of contractors to provide such services. For  example, all of the acquisition strategies and supporting documentation  we reviewed justified the need to obtain contractor support to perform  these functions due to a lack of government resources needed to meet  daily mission requirements. These documents and contracting officials  indicated that the offices supported by these acquisitions have long relied,  in some cases for over a decade, on contractor support to augment the  government workforce and perform tasks that closely support inherently  governmental functions. For example, although the Professional  Acquisition Support Services strategy that supports the Air Force’s  Electronic Systems Center was approved in 2006, the center has  contracted for similar acquisition support services continuously since  1984. Likewise, the acquisition management services provided to the Air  Force’s Air Combat Command through the Technical Acquisition  Management Support 3 acquisition have been obtained through previous  acquisitions dating back to 1989. Although officials stated that  documentation explaining the need to contract for these services in the  past was often unavailable, contracting officers and program officials  indicated that reductions in government personnel have led to the  increased use of contractors to perform activities government personnel  would have performed in the past.
Program and contracting officials stated that they would now prefer to use  government personnel to perform these activities, but they noted the  length of time it takes to hire federal employees and the lack of available  personnel funds or positions necessitates them to use contractor support.  Program and contract officials also informed us that the decision not to  pursue additional federal employees instead of contractors was made by  the supported programs before these officials became involved with the  acquisition process. Further, they indicated they were not provided with  the analyses used to support these decisions. We did not find any analysis  or discussion of how these decisions were made in the acquisition  strategies or supporting documentation submitted for the management  review.
section_title: Risks of Contractors Closely Supporting Inherently Governmental Functions Are Not Considered and Documented by DOD Personnel When Awarding Contracts or Issuing Task Orders
paragraphs: According to DOD officials, personnel are not required to consider and  document the risks associated with contractors closely supporting  inherently governmental functions when awarding contracts or issuing  task orders. Forty-nine of the 64 task orders we reviewed included  services that, as described in the FAR, are examples of activities that  closely support inherently governmental functions, including support for  developing statements of work or contract documents; or budget  preparation. Program managers and contracting officers we spoke with  acknowledged that contractors closely supported inherently government  functions, but none of the contract files identified them as such or  indicated if any steps were taken to address related risks.
The associated contract files for each of the task orders we reviewed  included provisions specifically prohibiting the contractor from  performing inherently governmental functions. Program managers and  contracting officers informed us that they were aware of the importance of  preventing contractors from performing inherently governmental  functions as required by the FAR. These officials acknowledged that  without contractor support, fulfilling mission requirements or conducting  certain program activities could not continue and some recognized that  the close working relationships that develop between government and  contractor support personnel increase the risks of contractors performing  inherently governmental functions. To prevent contractors from  performing such tasks, program and contracting officials indicated that  they reviewed task order requirements to ensure that they are within the  scope of the acquisition and do not require contractors to perform tasks  that should be left only to government employees. Officials further stated  that when developing performance work statements they emphasize that  the contractors’ role is to provide assistance to the government rather than  make program decisions.
Program and contract personnel that we interviewed who were  responsible for overseeing the work done under the task orders were  unaware, however, of the FAR requirement to provide greater scrutiny and  an enhanced degree of management oversight and surveillance when  contracting for services that closely support inherently governmental  functions. Additionally, federal internal control standards require that  agencies conduct an assessment of risks, such as risks that result from  heavy reliance on contractors to perform critical agency operations.  According to DOD officials, however, no specific guidance has been  provided by DOD that defines how contracting and program officials  should conduct such enhanced oversight. DPAP officials noted that  additional information on how to oversee contractors that closely support  inherently governmental functions would be useful to the military  departments, but acknowledged that they have no ongoing efforts do so.
section_title: Recent Guidance Requires DOD to Consider Using Civilian or Military Employees to Perform Activities That Closely Support Inherently Governmental Functions
paragraphs: In May 2009, DOD issued guidance in response to legislation requiring  DOD to devise and implement guidelines and procedures to ensure that  consideration is given to converting, or in-sourcing, functions currently  performed by contractors to DOD civilian personnel. This in-sourcing  guidance instructs DOD personnel to prioritize the conversion of any  currently contracted services to DOD civilian performance if the functions:  are valid and enduring mission requirements; are inherently governmental  functions; are exempted functions; are unauthorized personal services;  have problems with contract administration; or are services that require  “special consideration” for in-sourcing. Under the law and the guidance,  one of the categories of services that should be given “special  consideration” for in-sourcing is services that closely support inherently  governmental functions.
The May 2009 guidance also states that when making certain in-sourcing  decisions, agency personnel should consult workforce management  officials, as specified in DOD’s Guidance for Determining Workforce  Mix. This instruction requires DOD personnel to pay particular attention  when contracting for activities that closely support inherently  governmental functions. If an activity is so closely associated with an  inherently governmental function that it cannot be separated or  distinguished from it, the instruction requires that the function be  identified as inherently governmental and precluded from private sector  performance. This safeguard is intended to prevent the transferring of  governmental authority, responsibility, or accountability to the private  sector. However, neither the May 2009 in-sourcing guidance nor DOD’s  Guidance for Determining Workforce Mix require DOD contracting and  program personnel to identify, document, or mitigate risks posed when  contractors will be relied on to closely support inherently governmental  functions.
Further, under the May 2009 in-sourcing guidance, the conversion of  services that require “special consideration” to government performance is  to be based solely on cost and not risk; that is, these services may be in- sourced only if a cost analysis shows that performance by DOD civilian  employees is more cost-effective. Officials from the Office of the Director  for Cost Assessment and Program Evaluation stated that guidance for  standardizing how these cost analyses should be performed was expected  to be issued by December 2009. The May 2009 in-sourcing guidance also  requires that when cost analyses indicate that the private sector is the  more cost-effective provider of services, a written confirmation should be  provided to contracting officers. Further, the guidance states that all  documents leading to the decision to contract for such services should be  retained in contract files. DPAP and military department officials stated  that cost analyses are not required to be submitted in the documentation  supporting management reviews.
section_title: DOD Faces Challenges In Implementing Performance-Based Practices on Professional and Management Support Task Orders
paragraphs: DOD faces challenges in defining requirements and outcome-based  measures when using a performance-based approach to acquire  professional and management support services. DOD personnel generally  expressed task order requirements in terms of a broad range of services  that the contractors might be required to perform and used a mix of  objective and subjective measures to determine whether the contractor  achieved assigned tasks within expected cost, schedule, and quality  parameters. For example, 63 percent of the task orders that assessed  contractor cost performance principally used objective performance  measures, while two-thirds of the task orders that assessed the quality of  contractor performance principally used subjective measures. We found  objective measures generally provided more discrete information to assess  contractor performance. In contrast, subjective measures, especially those  to assess the quality of contractor work, tended to rely on customer  feedback, such as the number of complaints lodged against the contractor.  In several instances, DOD missed opportunities to include objective  performance measures that may have been better suited to assess contract  outcomes, in part because DOD personnel used the performance measures  established in the base contract rather than attempt to measure the  specific services being provided under the task order. DOD officials  acknowledged there are challenges in developing measures that assess the  outcomes of professional and management support contracts and noted  recent actions to improve existing guidance.
subsections: section_title: FAR and DOD Guidance Established Preference for Performance-Based Services Acquisition
paragraphs: In 2000, Congress established performance-based approaches as the  preferred acquisition method for acquiring most services. Under the FAR,  all performance-based acquisitions should include:  a performance work statement that describes outcome-oriented  requirements in terms of results required rather than the methods of  performance of the work;    measurable performance standards describing how to measure  contractor performance in terms of quality, timeliness, and quantity;  and  the method of assessing contract performance against performance  standards, commonly accomplished through the use of a quality  assurance surveillance plan.
DOD issued its Guidebook for Performance-Based Services Acquisition  in the Department of Defense in December 2000 to educate DOD  personnel on, and promote the use of, performance-based practices. The  guidebook suggests that personnel develop performance objectives that  encompass at a top level all the tasks that must be completed to arrive at  the desired outcome. The guidebook states that performance standards  should be identified for the performance objectives so personnel will  know if the desired outcome was satisfactorily achieved. It further states  that determining an appropriate performance standard is a judgment call  based on the needs of the mission and available expertise. DOD’s guidance  also identifies that surveillance personnel may use various measures to  assess the contractor’s performance, such as random or periodic sampling  of the contractor’s work as well as customer feedback. The guidebook  indicates, however, that customer feedback should be used prudently as it  is subjective and does not always relate to the requirements of the  contract. Lastly, the guidebook provides examples of performance  objectives with corresponding standards and measures for various  services and activities, but not specifically for professional and  management support services (see table 3).
section_title: Performance Standards and Measures Were Not Always Well Suited to Assess the Outcomes of the Broad Range of Contracted Services
paragraphs: While DOD identified as performance-based all but one of the task orders  we reviewed, we found that almost all of the task orders had broadly  defined requirements that listed various categories of services and related  activities the contractor may be required to perform over the course of the  order rather than expected results. The task orders we reviewed were  issued from base contracts that identified the categories of support  services a contractor may be required to perform. The task orders then  identified a broad range of activities that the contractor may be required to  perform based on the customer program office’s needs. For example, the  base contract for one task order identified four different categories of  support services: acquisition, financial management, contracting, and  administrative and human resources support. In turn, the task order  identified several activities the contractor could perform, such as  preparing acquisition-related documents, updating commanders on  policies and procedures, tracking and analyzing funds, maintaining  contract files, and preparing travel orders.
DOD generally grouped contractor performance into a number of different  performance objectives, including cost, schedule, and quality, and set  standards that the contractor had to meet. These performance objectives  required the contractor to, among other things, maintain control of costs  by completing work within an acceptable range of projected costs, adhere  to the government’s schedule by delivering products on time, and provide  the government with high-quality work products. DOD personnel used a  mix of objective and subjective measures to assess the contractors’  performance against the cost, schedule, and quality standards in 54 of the  64 task orders we reviewed; not all task orders established performance  measures in all three categories (see table 4). The measures used varied  depending on the area of performance assessed. For example, 63 percent  of the task orders that assessed the contractor’s cost performance  principally used objective performance measures, while two-thirds of the  task orders that assessed the quality of the contractor’s performance  principally used subjective measures.
We found that the objective performance measures yielded better  information on how well the contractor met desired cost and schedule  contract outcomes than subjective measures. For example, the objective  performance standards in one task order required the contractor to remain  within projected costs and perform tasks within schedule 97 percent of the  time. In this instance, the surveillance personnel maintained a database of  the contractor’s cost and periodically contacted customers to identify if  the contractor’s work was completed on time. The other task orders that  included cost and schedule performance standards were assessed  subjectively, based on the number of complaints lodged against the  contractor. According to task order documentation and surveillance  personnel we spoke with, a customer complaint is generated when a  contractor fails to meet performance requirements. Ten of these task  orders required the contractor to provide accurate cost forecasts and  accomplish tasks with minimum delay to program mission objectives.  None of these task orders, however, identified what would constitute a  level of performance that would result in a valid customer complaint.
Contractor performance documentation we reviewed for these task orders  indicated that DOD assessed the contractor as having met performance  requirements, but provided little to no information on what the contractor  accomplished.
Both the subjective and objective performance measures used to assess  the quality of the contractors’ services provided little insight into the  outcomes of the contractors’ work. DOD personnel relied on subjective  measures to assess the quality of contractor services provided in 40 of the  64 task orders we reviewed. For example, 30 of these tasks orders directed  surveillance personnel to assess quality based on the number of customer  complaints a contractor received. Surveillance personnel we spoke with  indicated that they regularly contacted government customers to inquire  about their overall satisfaction with the contractor’s performance. Some  surveillance personnel explained that it was difficult to determine if the  contractor met or exceeded performance standards because guidance on  making these determinations was not available. Consequently, they  generally documented the contractor’s performance as acceptable if the  customer said they were satisfied with the contractor’s performance. We  found that when the contractor’s performance was rated as meeting  requirements, surveillance personnel documented little detail about the  quality of the contractor’s work because such descriptions were often  required only when the contractor either exceeded or did not meet  expectations. In another 10 task orders involving the purchase of  engineering, technical, business, and acquisition support services, such as  helping to identify the program office’s contracting requirements and  assisting with developing requests for proposals and statements of work to  fill those requirements, quality performance standards required the  contractor to complete task order activities with little rework and with  few minor and no significant problems at least 80 percent of the time.  According to the documentation we reviewed, surveillance personnel  indicated they periodically sampled the contractor’s work to verify the  percentage that was redone and often rated the quality of the work as  exceptional because little rework was reported.
On four task orders for translation services in Afghanistan, Cuba, and  other areas, DOD personnel did not attempt to measure the quality of the  contractor’s services. In these cases, the task orders’ requirements  included that the contractor deploy translators in response to mission  requirements. The corresponding performance standard required the  contractor to meet staffing requirements no less than 95 percent of the  time, which was measured and documented by surveillance personnel.  The task orders, however, did not include a performance objective for  obtaining high-quality translation services. Contracting and program  officials explained that they did not try to measure the quality of the  translations provided because DOD lacks the personnel with translation  skills necessary to make such an assessment. These officials stated that  ensuring qualified personnel are provided in a timely manner is the best  alternative to determining if the translations provided are of high quality.
The military departments may have missed opportunities to include and  use objective performance measures that were better suited to assess  contract outcomes in several of the task orders we reviewed. In part, this  occurred because DOD personnel used the general performance measures  that were established in the base contract rather than develop measures  that were tailored for the specific work required in the task orders in more  than 80 percent of the task orders we reviewed. Consequently, in each of  the following examples, the quality of the contractor’s performance was  measured based on the number of validated complaints submitted by  government personnel.
Four task orders issued for acquisition, financial management,  contracting, and administrative support required contractors to  indicate at the end of each contract year at least two lessons learned,  best practices, or improvements it made to the government’s processes  in areas including acquisition and program management. The  requirements did not, however, identify at the outset of the task order  the type or extent of the improvements that DOD desired, such as  reductions in the time required to complete activities or costs savings.
In a $1.8 million task order for information and project management  support, the contractor was required to identify and reduce the  government’s unliquidated obligations by 25 percent in 6 months and  by 50 percent in a year when measured against an identified baseline.  Nevertheless, the task order did not include any performance  measures that were directly related to whether the contractor met the  reduction targets. The official responsible for assessing the  contractor’s performance noted, however, that he considered the  contractor’s efforts to reduce unliquidated obligations when he  assessed the contractor’s performance.
In four other task orders for acquisition and information management  support, the contractor was required to review technical proposals and  validate prices submitted by other contractors and make  recommendations on the acceptance or rejection of these proposals as  part of the support it provided to the program office’s pre-award  activities. The task orders, however, did not include objective  performance measures to assess the contractor’s performance, such as  whether the contracting officer returned the contractor’s work to  correct deficiencies or whether the reviews resulted in reducing the  government’s costs.
section_title: DOD Officials Identified Challenges in Developing Objective, Outcome- Oriented Measures
paragraphs: Our previous work at the Department of Homeland Security noted that  defining outcome-oriented requirements and measurable performance  standards may be challenging for professional and management support  services. We found similar concerns expressed by the DOD contracting,  program, and surveillance officials we interviewed. These officials  acknowledged that they find it difficult to identify and objectively measure  the outcomes of professional and management support services contracts  due to the broad range of support provided. They stated that these task  orders encompassed a range of activities which, while not inherently  governmental, would typically be performed by federal employees.  Consequently, officials stated the performance work statements needed to  be written broadly to provide the flexibility to obtain specific support as  needed and that contractors were often viewed as simply augmenting the  government’s workforce. Further, these officials noted that it was often  not practical to measure the work contractors performed and that  subjective measures, such as the number of customer complaints received,  are frequently used as an alternative to assess whether the contractor met  the government’s requirements. As a result, they generally considered the  outcome of these task orders to be obtaining qualified people rather than a  specific result the contractor was required to achieve.
To address the challenges of developing performance-based requirements  and measures for professional and management support, a Defense  Acquisition University (DAU) official noted that DAU was reviewing  performance work statements and surveillance plans from professional  and management support contracts across DOD to identify good examples  of outcome-based performance objectives, standards, and measures. DAU  plans to launch a Web site in January 2010 that includes templates derived  from these examples for contracting and program officials across DOD to  tailor to their own needs. Additionally, this official noted that since  January 2009, DAU has offered a 4-day services acquisition workshop  tailored to individual acquisitions developed by program offices across  DOD. According to the DAU official, the workshop brings together key  acquisition personnel, from contracting officers to customers, to support  the development of new acquisition strategies before they are reviewed by  the military departments. The official added that by the end of the  workshop, the requiring activity has a draft performance work statement  and a quality assurance surveillance plan that meets performance-based  requirements.
section_title: DOD Efforts to Designate Trained Surveillance Personnel Show Progress, but Concerns Remain
paragraphs: DOD’s efforts to ensure that trained surveillance personnel are assigned to  monitor contractor performance on services contracts have shown  progress, though on a number of task orders personnel were not always  designated or trained in a timely fashion. Surveillance personnel are  required to be qualified by training and designated in writing. In response  to our and inspector general reports on continued shortcomings in DOD’s  contract surveillance practices, DOD issued guidance on December 6,  2006, requiring that surveillance personnel be properly trained and  designated before contract performance begins, and that properly trained  surveillance personnel are identified on active contracts for services.
At the time of our review, trained surveillance personnel were designated  to all 64 task orders. DOD personnel responsible for five of the seven  acquisitions we reviewed, however, did not maintain documentation on all  surveillance personnel assigned over the task orders’ entire period of  performance. DOD officials stated that in some cases additional personnel  may have been designated to the task orders, and that some personnel  may have received surveillance training on a date earlier than was  indicated in the contract files, but were unable to provide documentation.
In most cases, DOD was able to provide documentation of the first person  designated to conduct surveillance and the person assigned at the time of  our review. On the basis of this information, we found that surveillance  personnel were designated after contract performance began on 3 of the  37 task orders awarded after the issuance of the December 2006  guidance. In 1 of these 3 cases, the person was designated more than 90  days after performance began on the task order.
We also found that 61 of the 64 surveillance personnel designated on the  task orders we reviewed had received training. For 1 of the 3 instances  where personnel were not trained, a program official explained that the  person was recently assigned and had been notified of the training  requirements, but had not completed the training. In the 2 other instances,  DOD officials were not able to identify a reason for the lack of training.  For the 61 task orders with trained personnel assigned, 20 personnel had  not received training prior to beginning their assignments. Furthermore, in  3 of these cases, surveillance personnel did not receive the required  training until at least a year after they were assigned to monitor a  contractor’s performance.
The training that surveillance personnel received varied across and within  the military departments, ranging in duration from 2 hours to 1 week, and  included, among other things, reviewing training slides, completing an on- line course offered through DAU’s Web site, and completing an in-class  course tailored by the program office responsible for the acquisition.  There are no DOD-wide requirements for the content of surveillance  training and personnel we spoke with provided mixed feedback on how  well their training prepared them to conduct surveillance. Surveillance  personnel noted that training provided an adequate basis for conducting  their duties, but did not always provide enough instruction on how to  effectively oversee contractors, especially for those who had little to no  previous experience with assessing contractor performance. Several  personnel stated that the most useful information that training provided  was the contact information for personnel in the contracting offices that  surveillance personnel could speak with if they had questions. DPAP  officials acknowledged that the type and content of the training  surveillance personnel receive varied and indicated that DOD is  considering a certification system for these personnel that may include  both training and experience requirements.
Surveillance personnel identified a number of challenges that may affect  their surveillance duties, such as numerous contractors to oversee in  multiple locations and surveillance responsibilities being secondary to  primary duties. For example, on one task order for translation services in  Afghanistan, surveillance personnel were each responsible for monitoring  over 1,000 contractors dispersed throughout the country. An official who  oversaw this task order stated the ratio of surveillance personnel to  contractors was so large that it affected the government’s ability to assess  contractor performance. For a task order for translation services in  Cuba, the contracting officer’s letter of designation to the contracting  officer’s representative stressed the importance of on-site surve illance.  Nevertheless, we found that there was an 8-month period during which  surveillance personnel were absent from the site of contractor  performance. Despite the absence of on-site surveillance personnel,  contracting officials determined that the contractor should receive the full  award fee based upon performance reports submitted before and after this  8-month period. Several personnel from other commands we visited told  us they did not have sufficient time to focus on their surveillance  responsibilities in addition to their primary duties. Finally, surveillance  personnel at many of the commands we visited stated that they were  unaware of requirements to provide enhanced oversight of contractors  that closely support inherently governmental functions.
section_title: Recent Initiative May Improve DOD’s Insight into Issues Affecting Professional and Management Support Contracts
paragraphs: DOD is in the process of implementing additional processes to review  services contracts both prior to and after contract award, which may  provide additional insight into DOD’s management and oversight of  professional and management support contracts. These reviews are  intended to assess a number of issues that are not currently addressed by  DOD’s management review, including contractors that closely support  inherently governmental functions, implementation of performance-based  practices, and proper surveillance of contractors.
The National Defense Authorization Act for Fiscal Year 2008 required DOD  to issue guidance to implement an independent management review of  services contracts. In response, the Director of DPAP issued  memorandums in September 2008 outlining pre- and post-award peer  reviews and in February 2009 detailing criteria for the peer reviews for  services acquisitions with estimated values of $1 billion or more,  consistent with the threshold for reviewing proposed services  acquisitions. To do so, DOD convenes peer review teams that consist of  senior contracting leaders from across DOD as well as legal counsel to  work closely with the offices responsible for developing acquisition  strategies. This policy also required the military departments to establish  their own procedures to conduct peer reviews for service acquisitions  valued at less than $1 billion. As of October 2009, DPAP conducted 48 pre- award peer reviews on 31 different proposed supplies and services  acquisitions. DPAP also conducted three post-award reviews on approved  and ongoing services acquisitions, which included a review of task order  documents. Of the 51 reviews conducted by DPAP, four pre-award reviews  and one post-award review were conducted on three different professional  and management support services acquisitions.
The peer review process differs from DOD’s management reviews in a  number of areas that may provide opportunities for the department to  address key aspects of managing and overseeing professional and  management support contracts. Whereas the premise of the management  review is to assess and approve proposed acquisition strategies, the peer  reviews are conducted after strategies have been approved and are  intended to be advisory in nature. Peer reviews are designed to help:  ensure that contracting officers across DOD implement policy and  regulations in a consistent and appropriate manner;  improve the quality of DOD’s contracting processes; and  facilitate cross-sharing of best practices and lessons learned.
Currently, a peer review team summarizes the results of its review in a  memorandum provided to both the contracting office responsible for the  acquisition and DPAP. According to DPAP officials, DOD is still  determining how to share best practices and lessons learned from these  reviews with the department’s acquisition community.
While both the peer reviews and the management reviews contain pre- award components, the multiple phases of the peer reviews provide DOD  the opportunity to address additional issues and examine documents not  available during the management review (see fig. 1). For example, the pre- award management review of proposed services acquisitions occurs  before performance work statements and quality assurance surveillance  plans are developed. As a result, the management reviews do not include  an assessment of how performance-based practices are implemented and  whether proper contractor surveillance is conducted. Further, as  previously noted, DOD’s management review guidance does not require  department personnel to identify whether the services to be provided  closely support inherently governmental functions or how the risks  associated with contractors providing such services will be addressed. In  contrast, the pre-award peer reviews occur later in the acquisition cycle  and include the review of additional documents that may provide  reviewers an opportunity to recommend improvements to performance  work statements and surveillance plans.
Additionally, the peer review process provides for a post-award phase for  services that expands upon the management review requirements. The  post-award peer review has eight metrics that provide criteria that  reviewers use to evaluate ongoing acquisitions in terms of how contractor  performance is assessed, including the use of objective criteria; whether  surveillance personnel are appropriately staffed; and the extent of reliance  on contractors to perform tasks closely associated with inherently  government functions. Table 5 shows the various areas of focus of the  peer reviews.
Our review of the summary memoranda for the five peer reviews  conducted on professional and management support contracts found that  while the memoranda generally focused on business aspects, some  memoranda made recommendations related to performance-based  approaches and surveillance issues specific to the individual acquisition.  For example, a pre-award peer review team recommended that the  contracting office work with DAU to develop performance-based  statements of work. The peer review team for the one post-award review  recommended that the program office identify objective performance  measures and present them to the contractor. Another pre-award peer  review team recommended that the contracting office appoint surveillance  personnel prior to the award of a contract. None of the memoranda we  reviewed noted issues with contractors closely supporting inherently  governmental functions. DOD officials stated that these issues were  discussed, but that none appeared to warrant inclusion in the memoranda.
While DPAP has completed peer reviews on 34 individual supplies and  services acquisitions since September 2008, developing an approach that  will lead to the achievement of the peer review’s objectives enterprisewide  may prove challenging given the nature and volume of service contract  activity. Nearly 1,900 task orders were issued under the seven professional  and management support services acquisitions we reviewed, all of which  required a contractor to perform multiple and varying tasks. Furthermore,  we identified thousands of individual task orders associated with the  professional and management support services acquisition strategies  approved by the military departments from fiscal years 2004 through 2007,  all of which require management and oversight by the department. These  numbers are even greater when looking at all services acquisition  strategies approved by the military departments in that same time period  (see table 6).
section_title: Conclusions
paragraphs: DOD’s reliance upon contractor-provided professional, administrative, and  management services to support its missions makes effective management  and oversight of these contracts critical. Certain activities such as budget  preparation, acquisition planning, and policy development can create risks  that the government needs to consider and effectively manage. Of key  concern is the loss of government control over and accountability for  policy and program decisions. Nevertheless, DOD’s policies do not require  that DOD personnel include an assessment of these risks when their  proposals for contractor support are submitted for approval under DOD’s  management review process. While recent legislation and DOD’s  implementing guidance require that DOD consider whether to continue the  use of contractors for critical services, including conducting a cost  analysis if the situation warrants, these determinations and analyses are  largely disconnected from the acquisition review process. Consequently,  senior DOD leadership does not have the benefit of such analyses when  making strategic decisions on obtaining long-term, high-dollar-value  professional and management support.
Similarly, key decisions at the transactional level— such as to award a  contract or to issue a task order—are made with the recognition that DOD  is dependent on contractors to support its missions and operations.  Despite this dependency, DOD officials generally did not consider whether  contractors may be unduly or inappropriately influencing government  decision making. Further, while these services were often acquired  through performance-based approaches, such efforts were hindered by  DOD’s use of broad statements of work and the use of performance  measures established in the base contract rather than attempting to  measure the specific services being provided under the task order. Within  DOD’s acquisition community, there is widespread recognition that  developing outcome-oriented measures is particularly difficult for  professional and management support contracts. DOD has efforts under  way to help develop better outcome-oriented measures for professional  and management support contracts, but it is too soon to know whether  this effort will prove successful. Perhaps the most critical tool in assessing  contractor performance is having properly trained personnel in sufficient  numbers to effectively monitor contractor performance. While  improvements were evident, lapses in designating such personnel, in  particular during the initial stages of the contract, continue to expose DOD  to an increased risk of poor contractor performance.
DOD consideration, at both the strategic and transactional levels, of the  risks of using contractors to closely support inherently governmental  functions can help improve the context for successful professional and  management support outcomes. DOD’s peer review process is beginning  to assess this issue just prior to and then after contract award, but with  only a handful of reviews performed on such contracts, it is too early to  gauge whether the process will be successful in encouraging DOD  personnel to address these issues across the range of DOD’s services  contracts. Having similar information provided at a level of detail  appropriate for when senior DOD and military department leadership  review proposed acquisition strategies would inform decision makers and  engender more proactive consideration earlier in the acquisition cycle. As  DOD gets closer to awarding contracts or issuing task orders for specific  services, risks move from potential to the more tangible. Reducing the  possibility that DOD would enter into a contractual arrangement that  exposes it to unintentional and undesired consequences requires that DOD  personnel consider—based on the facts and circumstances of a particular  acquisition—whether such risks are present and, if so, how best to  mitigate them. The fact that DOD program and contracting personnel we  contacted were generally unaware of the long-standing requirement to  provide greater scrutiny and enhanced oversight on services closely  supporting inherently governmental functions underscores the need to  address these problems at multiple levels and manners.
section_title: Recommendations for Executive Action
paragraphs: To better inform acquisition decisions, assist DOD personnel in  performing their management oversight responsibilities, and improve  DOD’s surveillance of services contracts, we recommend that the  Secretary of Defense take the following four actions:  revise documentation requirements for DOD’s current management  review to include information on the extent to which services to be  provided will closely support inherently governmental functions as  well as the consideration given to using DOD civilian employees to  perform such functions;  require before the award of any contract or issuance of task order for  services closely supporting inherently governmental functions that  program and contracting officials consider and document their  assessment of the unique risks of these services and the steps that  have been taken to mitigate such risks;    develop guidance to identify approaches that DOD should take to  enhance management oversight when contractors provide services  that closely support inherently governmental functions; and    direct the military departments to review their procedures to ensure  that properly trained surveillance personnel have been assigned prior  to and throughout a contract’s period of performance.
section_title: Agency Comments and Our Evaluation
paragraphs: DOD provided written comments on a draft of this report. DOD concurred  with the four recommendations and also identified a number of actions  that would be taken to address them. DOD acknowledged the need to  continually refine its policies and procedures regarding the management  of support contracts. DOD noted that while it intended to decrease  funding for contracted support and scale back the use of contractors, DOD  will continue to rely on service contracts to support its mission, making  the effective management of professional, administrative, and  management support contracts critical. DOD also provided technical  comments, which were incorporated as appropriate. DOD’s comments are  reprinted in appendix III.
We are sending copies of this report to the Secretary of Defense; the  Secretaries of the Air Force, Army, and Navy; the Administrator of the  Office of Federal Procurement Policy; and interested congressional  committees. In addition, the report will be made available at no charge on  GAO’s Web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this report. GAO staff who made major contributions to this report are  listed in appendix III.
section_title: Appendix I: Scope and Methodology
paragraphs: To determine whether the Department of Defense’s (DOD) policies and  actions to improve its management of services contracts addressed issues  affecting professional and management support contracts, we examined  (1) the extent to which DOD considered the risks associated with  contractors closely supporting inherently governmental functions at key  acquisition decision points;(2) how DOD was implementing performance- based acquisitions practices, such as identifying requirements in terms of  expected and measurable outcomes; (3) the extent to which DOD  designated trained surveillance personnel; and (4) whether recent actions  to implement a peer review process may improve DOD’s management and  use of such contracts.
To assess the extent to which DOD considered the risks associated with  contractors closely supporting inherently governmental functions at key  acquisition decision points, we reviewed relevant provisions of the  National Defense Authorization Acts for fiscal years 2002 through 2009  that pertained to DOD’s acquisition of services. We also reviewed guidance  issued by DOD in May 2002 and October 2006, as well as DOD Instruction  5000.02, Operation of the Defense Acquisition System, reissued in  December 2008, which collectively established DOD’s management review  processes to identify the risks that should be considered during these  reviews. Further, we reviewed Office of Federal Procurement Policy Letter  93-1, Management and Oversight of Service Contracting and Federal  Acquisition Regulation (FAR) requirements for the management and  oversight of contractors that closely support inherently governmental  functions. We also reviewed DOD Instruction 1100.22, Guidance for  Determining Workforce Mix and DOD’s May 2009 in-sourcing guidance,  In-Sourcing Contracted Services—Implementation Guidance, to  determine how personnel should consider and document the risks of  contractors performing activities that closely support inherently  governmental functions and assess the appropriate mix of DOD civilian,  military, and contractor personnel. We interviewed representatives from  the Office of Defense Procurement and Acquisition Policy (DPAP), and  representatives from each of the military departments that are responsible  for implementing these policies, guidance, and reviews to identify how  these risks are accounted for prior to approval of the proposed acquisition  strategy.
To assess how these risks were addressed under DOD’s management  review for specific services acquisitions, we reviewed a DOD-provided list  of 102 services acquisition strategies that were reviewed and approved by  the Air Force, Army, or Navy from fiscal years 2004 through 2007. We  obtained information from the military departments on the contracts that  had been awarded after these 102 strategies had been approved. Using this  information and data derived from the Federal Procurement Data System– Next Generation, we determined that the military departments had  awarded 361 contracts and issued 13,650 task orders from these 102  acquisitions during fiscal years 2004 through 2007. We then identified  product service codes associated with these contracts and used the  Federal Procurement Data System–Next Generation to determine the  number of contract actions and obligations for professional and  management support services. We found that 32 of these acquisitions, with  almost $15 billion in total combined obligations from fiscal years 2004  through 2008, included contracts for professional and management  support. From these 32 acquisitions, we selected 7 acquisitions based on  such factors as the percentage of obligations that were made for  professional and management support services, the specific types of  services acquired, and the agency awarding the contract. The 7  acquisitions we selected had over $4.3 billion in total combined obligations  from fiscal years 2004 through 2008 (see table 7 for more information on  these acquisitions).
The seven acquisition strategies we reviewed were approved under DOD’s  May 2002 acquisition of services policy, which required the Under  Secretary of Defense for Acquisition, Technology, and Logistics to review  acquisitions with an expected value of over $2 billion and for each of the  DOD components, which includes the military departments, to review  those acquisitions that were under that threshold. None of the acquisition  strategies we selected were approved after DOD issued its October 2006  acquisition of services policy, which lowered the dollar thresholds for  management reviews. The substance of the management reviews remained  largely unchanged, incorporating a few additional, specific, acquisition  strategy requirements, such as the inclusion of any required waivers and  top-level discussion of source selection processes that were not significant  to the objectives of our review.
To assess how such risks were addressed at the contract or task order  level, we used data from the Federal Procurement Data System–Next  Generation to determine the number of task orders that had obligations of  $500,000 or more that were issued from fiscal years 2004 through 2007  from each of these seven acquisitions. From that list, we randomly select  10 task orders for each acquisition, with the exception of the INSCOM  Linguistics Part 2 acquisition for which we selected all four task orders  that had been issued as of September 2007 that exceeded this threshold.  Overall, we selected 64 task orders, which ranged from $530,000 to        $227 million in obligations, for review. We did not review acquisitions  approved after fiscal year 2007 since our analysis indicated that it was  often a year or more from the time that the acquisition strategy was  approved to the time when task orders were actually issued. For each of  the task orders, we reviewed the acquisition strategy, base contract, task  orders, statements of work, and other documentation supporting the need  to acquire contract support and any risk assessments prepared. We also  interviewed program and contracting officials who managed these  acquisitions to obtain information concerning why these services were  contracted for, the risks that were considered, and any additional steps  that had been taken to enhance oversight of the contractors.
We assessed the reliability of the Federal Procurement Data System–Next  Generation to identify acquisitions and to select task orders that were  within the scope of our review by verifying (1) the contract and task order  identification numbers; (2) the contract award date; (3) that the task  orders associated with the acquisitions were for professional and  management support services; and (4) that the task orders had obligations  exceeding $500,000. On the basis of this assessment we determined that  the data were sufficiently reliable for the purposes of this review.
To assess how DOD implemented performance-based acquisitions  practices on contracts for professional and management support, we  reviewed relevant provisions in the FAR and DOD guidance. We  interviewed DOD and military department officials responsible for  reviewing and approving services acquisitions to identify how these  reviews addressed the implementation of performance-based practices.  We reviewed performance work statements from each of the 64 task  orders to assess whether contract requirements and performance  measures were consistent with performance-based guidance, such as  whether contract requirements were measurable and outcome based. We  also analyzed documentation to determine how contractor performance  was measured. We interviewed contracting and program officials  associated with these acquisitions to identify how contract requirements  and performance measures were developed. Finally, we interviewed DPAP  officials and a representative from the Defense Acquisition University to  obtain information on efforts to develop additional DOD guidance for  implementing performance-based services acquisitions.
To assess the extent to which DOD designated trained surveillance  personnel, we reviewed the Defense Federal Acquisition Regulation  Supplement and DOD policies and procedures to identify the department’s  surveillance and training requirements. We then analyzed surveillance  personnel appointment letters and training documentation associated with  each of the 64 task orders to determine whether these requirements were  met. We also interviewed DOD officials who were designated as  surveillance personnel on one or more of the task orders we reviewed to  obtain information on their training and responsibilities.
To identify how actions to implement additional reviews of services  acquisitions may improve DOD’s management and use of services  contracts, we reviewed provisions contained in the National Defense  Authorization Act for Fiscal Year 2008 that required DOD to establish an  independent management review process. We review memoranda issued  by DOD in September 2008 and February 2009 that provided guidance on  the scope of these reviews, which DOD refers to as peer reviews. To  obtain information on how peer reviews differ from the management  reviews, we spoke with officials from DPAP and the military departments  responsible for these reviews. We also obtained copies of the memoranda  summarizing the findings and recommendations of the five peer reviews  performed on professional and management support contracts as of  September 2009.
We conducted this performance audit from July 2008 through November  2009 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.
section_title: Appendix II: Examples of Inherently Governmental and Approaching Inherently Governmental Functions
paragraphs: Federal Acquisition Regulation section 7.503 provides examples of  inherently governmental functions as well as services or actions that are  not inherently governmental, but may approach being inherently  governmental functions based on the nature of the function, the manner in  which the contractor performs the contract, or the manner in which the  government administers contractor performance. These examples are  listed in tables 8 and 9 below.
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: In addition to the individual named above, key contributors to this report  were Timothy DiNapoli, Assistant Director; Gary Guggolz; Justin Jaynes;  Christopher Mulkins; Thomas Twambly; Richard Winsor; Arthur James,  Jr.; Julia Kennon; Susan Neil; and Noah Bleicher.