Defense Contracting: DOD Needs Better Information on Incentive Outcomes
Published: 2017-07-11T00:00:00

Fast Facts:
In fiscal year 2015, DOD spent $274 billion on contracts for products and services. Some of this money was used on incentive contracts to achieve specific outcomes, such as lower costs or tighter time frames.
For these contracts, we found that DOD has increased its use of incentive fees (which objectively measure contractor performance toward set goals) and decreased its use of award fees (which are more subjective).
However, DOD does not consistently collect data on the outcomes of these incentive contracts—so DOD can't determine if they are improving contractor performance. We recommended that DOD collect and analyze relevant contract data.

Highlights:
section_title: Why GAO Did This Study
paragraphs: In fiscal year 2015, DOD obligated $274 billion on contracts for products and services, a portion of which was for contracts that used incentive and award fee provisions—or incentive contracts—intended to improve cost, schedule, and technical performance outcomes. Work by GAO and others has shown that such contracts, when not well managed, can lead to unnecessary costs shouldered by the American taxpayer. Beginning in 2010, DOD made regulatory and policy changes related to incentives.
GAO was asked to review DOD's use of incentives. This report (1) identifies steps DOD has taken to improve its use of incentive contracts since 2010, and (2) assesses the extent to which selected DOD incentive contracts achieved desired acquisition outcomes.
To conduct this work, GAO reviewed relevant federal and DOD guidance; analyzed DOD obligations and new contract award data for fiscal years 2005 through 2015, before and after regulatory and policy changes; and analyzed a nongeneralizable sample of 26 contracts and task orders that contained incentives and 9 contract actions providing for award fees that were awarded between fiscal years 2011 and 2015 and reported as completed by the end of fiscal year 2015 to assess contract outcomes.
section_title: What GAO Found
paragraphs: Since 2010, the Department of Defense (DOD) has made changes to its regulations, policies, and guidance and taken other steps to improve its use of incentive contracts. DOD has promoted greater use of objective incentives—which measure contractor performance toward predetermined targets using a formula—through incentive fee contracts, partly to better motivate cost control. These changes are reflected in DOD's increased use of incentive fee contracts and decreased use of award fee contracts, which involve fees paid based on a more subjective evaluation of contractor performance and have not always been linked to acquisition outcomes (see figure).
DOD expects to achieve cost objectives on 15 of the 21 incentive fee contract actions that GAO reviewed and for which costs could be assessed. GAO could not assess cost performance on five additional selected incentive fee contracts because comparable cost estimates were not available. Across the 21 incentive fee contract actions, estimated costs for the incentivized portions were about 5 percent below target costs. Schedule and technical performance incentives mostly resulted in good outcomes. In two cases, however, although the contractor met specific schedule and technical performance goals, overall outcomes were either unsatisfactory or not yet determined. In the nine award fee contracts GAO reviewed, consistent with prior GAO recommendations, DOD did not allow unearned fees to be earned in a subsequent period, and GAO did not find evidence of award fees paid for unsatisfactory performance. Federal regulations require DOD to collect and evaluate information on incentives. In 2015, DOD stopped its previous effort to manually collect data twice a year on incentives valued at more than $50 million, which was burdensome and collected information that DOD did not use, according to DOD officials. GAO's review of current DOD systems found that they provide some useful data but do not allow DOD to determine how well incentives are achieving desired cost, schedule, and performance outcomes. Without such information, DOD may be disadvantaged in establishing incentive arrangements that achieve intended results.
section_title: What GAO Recommends
paragraphs: DOD should identify the type of information on incentives needed and collect and analyze relevant data to assess outcomes. DOD agreed to do so and stated it will take actions in fiscal year 2018 to address GAO's recommendation.

section_title: Letter
paragraphs: In fiscal year 2015, the Department of Defense (DOD) obligated  $274 billion on contracts for products and services. DOD obligated about  18 percent of those dollars on contracts that included incentive or award  fee provisions. Such contracts, broadly referred to as incentive contracts,  provide defense contractors the opportunity to earn fees or profits based  on their performance. DOD has looked to incentive contracting as one of  the ways it can promote desired acquisition outcomes, such as keeping  costs low, delivering a product or service on time, or achieving certain  technical results. When structured correctly, incentive contracts can more  closely align contractor motivations with the government’s desired  outcomes. Prior work by GAO and others, however, has shown that if not  well managed, incentive contracts can lead to unnecessary costs  shouldered by the American taxpayer. For example, in 2005, we  estimated that DOD paid out $8 billion in award fees that were not linked  to acquisition outcomes. In 2009, we estimated that DOD could save  over $450 million by limiting second chances at unearned fees.  Beginning in 2010, DOD made regulatory and policy changes aimed at  improving its use of incentive contracts.
You asked us to review DOD’s use of incentive contracts. This report (1)  identifies steps DOD has taken to improve its use of incentive contracts  since 2010, and (2) assesses the extent to which selected DOD incentive  contracts achieved desired acquisition outcomes.
To identify the steps DOD has taken to improve its use of incentive  contracts since 2010, we reviewed relevant legislation and provisions  within the Federal Acquisition Regulation (FAR) and memoranda issued  by the Office of Management and Budget (OMB). We also identified and  reviewed changes to DOD regulations, policies, and guidance regarding  the use of incentive contracts. To identify changes in DOD’s use of these  contract types over time that may have corresponded with regulatory and  policy changes, we analyzed data from the Federal Procurement Data  System-Next Generation (FPDS-NG) on obligations by contract type for  fiscal years 2005 through 2015. We reported our findings in constant  fiscal year 2015 dollars, adjusted for inflation using the fiscal year gross  domestic product price index. To assess the reliability of the FPDS-NG  data, we conducted electronic testing of the data and traced data from a  selection of contracts to contract sources to verify their accuracy. We  determined that the data were sufficiently reliable for the purposes of  analyzing and reporting general trends in DOD’s obligations for incentive  contracts from fiscal years 2005 through 2015. We also interviewed DOD,  military department, and Defense Acquisition University (DAU) officials  about efforts to improve the use of incentive contracts.
To determine the extent to which incentive provisions in selected DOD  contracts achieved desired cost, schedule, and technical performance  outcomes, we assessed a nongeneralizable sample, drawn from FPDS- NG data, of incentive contracts and orders that DOD awarded between  fiscal years 2011 and 2015 and that were reported as completed by the  end of fiscal year 2015. We chose these timeframes so as to select  awards that (1) were made after the regulatory and policy changes which  started in 2010, and (2) had completed performance by the time of our  review. These parameters yielded 38 contracts and more than 4,200  orders. We initially selected all 38 contracts and a subset of 15 orders  that (1) included different incentive types under the same base contract,  (2) were used to purchase similar products and services, and (3) reflected  a range of dollar values. These parameters yielded an initial sample of 53  contract actions (contracts and orders). Eighteen of the 53 actions were  dropped from our sample either because the contracts were miscoded in  FPDS-NG and were not incentive contracts based on our review of  contract documents (6), were terminated before performance began (1),  or were actually not yet complete (11). Consequently, we reviewed a total  of 35 contracts and orders representing incentive contract types (see  table 1). Because we used a nongeneralizable sample of contracts and  orders, results from this sample cannot be used to make inferences about  all incentive contracts and orders that DOD awarded.
In total, we reviewed 5 contract actions from the Air Force, 7 from the  Army, and 23 from the Navy. For each selected contract and order, we  collected relevant documentation, such as the initial contract or order,  modifications, statements of work, determination and findings  memoranda, award fee plans, and performance evaluations. We  interviewed contracting officials and obtained their written input to clarify  and collect additional information as needed. To supplement our  understanding of how incentive provisions helped achieve acquisition  outcomes, we interviewed two contractors about the effectiveness of  incentive provisions in motivating performance toward desired outcomes.  To identify contractors to interview, we focused on a sub-selection of  contracts with multiple incentives to obtain perspectives on how the  incentives interacted with respect to the contractors’ performance. We  also reviewed documents and interviewed DOD officials about relevant  data systems and processes used to collect and analyze contract and  program data, such as the Contract Business Analysis Repository  (CBAR) and peer reviews, to understand how these systems may provide  some insight into DOD’s use of incentive contracts. Appendix I contains  more detail on our objectives, scope, and methodology.
The performance audit upon which this report is based was conducted  from January 2016 to May 2017 in accordance with generally accepted  government auditing standards. Those standards require that we plan  and perform the audit to obtain sufficient, appropriate evidence to provide  a reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives.  DOD deemed that certain information in the draft report related to  contract costs and performance was sensitive and must be protected  from public disclosure. We subsequently worked with DOD from May  2017 to July 2017 to revise our presentation of this information and  prepare this report for public release.
subsections: section_title: Background
subsections: section_title: Contract Types
paragraphs: According to the FAR, a wide selection of contract types is available to  the government and contractors to allow flexibility in acquiring a variety of  products and services. Contract types vary according to the degree and  timing of the contractor’s responsibility for the costs of performance and  by the amount and nature of the profit incentive offered to the contractor  for meeting or exceeding specified goals. Contract types are grouped into  two broad categories: fixed-price or cost-reimbursement contracts. Within  these categories, the specific types range from firm-fixed-price (FFP), in  which the contractor has full responsibility for the costs of performance  and the resulting profit or loss, to cost-plus-fixed-fee (CPFF), in which the  contractor has minimal responsibility for the costs of performance and the  fee is a fixed dollar amount, established as a percentage of the estimated  target cost at the start of the contract. In between these are incentive  contracts, in which the contractor’s responsibility for costs and the profit  or fee incentives offered are tailored to performance uncertainties.
The FAR also notes that incentive contracts are appropriate when a firm- fixed-price contract is not, and the required items can be acquired at  lower costs and possibly with improved delivery or technical performance  by tying fee or profit to the contractor’s performance. Incentive and award  fee provisions can be used together in the same contract, but each uses  a different approach with respect to how performance is assessed and  how fees or profits are determined.
Incentive fees—For contracts with incentive fees or profits, the  amount of fee or profit payable is related to the contractor’s  performance. Incentive fees or profits generally focus on cost control,  though they may be used to motivate performance toward specific  delivery (e.g., schedule) targets or technical goals. Incentive fees or  profits involve an objective evaluation by the government through a  process that is generally less administratively burdensome than award  fee evaluations. The government usually applies a fee- or profit- determination formula that is specified in the contract to evaluate  performance at the end of the contract or at program milestones. The  formula may include a target cost, a target profit or fee, a ceiling price,  and a profit- or fee-adjustment formula, sometimes referred to as a  share ratio for fixed-price incentive (FPI) contracts (see figure 1).
In the hypothetical example above, if the contractor incurred $100 in  costs, the contractor would receive $10 in profit, and the government  would pay the target price of $110. If the contractor kept costs under  the $100 target cost, it would split cost savings equally with the  government and receive a larger profit; if the contractor exceeded the  cost target, it would share the additional costs with the government up  to the ceiling price but earn a smaller profit. At ceiling, the contractor  earns no profit, and the contractor is responsible for any costs  incurred above the ceiling price.
A cost-plus-incentive-fee (CPIF) contract reimburses the contractor for  its allowable costs, but still uses a formula of total allowable costs to  target costs to determine fee and includes a target fee instead of a  target profit. A CPIF contract also has a minimum fee—the lowest fee  the contractor may receive when total allowable costs exceed target  costs—and maximum fee—the highest fee the contractor may earn  when total allowable costs are less than target costs, and, unlike an  FPI contract, there is no ceiling price.
Award fees—Award fees typically emphasize multiple aspects of  contractor performance in areas that are more subjectively assessed,  such as the contractor’s responsiveness, technical ingenuity, or cost  management. From the government’s perspective, development and  administration of award fee contracts often involve substantially more  effort over the life of a contract than incentive fee contracts, requiring  government officials, through an award fee evaluation board, to  conduct periodic evaluations of the contractor’s performance against  specified criteria and to make recommendations on the amount of fee  to be paid. Criteria are specified in an award fee plan, which  contracting officials may revise from one evaluation period to another  to redirect contractor emphasis. Following the award fee evaluation, a  fee-determining official makes the final decision about the amount of  fee paid to the contractor.
Table 2 identifies the range of incentive contract types and their  appropriate use based on acquisition regulations.
section_title: Prior Work
paragraphs: We have previously identified issues with the use of incentive and award  fees that called into question whether they were used effectively to  achieve their intended purpose. In our December 2005 review of incentive  and award fee contracts, we found that award fees were generally not  linked to acquisition outcomes, and that DOD had paid an estimated $8  billion in award fees regardless of outcomes. In addition, we estimated  that in 52 percent of the award fee contracts, DOD moved unearned  award fees from one evaluation period to a subsequent period—a  practice referred to as “rollover”—which provides contractors at least a  second chance to earn fees after failing to perform well enough to earn  them initially. We also found that DOD had not compiled data, conducted  analyses, or developed performance measures to evaluate the  effectiveness of incentive and award fees. We recommended that DOD  apply more outcome-based award fee criteria, pay award fees only for  above satisfactory performance, issue guidance on the appropriate use of  rollover, develop a mechanism for capturing incentive and award fee data  within existing data systems, and develop performance measures to  evaluate the effectiveness of incentive and award fees at improving  contractor performance and achieving desired outcomes. DOD concurred  with two of these recommendations, and partially concurred with our  recommendations to only pay award fees for above satisfactory  performance, collect incentive and award fee data, and develop  performance measures to evaluate the effectiveness of incentive and  award fees. DOD implemented all but one of these recommendations— paying award fees only for above satisfactory performance—though a  provision was later added to the FAR prohibiting payment of award fees  for below satisfactory performance.
In our May 2009 review of 50 DOD contracts containing award fees, we  found that DOD had made progress toward minimizing payments of  award fees for unsatisfactory performance, limiting overpayment for  satisfactory performance, and reducing the number of programs that used  rollover. DOD, however, still struggled to use data collected on award fee  contracts to evaluate their effectiveness. We did not make new  recommendations in these areas.
Most recently, in a March 2017 report on selected FPI contracts awarded  by the Navy for new ship construction, we found the Navy often structured  the contracts such that it absorbed more cost risk than DOD’s regulation  suggests, indicating it may not achieve the expected benefits of using the  FPI contract type. For example, we found that 8 of 11 ships delivered  under the contracts reviewed experienced cost growth. We recommended  that DOD conduct a portfolio-wide assessment of the Navy’s use of  additional incentives on FPI contracts across shipbuilding programs. DOD  concurred with our recommendation.
section_title: Government-wide Changes
paragraphs: In 2007, OMB issued government-wide guidance highlighting preferred  practices for incentive contracting and directing agencies to review and  update their acquisition policies. In 2009, the FAR was revised to  implement legislative provisions and OMB’s guidance on the appropriate  use of incentive contracts. These changes addressed some of the  issues that we identified in 2005 and 2009 relative to the use of award  fees. The FAR now prohibits rollover of unearned award fees from one evaluation period  requires award fees to be linked to cost, schedule, and technical  performance acquisition objectives;  restricts payment of award fees in instances of unsatisfactory  contractor performance; and  requires agencies to collect relevant data on incentive and award fee  payments and evaluate the effectiveness of these contract types in  achieving desired outcomes.
section_title: DOD Has Changed Regulations, Policies, and Practices to Improve Its Use of Incentive Contracts, in Particular by Emphasizing Use of Objective Incentives
paragraphs: Since 2010, DOD has taken steps to improve its use of incentive  contracts—often beyond what is required by the FAR—by revising the  Defense Federal Acquisition Regulation Supplement (DFARS), instituting  its Better Buying Power initiative, and developing new guidance and  training courses. In particular, DOD has emphasized the use of objective  incentives through FPI and CPIF contracts rather than award fees  whenever possible, in part to better motivate contractors to control costs.  These efforts are reflected in DOD’s reported use of incentive contracts  since 2010, which indicate a substantial growth in obligations for incentive  fee contracts and a corresponding decrease in obligations for award fee  contracts.
subsections: section_title: DOD Has Updated Regulations, Policy, and Guidance
paragraphs: DOD made changes to the DFARS and its accompanying Procedures,  Guidance, and Information in recent years intended to support  appropriate use of incentive contracts. For example, DOD made the  following updates to the DFARS in 2011:
Directed contracting officers to utilize objective criteria—associated  with incentive fee contracts—to the maximum extent possible for  measuring contract performance. DOD noted concerns that award  fee contracts have a limited ability to motivate contractors to control  costs, and that there had been instances in which award fee  payments were not consistent with outcomes.
Directed contracting officers to give particular consideration to FPI  contracts, especially for acquisitions moving from development to  production or in contracts for which previous FFP contract costs had  varied by more than 4 percent from negotiated costs. By looking at  historical pricing and contract performance data, DOD officials stated  they determined that in some FFP contracts, actual costs have come  in noticeably lower than negotiated costs (e.g., at 4 percent or more),  indicating that costs are not stable or that the government may not  have negotiated a good deal when it awarded the contract.
Directed contracting officers to include a contract clause prohibiting  the payment of award fees when a contractor’s performance is rated  below satisfactory, as required by the FAR, which emphasizes that  the amount of award fee paid should correspond with the contractor’s  performance.
DOD released memoranda between 2010 and 2015 through its Better  Buying Power initiative, which focused, in part, on the use of incentive  contracts. The Better Buying Power memoranda established a  preference for FPI contracts and advised contracting officers to increase  the use of this contract type, when appropriate, such as early in  production and in single-source production where year-over-year price  improvement can be rewarded. DOD acknowledged that some officials  interpreted the first memorandum to mean that FPI contracts should be  used to the exclusion of other contract types. As a result, subsequent  memoranda advised officials to consider the full range of contract types  and employ the appropriate type, while giving particular consideration to  FPI and CPIF contracts. In addition, Better Buying Power called to limit  the use of award fee contracts for services, noting that services  acquisitions should be predisposed to FFP, CPFF, or CPIF. Better Buying  Power also instructed the military departments to provide a justification of  contract type for proposed contracts over $100 million for major  programs.
Better Buying Power also called for new DOD guidance on selecting  contract types and employing incentive contracts. Subsequently, in April  2016, Defense Procurement and Acquisition Policy (DPAP) released  Guidance on Using Incentive and Other Contract Types. This guidance  provides direction on selecting contract types, structuring appropriate  incentive arrangements, and negotiating target costs and share ratios  with contractors. It explains that, when appropriately structured, incentive  contracts can allow the government to share in cost savings, focus the  contractor on the areas that are important to the government, and provide  the government with valuable data on actual costs incurred. It also  reinforces some key updates to DFARS, such as emphasizing that  objective criteria must be used whenever possible to measure contract  performance.
DOD’s DAU is developing two new continuous learning courses to  reinforce concepts reflected in the April 2016 guidance. According to  DAU, all contracting personnel involved in using incentive arrangements  will be encouraged to take these courses, though they are not required for  Defense Acquisition Workforce Improvement Act certification, a process  DOD uses to determine that acquisition officials meet certain standards.  One course, Understanding Incentive and Other Contract Types, is  currently available and provides training on how to align contract types  and incentives with acquisition outcomes. The second course is expected  to be available in August 2017 to provide training on appropriate use of  advanced incentive concepts, such as quantifying cost, schedule, and  performance risks and incorporating that information into decisions on  contract incentives. DAU also provides other courses with elements  addressing aspects of incentive contracting. Program officials—who can  be involved in selecting contract types and structuring incentives,  according to a senior DPAP official—also undergo some training on  contract types and incentives through DAU training courses.
Further, to inform selection of contract type, contract negotiations, and  projections of program and contract costs, DPAP and the Director of  Defense Pricing have encouraged collaboration among contracting  officials, program officials, and cost analysts to collect and share cost  information with one another. DOD has also required contracting officers  to share information through CBAR—which captures information to assist  contracting officers in preparing for negotiations with contractors, such as  contractor business systems status and compliance with cost accounting  standards. Contracting officials use CBAR to upload and share contract  negotiation documents, which, according to DOD, can help contracting  officials benefit from others’ experiences, particularly when negotiating  with the same contractor.
Finally, DOD has used independent management reviews, or peer  reviews, to advise contracting officers on selecting the appropriate  contract type and structuring and negotiating contract incentives, among  other topics. The Director of Defense Pricing and DPAP lead peer  reviews to ensure that certain high-dollar acquisitions are carried out in  accordance with applicable laws, regulations, and policies. According to  senior DOD and military department officials, peer reviews provide an  opportunity for DPAP leadership to share knowledge about key  contracting decisions. For example, one peer review advised the  contracting officer to consider whether increasing the available fee under  a CPIF contract would reduce costs to the government by incentivizing  greater cost control from the contractor.
DOD’s focus on incentive contracts is evident in many major defense  acquisition program contracts. Based on information provided by DOD, as  of January 2017 the department is using incentive contracts—either FPI  or CPIF—on 65 of 78 major defense acquisition programs. The Army’s  Patriot Advanced Capability-3 (PAC-3) program—which provides mobile  defense against short-range ballistic missiles and other threats—offers an  example of DOD’s use of FPI contracts in particular. According to DOD,  recent contracts for PAC-3 reflect the department’s consideration of an  FPI contract type for programs that previously used FFP contracts in  which actual costs varied significantly from negotiated costs. Through an  analysis of actual costs on prior production contracts for the PAC-3  program, DOD determined that the prime contractor was underrunning  negotiated costs—that is, actual costs were lower—in these contracts in  amounts ranging from 8 to 15 percent, triggering consideration of FPI  type based on defense regulations. Consequently, after reviewing  historical pricing data and applying lessons learned, Army contracting  officials stated they were able to negotiate FPI contracts for missile  production over fiscal years 2014 through 2016 that were a total of $860  million lower than the contractor’s initial proposals. Army officials stated  that they plan to use cost information gathered through these contracts to  determine the appropriate contract type and inform negotiations for future  production contracts. Because these contracts are ongoing, however,  actual costs and other outcomes have not yet been determined, though  current costs for the fiscal year 2014 contract indicate it may result in a  cost overrun, in which actual costs exceed target costs.
section_title: Data on Obligations for Incentive Contracts Are Largely Consistent with Changes in Policy
paragraphs: Our analysis of data from FPDS-NG found that obligations for incentive  contracts ranged from about 13 to 22 percent of DOD’s total annual  contract obligations from fiscal year 2005 through fiscal year 2015. In  fiscal year 2015, incentive contracts accounted for nearly 18 percent of  DOD’s total annual contract obligations. Consistent with DOD’s emphasis  on using incentive fee contracts and decreasing the use of award fee  contracts, our analysis of DOD’s reported contract obligations from fiscal  years 2005 through 2015 shows a shift toward using incentive fee  contracts (see figure 2).
More specifically, our analysis found that the changes were largely driven  by increased obligations for FPI contracts, and decreased obligations for  CPAF contracts (see figure 3).
Among specific products and services for which incentive contracts were  used, we found the mix varied by contract type. Specifically:
FPI contracts were mostly used to purchase products. In fiscal year  2015, three product categories accounted for almost 90 percent of FPI  obligations: aircraft, ships/submarines, and land vehicles; weapons  and ammunition; and sustainment supplies and equipment.
CPIF contracts were used for a mix of products and services: aircraft,  ships/submarines, and land vehicles; research and development;  equipment related services; weapons and ammunition; and  knowledge based services—such as engineering, program  management, and education and training—accounted for roughly  three quarters of CPIF obligations in fiscal year 2015.
Award fee contracts were mostly used for services, including facility  related; transportation; and equipment related services.
section_title: DOD Expects to Achieve Cost Objectives on Selected Contracts, but Lacks Full Information on Outcomes for Incentive Contracts
paragraphs: DOD expects to achieve positive cost outcomes—with contractors’  estimated costs coming in lower than target costs—for most of the 21  selected incentive fee contract actions we were able to measure. Overall,  the estimated costs for the incentivized portions of these selected  contract actions were about $30 million—or about 5 percent—below  target costs. Among the contract actions we reviewed, schedule and  technical performance incentives were included in multiple-incentive  contracts. Officials reported good outcomes overall for the contracts with  multiple incentives that we reviewed, but we could not isolate the effects  of any particular schedule or technical performance incentive. The nine  award fee actions in our sample, which were mainly used to procure  services, did not allow for rollover and payments for unsatisfactory  performance—both of which were issues we found in our prior work. DOD  collects some information on incentive contracts, but it generally has not  assessed the extent to which particular contract types or incentive  arrangements have achieved cost, schedule, or technical performance  goals.
subsections: section_title: Estimated Costs Were Below Target for Most of the Incentive Fee Contract Actions That We Could Measure
paragraphs: DOD expects contractors to underrun cost targets for incentive fee or  profit provisions in 15 of the 21 cases for which we could compare target  and estimated costs. Overall, these contract actions were expected to  underrun target costs on the incentivized portions of the contracts by  about 5 percent, amounting to $30 million in expected savings shared  between the government and contractors on the 21 contract actions.  These 21 contracts had a total value of about $957 million. The contract  actions that were expected to underrun target costs represented  procurements of both goods and services, and were a mixture of FPI and  CPIF actions (see table 3).
In two cases, contracting officials raised potential benefits of using an FPI  contract that extended beyond underrunning target costs in a single  contract by using knowledge of cost efficiencies to decrease prices from  one production lot to the next. For example, contracting officials for one  Air Force missile program determined that an engine produced by a  subcontractor was a cost driver, but had little insight into the actual costs  for this component. The Air Force created a separately-priced FPI line  item with unit pricing for the engines. Using an FPI contract type required  the contractor to provide cost data, a requirement that was to be  incorporated in turn into the prime contractor’s FPI agreement with its  subcontractor. Officials reported that through this arrangement, they  obtained insight into subcontractor costs and were able to obtain cost  savings in subsequent lots of more than $104,000 per engine.
We were unable to compare actual or estimated costs and target costs for  5 of the 26 FPI or CPIF actions in our sample due to various factors.  Specifically:  In three cases, contracts were terminated before performance was  complete. Upon termination, the government and contactor negotiated  a final settlement which accounted for factors in addition to cost, and  the original target costs—which were based on the assumption of  completed performance—were no longer relevant.
In another case, officials explained that requests for adjustments and  other actions had not been finally settled for this order, so the actual  costs will likely change further. Because these adjustments and  actions had yet to be settled, we did not compare the target costs and  current estimates for actual costs.
In one additional case, contract records did not differentiate between  target costs and actual costs, and so we were unable to compare the  two.
section_title: DOD Reported Positive Outcomes for Most Incentive Fee Contracts We Reviewed Containing Schedule and Technical Performance Incentives
paragraphs: Contracting officials reported generally positive cost, schedule, technical  performance, and overall outcomes for those contract actions containing  schedule and technical performance incentives. Fifteen of the 26 FPI or  CPIF contracts in our review used schedule or technical performance  incentives along with cost incentives, and therefore were multiple- incentive contracts. Three of the 15 contracts used a combination of  incentive fees or profits and award fees. For 13 of the 15 contracts we  reviewed with schedule or technical performance incentives, contracting  officials reported that overall outcomes were positive. In two cases,  although the contractor met specific schedule and technical performance  goals, overall outcomes were either unsatisfactory or were yet to be  determined (see table 4).
DOD’s April 2016 guidance, issued after the contracts we reviewed were  awarded, advises contracting officials to carefully consider the use of  multiple incentives that may compete with one another. In most cases,  acquisition planning documents we reviewed indicated that contracting  officials used multiple incentives in contracts with the goal of encouraging  the contractor to achieve specific cost, schedule, or technical  performance targets, while ensuring that achieving one outcome did not  come at the expense of others. The DOD guidance notes that the  contractor will aim to maximize the profits or fees it earns, and  consequently make trade-offs that may not be consistent with how the  government views the relative importance of the various incentives. For  contracts with multiple incentives that we reviewed, it was unclear how  these incentives interacted, and we could not isolate the effects of any  particular schedule or technical performance incentive. Representatives  of the two contractors with whom we spoke indicated that they did not  have a precise method for making trade-offs among incentives, and could  not tell us how the presence of any particular incentive may have  interacted with other incentives. Representatives of one contractor told us  that they viewed the technical performance incentives as a chance to  “make up” what they “lost” on the cost incentives. In other words, if they  did not receive all possible profits or fees from cost incentives, they could  still aim to earn profits or fees from technical performance incentives.  Representatives from the other contractor told us that the schedule  incentive fee was not a primary factor motivating them to maintain  schedule. The amount of potential schedule incentive was less than half  of the potential cost incentive and technical performance award fees, so  the schedule incentive was likely less effective than these other  incentives.
section_title: Award Fee Contracts Reviewed Were Used for Services and Prior Issues Were Generally Addressed
paragraphs: Nine of the 35 contract actions we reviewed contained only award fee  provisions. All nine of these contract actions were to procure services,  such as providing support for testing, data collection, and  experimentation; weapons testing; and operation of military base  childcare facilities. Of the nine contract actions, three were FPAF actions,  and six were CPAF actions (see table 5). These contract actions  contained a mix of incentives targeting cost, schedule, and technical  performance. For the eight actions we could assess, contractors earned  90 percent of potential award fees overall. Award fees earned on the  eight actions for which we had data totaled $16.1 million between fiscal  years 2011 and 2015.
In cases where multiple types of outcomes (e.g., cost, schedule, and  technical performance) were targeted, it was sometimes not possible to  determine the amount of fee awarded based on a particular outcome. For  example, schedule was a subpart of one of several categories for the  award fee, while award fee memoranda reported only a single overall  category score. Therefore, in those cases, it was not possible to  determine from the award fee documentation what portion of the fee  award was meant to correspond specifically to schedule performance.
The contract actions we reviewed generally addressed some issues we  had previously identified in relation to award fees, and reflected changes  made to federal acquisition regulations in 2009. For example, we found  that all of the nine contract actions contained provisions prohibiting  rollover. Most of the award fee plans for these actions specifically  prohibited paying award fees for unsatisfactory performance and we  found no evidence of fees earned for unsatisfactory performance.
DOD guidance says award fees should be tied to acquisition outcomes to  the maximum extent possible. We reviewed the criteria contained in the  award fee plans for our sample contract actions and found them to  contain a mix of outcome-based criteria and process-based criteria. For  example, three award fee plans covering five contract actions contained  criteria related to meeting schedule deadlines, which we consider to be  related to outcomes. One award fee plan contained criteria related to  management, such as proactivity and responsiveness to problems, which  we consider to be related to process. In the nine contracts we reviewed,  we also found examples of award fees based on both outcome- and  process-based criteria. For example, one contract, valued at $78 million  for management and operation of transportation services on an Army  installation, specified that 25 percent of the award fee would be based on  technical performance criteria. Under these criteria, points were assigned  on the basis of quality and timeliness assessments, which we consider to  be outcome-based, as well as reporting, which we consider to be  process-based. Similarly, a $16 million Army contract for installation  support services based 75 percent of its award fee on a “Performance of  Work” criterion, which contained several sub-criteria including “Quality”  and “Efficiency/Timeliness,” both of which contained a mix of outcome-  and process- based criteria. Even where award fee plan criteria were  outcome-based, we could not assess whether the outcomes cited were  identified by DOD as positive outcomes for the acquisition of services. As  we have previously reported, DOD has struggled to define and track  desired outcomes for services contracts, which differ from products in  several aspects and can pose challenges to establishing measurable and  performance-based outcomes.
section_title: DOD Conducts Limited Data Collection on Incentive and Award Fee Contracts and Does Not Assess Incentive Outcomes Across Contracts
paragraphs: DOD has not consistently assessed how the selection of a particular  contract type or incentive arrangement has promoted the achievement of  cost, schedule, or technical performance goals. A fiscal year 2007  legislative provision directed DOD to collect and evaluate relevant data on  incentive and award fee payments on a regular basis to determine the  effectiveness of incentives for improving contractor performance and  achieving desired program outcomes, and in 2009, the FAR was updated  to require this of all agencies. As a result, DOD previously required  military departments and defense agencies to collect data on incentive  and award fees—such as the amount of fees available on each contract  and the amount paid to contractors—twice a year for contracts with  incentive provisions greater than $50 million. According to a senior  DPAP official, the information collected was not being used at the time, as  DOD saw more value in focusing on efforts prior to contract award than  analyzing incentive trends. Additionally, DPAP officials noted that the  effort amounted to a manual data collection exercise during a time of  reduced staffing levels. DOD rescinded this requirement in April 2015,  explaining in the Federal Register that it could obtain relevant data  through other sources, including CBAR and peer reviews. Our review  found, however, that DOD is not using these sources to assess the  effectiveness of incentive contracts. Further, we found that CBAR, peer  reviews, and other potential sources DOD identified have limited utility in  providing information to assess the effectiveness of improving contractor  performance and achieving desired program outcomes (see table 6).
These systems have distinct purposes and are not specifically intended to  provide information for DOD to analyze how well it is achieving incentive  outcomes, though some have the potential to provide insight into  outcomes. For example, DOD officials said they used Cost and Software  Data Reporting and earned value management data—which are  maintained in the CADE system— to evaluate the effectiveness of factors  that motivate contractor performance in its 2014 annual report on  performance of the defense acquisition system. Among the contracts  that DOD reviewed, it found that those with incentive fees or profits  typically experienced lower cost growth than other contract types. This  analysis, however, used total contract costs (including non-incentivized  portions), according to senior DOD officials. DOD has not conducted this  analysis in subsequent annual reports. DOD has also aggregated  feedback from past peer reviews—including reviews focused on  incentives—and identified lessons learned and best practices for  structuring incentives.
DOD’s efforts to better manage its acquisition of services, which  accounted for more than half of the $274 billion in total DOD contract  obligations in fiscal year 2015, could also help assess the merits of using  incentives for various portfolios of services. As part of these efforts, DOD  has identified senior officials within DOD to serve as functional domain  experts responsible for specific portfolios of services. DOD’s January  2016 services acquisition instruction tasked these experts, among other  responsibilities, to  identify and share portfolio group best practices and employ lessons  learned to improve the acquisition and management of services  across their respective categories, and develop appropriate metrics to track cost and performance of  contracted services within the portfolio group to leverage best  practices, reduce redundant business arrangements, identify trends,  and develop year-to-year comparisons to improve the efficiency and  effectiveness of contracted services.
We have ongoing work to assess how DOD and the military departments  are implementing aspects of DOD’s services acquisition instruction. As  we noted in our February 2017 high-risk update, however, DOD does not  have an action plan that would enable it to assess progress toward  achieving its goals for improving service acquisitions, and its efforts to  develop goals and associated metrics unique to each category of service  it acquires are in the early stages of development.
As previously noted, the FAR requires agencies to collect and evaluate  relevant data on incentive and award fee payments on a regular basis to  determine the effectiveness of incentives for improving contractor  performance and achieving desired program outcomes. Since DOD  removed the requirement from its own regulations in April 2015, it has not  identified a new approach to collect the required information beyond using  CBAR and peer reviews, which we found do not allow DOD to assess  incentive outcomes as currently used. It may not be necessary to collect  data on each individual contract, but rather identify what best meets the  department’s needs for assessing performance outcomes and collect  data accordingly. Without assessing how incentives have contributed to  intended outcomes, contracting officers may be at a disadvantage in  establishing appropriate incentives for the requirements they work to fulfill  through contracts.
section_title: Conclusions
paragraphs: The government should be vigilant about getting the best value for its  dollar, and incentive and award fees can be effective tools for motivating  contractors and achieving desired outcomes for DOD acquisitions, if  appropriately applied. Over the past decade, DOD’s guidance and  training has emphasized the use of objective incentives, which is reflected  in a noticeable shift toward incentive fee contracts and away from the  more subjective award fee contracts. As some of our prior work has  found, however, incentives do not always lead to better outcomes. Given  the emphasis on cost incentives, it is important that DOD determine  whether and under what circumstances the use of these incentives is  achieving intended cost objectives. Federal acquisition regulations require  DOD to collect and analyze information on the use of incentives.  However, it may not be necessary for DOD to embark on a broad, manual  data collection effort similar to what has been tried in the past. Rather,  DOD could focus its effort on specific areas of interest or risk to the  department. For example, given the widespread use of incentive  contracts on major weapon systems, DOD could continue to focus its  analyses on the factors that facilitate or hinder the achievement of cost  objectives or consider expanding the collection of information on lower  dollar weapon systems. Alternatively, DOD could focus more on  identifying what types of incentives prove useful for the different services  DOD acquires as part of its efforts to manage portfolios of services  acquisitions. Such efforts should not preclude DOD from continuing to  assess other approaches to motivate contractor performance, such as the  use of technical and schedule incentives. Prioritizing the area or areas  DOD intends to focus on would enable the department to determine how  best to collect that information and, in turn, use it to identify opportunities  to improve the use of incentive contracts.
section_title: Recommendation for Executive Action
paragraphs: We recommend that the Under Secretary of Defense for Acquisition,  Technology and Logistics identify the specific types of information that  would best meet the department’s needs and, based on that  determination, collect and analyze relevant data after contract  performance is sufficiently complete to determine the extent to which  contracts with incentives achieved their desired outcomes.
section_title: Agency Comments
paragraphs: We provided a draft of this product to DOD for comment. In its comments,  reproduced in appendix III, DOD concurred and indicated it will establish  a process for identifying specific types of information to collect and  assess the data after the completion of contract closeout to determine the  extent to which incentives achieved their desired outcomes. DOD stated it  will complete this process by the second quarter of fiscal year 2018. DOD  also provided technical comments, which we incorporated as appropriate.
We are sending copies of this report to the appropriate congressional  committees and the Secretary of Defense; the Under Secretary of  Defense for Acquisition, Technology, and Logistics; the Secretaries of the  Air Force, Army, and Navy; the President, Defense Acquisition University;  and the Director, Office of Management and Budget. In addition, the  report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or DiNapoliT@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in Appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of this review were to (1) identify steps the Department of  Defense (DOD) has taken to improve its use of incentive contracts since  2010, and (2) assess the extent to which selected DOD incentive  contracts achieved desired acquisition outcomes.
To identify the steps DOD has taken to improve its use of incentive  contracts since 2010, we reviewed relevant legislation and provisions  within the Federal Acquisition Regulation (FAR); memoranda issued by  the Office of Management and Budget (OMB); and identified and  reviewed changes to DOD regulations, policies, and guidance regarding  the use of incentive contracts. To identify changes in DOD’s use of these  contract types over time that may have corresponded with regulatory and  policy changes, we analyzed data from the Federal Procurement Data  System-Next Generation (FPDS-NG) on obligations by contract type for  fiscal years 2005 through 2015. We reported our findings in constant  fiscal year 2015 dollars, adjusted for inflation using the fiscal year gross  domestic product price index. To assess the reliability of the FPDS-NG  data, we conducted electronic testing of the data. We also reviewed a  selection of 53 contracts in FPDS-NG and, in reviewing contract  documents, found that eleven percent was incorrectly coded as being  incentive contracts. We assessed these entries and found that they  represented a range of contract award dates and products and services,  and generally aligned with the distribution across military departments of  contract actions in our sample. We determined that the miscoded  contracts had minimal potential for impacting our analysis and that the  data were sufficiently reliable to report general trends in DOD’s  obligations for incentive contract actions. In addition, for a sub-selection  of the 53 contracts and orders, we traced selected data fields to contract  file documents to verify their accuracy. We also interviewed DOD,  military department, and DAU officials about efforts to improve the use of  incentive contracts.
To determine the extent to which incentive provisions in selected DOD  contracts achieved desired outcomes in cost, schedule, and technical  performance, we assessed a nongeneralizable sample, drawn from  FPDS-NG data, of incentive contracts and orders that DOD awarded  between fiscal years 2011 and 2015 and that were reported as completed  by the end of fiscal year 2015. We chose these timeframes so as to select  awards that were made after regulatory and policy changes starting in  2010 and had completed performance by the time of our review. Because  of these parameters, certain types of contract actions were inherently  excluded from our analysis, including those that exceeded a performance  period of 5 years, such as longer term satellite and shipbuilding  contracts.
For our contract selection, we excluded contracts with values below the  $150,000 simplified acquisition threshold, indefinite delivery contracts,  and blanket purchase agreements. We initially selected all 38 contracts  that met these criteria. Out of more than 4,200 orders, we selected a  subset of 15 that (1) included different incentive types under the same  base contract, (2) were used to purchase similar products and services,  and (3) reflected a range of dollar values. These parameters yielded an  initial sample of 53 contract actions (contracts and orders). Eighteen of  the 53 actions were dropped from our sample either because the  contracts were miscoded in FPDS-NG and were not incentive contracts  based on our review of contract documents (6), were terminated before  performance began (1), or were actually not yet complete (11).  Consequently, we reviewed a total of 35 contracts and orders  representing fixed-price incentive (FPI), fixed-price-award-fee (FPAF),  cost-plus-incentive-fee (CPIF), and cost-plus-award-fee (CPAF) contract  types (see table 7). Because we used a nongeneralizable sample of  contracts and orders, results from this sample cannot be used to make  inferences about all incentive contracts and orders that DOD awarded.
In total, we reviewed 5 contract actions from the Air Force, 7 from the  Army, and 23 from the Navy. For each selected contract and order, we  collected relevant documentation, such as the initial contract or order,  modifications, statements of work, determination and findings  memoranda, award fee plans, and performance evaluations. We  interviewed contracting officials and obtained their written input to clarify  and collect additional information as needed.
To determine cost performance on CPIF and FPI contracts and orders,  we compared actual or estimated costs to target costs listed in the  conformed contracts or applicable contract modifications specifically for  incentivized line items. For contracts where actual costs were not yet  finally settled, we obtained the best available current estimate from  contractor cost reports and similar sources. For contracts with multiple  line items with incentives, we compared the total target costs to total  actual or estimated costs. Within these types of contracts, if the contractor  performed one line item at a cost overrun but another at a cost underrun,  only the net results would be reflected in our findings. For contracts and  orders with schedule and technical performance incentives, we identified  schedule and technical performance goals and outcomes using the  contracting office responses and contract documents. We obtained total  action values from the conformed contracts. For contracts and orders  containing award fees, we calculated the amount of award fees earned  based on data provided by the contracting offices, including total action  values and total potential award fee amounts, and corresponding contract  documentation. We examined award fee plans to identify provisions  related to rollover and criteria for award, and we examined award fee  board memos and other documents to understand how award fee  amounts were determined.
To supplement our understanding of how incentive provisions helped  achieve acquisition outcomes, we interviewed two contractors about the  effectiveness of incentive provisions in motivating performance toward  desired outcomes. To identify contractors to interview, we focused on a  sub-selection of contracts containing multiple incentives to obtain  perspectives on how the incentives interacted with respect to the  contractors’ performance. The information obtained from interviews with  the two contractors cannot be generalized to all contractors; however, the  interviews provide important insights on the experiences of contractors.
We also reviewed documents and interviewed DOD officials about  relevant data systems and processes used to collect and analyze contract  and program data, including the Contract Business Analysis Repository,  the Cost Assessment Data Enterprise, the Contractor Performance  Assessment Reporting System (CPARS), and peer reviews, to  understand how these systems may provide some insight into DOD’s use  of incentive contracts. Specifically, we reviewed 13 CPARS reports from  contracts in our sample, selected for a distribution among the military  departments and various contract types. We also reviewed a  nongeneralizable selection of 30 peer review summaries from fiscal years  2014 and 2015 to develop an understanding of how peer reviews were  used to provide guidance on key contracting decisions.
The performance audit upon which this report is based was conducted  from January 2016 to May 2017 in accordance with generally accepted  government auditing standards. Those standards require that we plan  and perform the audit to obtain sufficient, appropriate evidence to provide  a reasonable basis for our findings and conclusions based on our audit  objectives. We believe that the evidence obtained provides a reasonable  basis for our findings and conclusions based on our audit objectives.  DOD deemed that certain information in the draft report related to  contract costs and performance was sensitive and must be protected  from public disclosure. We subsequently worked with DOD from May  2017 to July 2017 to revise our presentation of this information and  prepare this report for public release.
section_title: Appendix II: Trends in Obligations for Incentive Contracts by DOD Component from Fiscal Years 2005 through 2015
paragraphs: We analyzed data from the Federal Procurement Data System-Next  Generation on incentive contract obligations by Department of Defense  (DOD) component: Air Force, Army, Navy, and all other DOD. We found  that, similar to DOD, the three military departments increased their  obligations for incentive fee contracts overall from fiscal years 2005  through 2015. The only notable anomaly we found was that other DOD  components obligated fewer dollars for incentive fee contracts over the  period we reviewed (see figure 4).
We also found that each of the military departments as well as other DOD  components had decreased their obligations for award fee contracts from  fiscal years 2005 through 2015 (see figure 5).
section_title: Appendix III: Comments from the Department of Defense
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact name above, the following staff members made  key contributions to this report: Janet McKelvey (Assistant Director), Pete  Anderson, MacKenzie Cooper, Brenna Derritt, Alexandra Dew Silva,  Lorraine Ettaro, Kurt Gurka, Julia Kennon, Liam O’Laughlin, Carol  Petersen, Raffaele (Ralph) Roffo, Ann Marie Udale, and Robin Wilson.