Defense Contracting: DOD Needs Better Information on Incentive Outcomes
Released: 2017-07-11T00:00:00

Letter

Background

Contract Types

According to the FAR, a wide selection of contract types is available to the government and contractors to allow flexibility in acquiring a variety of products and services. Contract types vary according to the degree and timing of the contractor’s responsibility for the costs of performance and by the amount and nature of the profit incentive offered to the contractor for meeting or exceeding specified goals. Contract types are grouped into two broad categories: fixed-price or cost-reimbursement contracts. Within these categories, the specific types range from firm-fixed-price (FFP), in which the contractor has full responsibility for the costs of performance and the resulting profit or loss, to cost-plus-fixed-fee (CPFF), in which the contractor has minimal responsibility for the costs of performance and the fee is a fixed dollar amount, established as a percentage of the estimated target cost at the start of the contract. In between these are incentive contracts, in which the contractor’s responsibility for costs and the profit or fee incentives offered are tailored to performance uncertainties.

The FAR also notes that incentive contracts are appropriate when a firm- fixed-price contract is not, and the required items can be acquired at lower costs and possibly with improved delivery or technical performance by tying fee or profit to the contractor’s performance. Incentive and award fee provisions can be used together in the same contract, but each uses a different approach with respect to how performance is assessed and how fees or profits are determined.

Incentive fees—For contracts with incentive fees or profits, the amount of fee or profit payable is related to the contractor’s performance. Incentive fees or profits generally focus on cost control, though they may be used to motivate performance toward specific delivery (e.g., schedule) targets or technical goals. Incentive fees or profits involve an objective evaluation by the government through a process that is generally less administratively burdensome than award fee evaluations. The government usually applies a fee- or profit- determination formula that is specified in the contract to evaluate performance at the end of the contract or at program milestones. The formula may include a target cost, a target profit or fee, a ceiling price, and a profit- or fee-adjustment formula, sometimes referred to as a share ratio for fixed-price incentive (FPI) contracts (see figure 1).

In the hypothetical example above, if the contractor incurred $100 in costs, the contractor would receive $10 in profit, and the government would pay the target price of $110. If the contractor kept costs under the $100 target cost, it would split cost savings equally with the government and receive a larger profit; if the contractor exceeded the cost target, it would share the additional costs with the government up to the ceiling price but earn a smaller profit. At ceiling, the contractor earns no profit, and the contractor is responsible for any costs incurred above the ceiling price.

A cost-plus-incentive-fee (CPIF) contract reimburses the contractor for its allowable costs, but still uses a formula of total allowable costs to target costs to determine fee and includes a target fee instead of a target profit. A CPIF contract also has a minimum fee—the lowest fee the contractor may receive when total allowable costs exceed target costs—and maximum fee—the highest fee the contractor may earn when total allowable costs are less than target costs, and, unlike an FPI contract, there is no ceiling price.

Award fees—Award fees typically emphasize multiple aspects of contractor performance in areas that are more subjectively assessed, such as the contractor’s responsiveness, technical ingenuity, or cost management. From the government’s perspective, development and administration of award fee contracts often involve substantially more effort over the life of a contract than incentive fee contracts, requiring government officials, through an award fee evaluation board, to conduct periodic evaluations of the contractor’s performance against specified criteria and to make recommendations on the amount of fee to be paid. Criteria are specified in an award fee plan, which contracting officials may revise from one evaluation period to another to redirect contractor emphasis. Following the award fee evaluation, a fee-determining official makes the final decision about the amount of fee paid to the contractor.

Table 2 identifies the range of incentive contract types and their appropriate use based on acquisition regulations.

Prior Work

We have previously identified issues with the use of incentive and award fees that called into question whether they were used effectively to achieve their intended purpose. In our December 2005 review of incentive and award fee contracts, we found that award fees were generally not linked to acquisition outcomes, and that DOD had paid an estimated $8 billion in award fees regardless of outcomes. In addition, we estimated that in 52 percent of the award fee contracts, DOD moved unearned award fees from one evaluation period to a subsequent period—a practice referred to as “rollover”—which provides contractors at least a second chance to earn fees after failing to perform well enough to earn them initially. We also found that DOD had not compiled data, conducted analyses, or developed performance measures to evaluate the effectiveness of incentive and award fees. We recommended that DOD apply more outcome-based award fee criteria, pay award fees only for above satisfactory performance, issue guidance on the appropriate use of rollover, develop a mechanism for capturing incentive and award fee data within existing data systems, and develop performance measures to evaluate the effectiveness of incentive and award fees at improving contractor performance and achieving desired outcomes. DOD concurred with two of these recommendations, and partially concurred with our recommendations to only pay award fees for above satisfactory performance, collect incentive and award fee data, and develop performance measures to evaluate the effectiveness of incentive and award fees. DOD implemented all but one of these recommendations— paying award fees only for above satisfactory performance—though a provision was later added to the FAR prohibiting payment of award fees for below satisfactory performance.

In our May 2009 review of 50 DOD contracts containing award fees, we found that DOD had made progress toward minimizing payments of award fees for unsatisfactory performance, limiting overpayment for satisfactory performance, and reducing the number of programs that used rollover. DOD, however, still struggled to use data collected on award fee contracts to evaluate their effectiveness. We did not make new recommendations in these areas.

Most recently, in a March 2017 report on selected FPI contracts awarded by the Navy for new ship construction, we found the Navy often structured the contracts such that it absorbed more cost risk than DOD’s regulation suggests, indicating it may not achieve the expected benefits of using the FPI contract type. For example, we found that 8 of 11 ships delivered under the contracts reviewed experienced cost growth. We recommended that DOD conduct a portfolio-wide assessment of the Navy’s use of additional incentives on FPI contracts across shipbuilding programs. DOD concurred with our recommendation.

Government-wide Changes

In 2007, OMB issued government-wide guidance highlighting preferred practices for incentive contracting and directing agencies to review and update their acquisition policies. In 2009, the FAR was revised to implement legislative provisions and OMB’s guidance on the appropriate use of incentive contracts. These changes addressed some of the issues that we identified in 2005 and 2009 relative to the use of award fees. The FAR now prohibits rollover of unearned award fees from one evaluation period requires award fees to be linked to cost, schedule, and technical performance acquisition objectives; restricts payment of award fees in instances of unsatisfactory contractor performance; and requires agencies to collect relevant data on incentive and award fee payments and evaluate the effectiveness of these contract types in achieving desired outcomes.

DOD Has Changed Regulations, Policies, and Practices to Improve Its Use of Incentive Contracts, in Particular by Emphasizing Use of Objective Incentives

Since 2010, DOD has taken steps to improve its use of incentive contracts—often beyond what is required by the FAR—by revising the Defense Federal Acquisition Regulation Supplement (DFARS), instituting its Better Buying Power initiative, and developing new guidance and training courses. In particular, DOD has emphasized the use of objective incentives through FPI and CPIF contracts rather than award fees whenever possible, in part to better motivate contractors to control costs. These efforts are reflected in DOD’s reported use of incentive contracts since 2010, which indicate a substantial growth in obligations for incentive fee contracts and a corresponding decrease in obligations for award fee contracts.

DOD Has Updated Regulations, Policy, and Guidance

DOD made changes to the DFARS and its accompanying Procedures, Guidance, and Information in recent years intended to support appropriate use of incentive contracts. For example, DOD made the following updates to the DFARS in 2011:

Directed contracting officers to utilize objective criteria—associated with incentive fee contracts—to the maximum extent possible for measuring contract performance. DOD noted concerns that award fee contracts have a limited ability to motivate contractors to control costs, and that there had been instances in which award fee payments were not consistent with outcomes.

Directed contracting officers to give particular consideration to FPI contracts, especially for acquisitions moving from development to production or in contracts for which previous FFP contract costs had varied by more than 4 percent from negotiated costs. By looking at historical pricing and contract performance data, DOD officials stated they determined that in some FFP contracts, actual costs have come in noticeably lower than negotiated costs (e.g., at 4 percent or more), indicating that costs are not stable or that the government may not have negotiated a good deal when it awarded the contract.

Directed contracting officers to include a contract clause prohibiting the payment of award fees when a contractor’s performance is rated below satisfactory, as required by the FAR, which emphasizes that the amount of award fee paid should correspond with the contractor’s performance.

DOD released memoranda between 2010 and 2015 through its Better Buying Power initiative, which focused, in part, on the use of incentive contracts. The Better Buying Power memoranda established a preference for FPI contracts and advised contracting officers to increase the use of this contract type, when appropriate, such as early in production and in single-source production where year-over-year price improvement can be rewarded. DOD acknowledged that some officials interpreted the first memorandum to mean that FPI contracts should be used to the exclusion of other contract types. As a result, subsequent memoranda advised officials to consider the full range of contract types and employ the appropriate type, while giving particular consideration to FPI and CPIF contracts. In addition, Better Buying Power called to limit the use of award fee contracts for services, noting that services acquisitions should be predisposed to FFP, CPFF, or CPIF. Better Buying Power also instructed the military departments to provide a justification of contract type for proposed contracts over $100 million for major programs.

Better Buying Power also called for new DOD guidance on selecting contract types and employing incentive contracts. Subsequently, in April 2016, Defense Procurement and Acquisition Policy (DPAP) released Guidance on Using Incentive and Other Contract Types. This guidance provides direction on selecting contract types, structuring appropriate incentive arrangements, and negotiating target costs and share ratios with contractors. It explains that, when appropriately structured, incentive contracts can allow the government to share in cost savings, focus the contractor on the areas that are important to the government, and provide the government with valuable data on actual costs incurred. It also reinforces some key updates to DFARS, such as emphasizing that objective criteria must be used whenever possible to measure contract performance.

DOD’s DAU is developing two new continuous learning courses to reinforce concepts reflected in the April 2016 guidance. According to DAU, all contracting personnel involved in using incentive arrangements will be encouraged to take these courses, though they are not required for Defense Acquisition Workforce Improvement Act certification, a process DOD uses to determine that acquisition officials meet certain standards. One course, Understanding Incentive and Other Contract Types, is currently available and provides training on how to align contract types and incentives with acquisition outcomes. The second course is expected to be available in August 2017 to provide training on appropriate use of advanced incentive concepts, such as quantifying cost, schedule, and performance risks and incorporating that information into decisions on contract incentives. DAU also provides other courses with elements addressing aspects of incentive contracting. Program officials—who can be involved in selecting contract types and structuring incentives, according to a senior DPAP official—also undergo some training on contract types and incentives through DAU training courses.

Further, to inform selection of contract type, contract negotiations, and projections of program and contract costs, DPAP and the Director of Defense Pricing have encouraged collaboration among contracting officials, program officials, and cost analysts to collect and share cost information with one another. DOD has also required contracting officers to share information through CBAR—which captures information to assist contracting officers in preparing for negotiations with contractors, such as contractor business systems status and compliance with cost accounting standards. Contracting officials use CBAR to upload and share contract negotiation documents, which, according to DOD, can help contracting officials benefit from others’ experiences, particularly when negotiating with the same contractor.

Finally, DOD has used independent management reviews, or peer reviews, to advise contracting officers on selecting the appropriate contract type and structuring and negotiating contract incentives, among other topics. The Director of Defense Pricing and DPAP lead peer reviews to ensure that certain high-dollar acquisitions are carried out in accordance with applicable laws, regulations, and policies. According to senior DOD and military department officials, peer reviews provide an opportunity for DPAP leadership to share knowledge about key contracting decisions. For example, one peer review advised the contracting officer to consider whether increasing the available fee under a CPIF contract would reduce costs to the government by incentivizing greater cost control from the contractor.

DOD’s focus on incentive contracts is evident in many major defense acquisition program contracts. Based on information provided by DOD, as of January 2017 the department is using incentive contracts—either FPI or CPIF—on 65 of 78 major defense acquisition programs. The Army’s Patriot Advanced Capability-3 (PAC-3) program—which provides mobile defense against short-range ballistic missiles and other threats—offers an example of DOD’s use of FPI contracts in particular. According to DOD, recent contracts for PAC-3 reflect the department’s consideration of an FPI contract type for programs that previously used FFP contracts in which actual costs varied significantly from negotiated costs. Through an analysis of actual costs on prior production contracts for the PAC-3 program, DOD determined that the prime contractor was underrunning negotiated costs—that is, actual costs were lower—in these contracts in amounts ranging from 8 to 15 percent, triggering consideration of FPI type based on defense regulations. Consequently, after reviewing historical pricing data and applying lessons learned, Army contracting officials stated they were able to negotiate FPI contracts for missile production over fiscal years 2014 through 2016 that were a total of $860 million lower than the contractor’s initial proposals. Army officials stated that they plan to use cost information gathered through these contracts to determine the appropriate contract type and inform negotiations for future production contracts. Because these contracts are ongoing, however, actual costs and other outcomes have not yet been determined, though current costs for the fiscal year 2014 contract indicate it may result in a cost overrun, in which actual costs exceed target costs.

Data on Obligations for Incentive Contracts Are Largely Consistent with Changes in Policy

Our analysis of data from FPDS-NG found that obligations for incentive contracts ranged from about 13 to 22 percent of DOD’s total annual contract obligations from fiscal year 2005 through fiscal year 2015. In fiscal year 2015, incentive contracts accounted for nearly 18 percent of DOD’s total annual contract obligations. Consistent with DOD’s emphasis on using incentive fee contracts and decreasing the use of award fee contracts, our analysis of DOD’s reported contract obligations from fiscal years 2005 through 2015 shows a shift toward using incentive fee contracts (see figure 2).

More specifically, our analysis found that the changes were largely driven by increased obligations for FPI contracts, and decreased obligations for CPAF contracts (see figure 3).

Among specific products and services for which incentive contracts were used, we found the mix varied by contract type. Specifically:

FPI contracts were mostly used to purchase products. In fiscal year 2015, three product categories accounted for almost 90 percent of FPI obligations: aircraft, ships/submarines, and land vehicles; weapons and ammunition; and sustainment supplies and equipment.

CPIF contracts were used for a mix of products and services: aircraft, ships/submarines, and land vehicles; research and development; equipment related services; weapons and ammunition; and knowledge based services—such as engineering, program management, and education and training—accounted for roughly three quarters of CPIF obligations in fiscal year 2015.

Award fee contracts were mostly used for services, including facility related; transportation; and equipment related services.

DOD Expects to Achieve Cost Objectives on Selected Contracts, but Lacks Full Information on Outcomes for Incentive Contracts

DOD expects to achieve positive cost outcomes—with contractors’ estimated costs coming in lower than target costs—for most of the 21 selected incentive fee contract actions we were able to measure. Overall, the estimated costs for the incentivized portions of these selected contract actions were about $30 million—or about 5 percent—below target costs. Among the contract actions we reviewed, schedule and technical performance incentives were included in multiple-incentive contracts. Officials reported good outcomes overall for the contracts with multiple incentives that we reviewed, but we could not isolate the effects of any particular schedule or technical performance incentive. The nine award fee actions in our sample, which were mainly used to procure services, did not allow for rollover and payments for unsatisfactory performance—both of which were issues we found in our prior work. DOD collects some information on incentive contracts, but it generally has not assessed the extent to which particular contract types or incentive arrangements have achieved cost, schedule, or technical performance goals.

Estimated Costs Were Below Target for Most of the Incentive Fee Contract Actions That We Could Measure

DOD expects contractors to underrun cost targets for incentive fee or profit provisions in 15 of the 21 cases for which we could compare target and estimated costs. Overall, these contract actions were expected to underrun target costs on the incentivized portions of the contracts by about 5 percent, amounting to $30 million in expected savings shared between the government and contractors on the 21 contract actions. These 21 contracts had a total value of about $957 million. The contract actions that were expected to underrun target costs represented procurements of both goods and services, and were a mixture of FPI and CPIF actions (see table 3).

In two cases, contracting officials raised potential benefits of using an FPI contract that extended beyond underrunning target costs in a single contract by using knowledge of cost efficiencies to decrease prices from one production lot to the next. For example, contracting officials for one Air Force missile program determined that an engine produced by a subcontractor was a cost driver, but had little insight into the actual costs for this component. The Air Force created a separately-priced FPI line item with unit pricing for the engines. Using an FPI contract type required the contractor to provide cost data, a requirement that was to be incorporated in turn into the prime contractor’s FPI agreement with its subcontractor. Officials reported that through this arrangement, they obtained insight into subcontractor costs and were able to obtain cost savings in subsequent lots of more than $104,000 per engine.

We were unable to compare actual or estimated costs and target costs for 5 of the 26 FPI or CPIF actions in our sample due to various factors. Specifically: In three cases, contracts were terminated before performance was complete. Upon termination, the government and contactor negotiated a final settlement which accounted for factors in addition to cost, and the original target costs—which were based on the assumption of completed performance—were no longer relevant.

In another case, officials explained that requests for adjustments and other actions had not been finally settled for this order, so the actual costs will likely change further. Because these adjustments and actions had yet to be settled, we did not compare the target costs and current estimates for actual costs.

In one additional case, contract records did not differentiate between target costs and actual costs, and so we were unable to compare the two.

DOD Reported Positive Outcomes for Most Incentive Fee Contracts We Reviewed Containing Schedule and Technical Performance Incentives

Contracting officials reported generally positive cost, schedule, technical performance, and overall outcomes for those contract actions containing schedule and technical performance incentives. Fifteen of the 26 FPI or CPIF contracts in our review used schedule or technical performance incentives along with cost incentives, and therefore were multiple- incentive contracts. Three of the 15 contracts used a combination of incentive fees or profits and award fees. For 13 of the 15 contracts we reviewed with schedule or technical performance incentives, contracting officials reported that overall outcomes were positive. In two cases, although the contractor met specific schedule and technical performance goals, overall outcomes were either unsatisfactory or were yet to be determined (see table 4).

DOD’s April 2016 guidance, issued after the contracts we reviewed were awarded, advises contracting officials to carefully consider the use of multiple incentives that may compete with one another. In most cases, acquisition planning documents we reviewed indicated that contracting officials used multiple incentives in contracts with the goal of encouraging the contractor to achieve specific cost, schedule, or technical performance targets, while ensuring that achieving one outcome did not come at the expense of others. The DOD guidance notes that the contractor will aim to maximize the profits or fees it earns, and consequently make trade-offs that may not be consistent with how the government views the relative importance of the various incentives. For contracts with multiple incentives that we reviewed, it was unclear how these incentives interacted, and we could not isolate the effects of any particular schedule or technical performance incentive. Representatives of the two contractors with whom we spoke indicated that they did not have a precise method for making trade-offs among incentives, and could not tell us how the presence of any particular incentive may have interacted with other incentives. Representatives of one contractor told us that they viewed the technical performance incentives as a chance to “make up” what they “lost” on the cost incentives. In other words, if they did not receive all possible profits or fees from cost incentives, they could still aim to earn profits or fees from technical performance incentives. Representatives from the other contractor told us that the schedule incentive fee was not a primary factor motivating them to maintain schedule. The amount of potential schedule incentive was less than half of the potential cost incentive and technical performance award fees, so the schedule incentive was likely less effective than these other incentives.

Award Fee Contracts Reviewed Were Used for Services and Prior Issues Were Generally Addressed

Nine of the 35 contract actions we reviewed contained only award fee provisions. All nine of these contract actions were to procure services, such as providing support for testing, data collection, and experimentation; weapons testing; and operation of military base childcare facilities. Of the nine contract actions, three were FPAF actions, and six were CPAF actions (see table 5). These contract actions contained a mix of incentives targeting cost, schedule, and technical performance. For the eight actions we could assess, contractors earned 90 percent of potential award fees overall. Award fees earned on the eight actions for which we had data totaled $16.1 million between fiscal years 2011 and 2015.

In cases where multiple types of outcomes (e.g., cost, schedule, and technical performance) were targeted, it was sometimes not possible to determine the amount of fee awarded based on a particular outcome. For example, schedule was a subpart of one of several categories for the award fee, while award fee memoranda reported only a single overall category score. Therefore, in those cases, it was not possible to determine from the award fee documentation what portion of the fee award was meant to correspond specifically to schedule performance.

The contract actions we reviewed generally addressed some issues we had previously identified in relation to award fees, and reflected changes made to federal acquisition regulations in 2009. For example, we found that all of the nine contract actions contained provisions prohibiting rollover. Most of the award fee plans for these actions specifically prohibited paying award fees for unsatisfactory performance and we found no evidence of fees earned for unsatisfactory performance.

DOD guidance says award fees should be tied to acquisition outcomes to the maximum extent possible. We reviewed the criteria contained in the award fee plans for our sample contract actions and found them to contain a mix of outcome-based criteria and process-based criteria. For example, three award fee plans covering five contract actions contained criteria related to meeting schedule deadlines, which we consider to be related to outcomes. One award fee plan contained criteria related to management, such as proactivity and responsiveness to problems, which we consider to be related to process. In the nine contracts we reviewed, we also found examples of award fees based on both outcome- and process-based criteria. For example, one contract, valued at $78 million for management and operation of transportation services on an Army installation, specified that 25 percent of the award fee would be based on technical performance criteria. Under these criteria, points were assigned on the basis of quality and timeliness assessments, which we consider to be outcome-based, as well as reporting, which we consider to be process-based. Similarly, a $16 million Army contract for installation support services based 75 percent of its award fee on a “Performance of Work” criterion, which contained several sub-criteria including “Quality” and “Efficiency/Timeliness,” both of which contained a mix of outcome- and process- based criteria. Even where award fee plan criteria were outcome-based, we could not assess whether the outcomes cited were identified by DOD as positive outcomes for the acquisition of services. As we have previously reported, DOD has struggled to define and track desired outcomes for services contracts, which differ from products in several aspects and can pose challenges to establishing measurable and performance-based outcomes.

DOD Conducts Limited Data Collection on Incentive and Award Fee Contracts and Does Not Assess Incentive Outcomes Across Contracts

DOD has not consistently assessed how the selection of a particular contract type or incentive arrangement has promoted the achievement of cost, schedule, or technical performance goals. A fiscal year 2007 legislative provision directed DOD to collect and evaluate relevant data on incentive and award fee payments on a regular basis to determine the effectiveness of incentives for improving contractor performance and achieving desired program outcomes, and in 2009, the FAR was updated to require this of all agencies. As a result, DOD previously required military departments and defense agencies to collect data on incentive and award fees—such as the amount of fees available on each contract and the amount paid to contractors—twice a year for contracts with incentive provisions greater than $50 million. According to a senior DPAP official, the information collected was not being used at the time, as DOD saw more value in focusing on efforts prior to contract award than analyzing incentive trends. Additionally, DPAP officials noted that the effort amounted to a manual data collection exercise during a time of reduced staffing levels. DOD rescinded this requirement in April 2015, explaining in the Federal Register that it could obtain relevant data through other sources, including CBAR and peer reviews. Our review found, however, that DOD is not using these sources to assess the effectiveness of incentive contracts. Further, we found that CBAR, peer reviews, and other potential sources DOD identified have limited utility in providing information to assess the effectiveness of improving contractor performance and achieving desired program outcomes (see table 6).

These systems have distinct purposes and are not specifically intended to provide information for DOD to analyze how well it is achieving incentive outcomes, though some have the potential to provide insight into outcomes. For example, DOD officials said they used Cost and Software Data Reporting and earned value management data—which are maintained in the CADE system— to evaluate the effectiveness of factors that motivate contractor performance in its 2014 annual report on performance of the defense acquisition system. Among the contracts that DOD reviewed, it found that those with incentive fees or profits typically experienced lower cost growth than other contract types. This analysis, however, used total contract costs (including non-incentivized portions), according to senior DOD officials. DOD has not conducted this analysis in subsequent annual reports. DOD has also aggregated feedback from past peer reviews—including reviews focused on incentives—and identified lessons learned and best practices for structuring incentives.

DOD’s efforts to better manage its acquisition of services, which accounted for more than half of the $274 billion in total DOD contract obligations in fiscal year 2015, could also help assess the merits of using incentives for various portfolios of services. As part of these efforts, DOD has identified senior officials within DOD to serve as functional domain experts responsible for specific portfolios of services. DOD’s January 2016 services acquisition instruction tasked these experts, among other responsibilities, to identify and share portfolio group best practices and employ lessons learned to improve the acquisition and management of services across their respective categories, and develop appropriate metrics to track cost and performance of contracted services within the portfolio group to leverage best practices, reduce redundant business arrangements, identify trends, and develop year-to-year comparisons to improve the efficiency and effectiveness of contracted services.

We have ongoing work to assess how DOD and the military departments are implementing aspects of DOD’s services acquisition instruction. As we noted in our February 2017 high-risk update, however, DOD does not have an action plan that would enable it to assess progress toward achieving its goals for improving service acquisitions, and its efforts to develop goals and associated metrics unique to each category of service it acquires are in the early stages of development.

As previously noted, the FAR requires agencies to collect and evaluate relevant data on incentive and award fee payments on a regular basis to determine the effectiveness of incentives for improving contractor performance and achieving desired program outcomes. Since DOD removed the requirement from its own regulations in April 2015, it has not identified a new approach to collect the required information beyond using CBAR and peer reviews, which we found do not allow DOD to assess incentive outcomes as currently used. It may not be necessary to collect data on each individual contract, but rather identify what best meets the department’s needs for assessing performance outcomes and collect data accordingly. Without assessing how incentives have contributed to intended outcomes, contracting officers may be at a disadvantage in establishing appropriate incentives for the requirements they work to fulfill through contracts.

Conclusions

The government should be vigilant about getting the best value for its dollar, and incentive and award fees can be effective tools for motivating contractors and achieving desired outcomes for DOD acquisitions, if appropriately applied. Over the past decade, DOD’s guidance and training has emphasized the use of objective incentives, which is reflected in a noticeable shift toward incentive fee contracts and away from the more subjective award fee contracts. As some of our prior work has found, however, incentives do not always lead to better outcomes. Given the emphasis on cost incentives, it is important that DOD determine whether and under what circumstances the use of these incentives is achieving intended cost objectives. Federal acquisition regulations require DOD to collect and analyze information on the use of incentives. However, it may not be necessary for DOD to embark on a broad, manual data collection effort similar to what has been tried in the past. Rather, DOD could focus its effort on specific areas of interest or risk to the department. For example, given the widespread use of incentive contracts on major weapon systems, DOD could continue to focus its analyses on the factors that facilitate or hinder the achievement of cost objectives or consider expanding the collection of information on lower dollar weapon systems. Alternatively, DOD could focus more on identifying what types of incentives prove useful for the different services DOD acquires as part of its efforts to manage portfolios of services acquisitions. Such efforts should not preclude DOD from continuing to assess other approaches to motivate contractor performance, such as the use of technical and schedule incentives. Prioritizing the area or areas DOD intends to focus on would enable the department to determine how best to collect that information and, in turn, use it to identify opportunities to improve the use of incentive contracts.

Recommendation for Executive Action

We recommend that the Under Secretary of Defense for Acquisition, Technology and Logistics identify the specific types of information that would best meet the department’s needs and, based on that determination, collect and analyze relevant data after contract performance is sufficiently complete to determine the extent to which contracts with incentives achieved their desired outcomes.

Agency Comments

We provided a draft of this product to DOD for comment. In its comments, reproduced in appendix III, DOD concurred and indicated it will establish a process for identifying specific types of information to collect and assess the data after the completion of contract closeout to determine the extent to which incentives achieved their desired outcomes. DOD stated it will complete this process by the second quarter of fiscal year 2018. DOD also provided technical comments, which we incorporated as appropriate.

We are sending copies of this report to the appropriate congressional committees and the Secretary of Defense; the Under Secretary of Defense for Acquisition, Technology, and Logistics; the Secretaries of the Air Force, Army, and Navy; the President, Defense Acquisition University; and the Director, Office of Management and Budget. In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-4841 or DiNapoliT@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in Appendix IV.

Appendix I: Objectives, Scope, and Methodology

The objectives of this review were to (1) identify steps the Department of Defense (DOD) has taken to improve its use of incentive contracts since 2010, and (2) assess the extent to which selected DOD incentive contracts achieved desired acquisition outcomes.

To identify the steps DOD has taken to improve its use of incentive contracts since 2010, we reviewed relevant legislation and provisions within the Federal Acquisition Regulation (FAR); memoranda issued by the Office of Management and Budget (OMB); and identified and reviewed changes to DOD regulations, policies, and guidance regarding the use of incentive contracts. To identify changes in DOD’s use of these contract types over time that may have corresponded with regulatory and policy changes, we analyzed data from the Federal Procurement Data System-Next Generation (FPDS-NG) on obligations by contract type for fiscal years 2005 through 2015. We reported our findings in constant fiscal year 2015 dollars, adjusted for inflation using the fiscal year gross domestic product price index. To assess the reliability of the FPDS-NG data, we conducted electronic testing of the data. We also reviewed a selection of 53 contracts in FPDS-NG and, in reviewing contract documents, found that eleven percent was incorrectly coded as being incentive contracts. We assessed these entries and found that they represented a range of contract award dates and products and services, and generally aligned with the distribution across military departments of contract actions in our sample. We determined that the miscoded contracts had minimal potential for impacting our analysis and that the data were sufficiently reliable to report general trends in DOD’s obligations for incentive contract actions. In addition, for a sub-selection of the 53 contracts and orders, we traced selected data fields to contract file documents to verify their accuracy. We also interviewed DOD, military department, and DAU officials about efforts to improve the use of incentive contracts.

To determine the extent to which incentive provisions in selected DOD contracts achieved desired outcomes in cost, schedule, and technical performance, we assessed a nongeneralizable sample, drawn from FPDS-NG data, of incentive contracts and orders that DOD awarded between fiscal years 2011 and 2015 and that were reported as completed by the end of fiscal year 2015. We chose these timeframes so as to select awards that were made after regulatory and policy changes starting in 2010 and had completed performance by the time of our review. Because of these parameters, certain types of contract actions were inherently excluded from our analysis, including those that exceeded a performance period of 5 years, such as longer term satellite and shipbuilding contracts.

For our contract selection, we excluded contracts with values below the $150,000 simplified acquisition threshold, indefinite delivery contracts, and blanket purchase agreements. We initially selected all 38 contracts that met these criteria. Out of more than 4,200 orders, we selected a subset of 15 that (1) included different incentive types under the same base contract, (2) were used to purchase similar products and services, and (3) reflected a range of dollar values. These parameters yielded an initial sample of 53 contract actions (contracts and orders). Eighteen of the 53 actions were dropped from our sample either because the contracts were miscoded in FPDS-NG and were not incentive contracts based on our review of contract documents (6), were terminated before performance began (1), or were actually not yet complete (11). Consequently, we reviewed a total of 35 contracts and orders representing fixed-price incentive (FPI), fixed-price-award-fee (FPAF), cost-plus-incentive-fee (CPIF), and cost-plus-award-fee (CPAF) contract types (see table 7). Because we used a nongeneralizable sample of contracts and orders, results from this sample cannot be used to make inferences about all incentive contracts and orders that DOD awarded.

In total, we reviewed 5 contract actions from the Air Force, 7 from the Army, and 23 from the Navy. For each selected contract and order, we collected relevant documentation, such as the initial contract or order, modifications, statements of work, determination and findings memoranda, award fee plans, and performance evaluations. We interviewed contracting officials and obtained their written input to clarify and collect additional information as needed.

To determine cost performance on CPIF and FPI contracts and orders, we compared actual or estimated costs to target costs listed in the conformed contracts or applicable contract modifications specifically for incentivized line items. For contracts where actual costs were not yet finally settled, we obtained the best available current estimate from contractor cost reports and similar sources. For contracts with multiple line items with incentives, we compared the total target costs to total actual or estimated costs. Within these types of contracts, if the contractor performed one line item at a cost overrun but another at a cost underrun, only the net results would be reflected in our findings. For contracts and orders with schedule and technical performance incentives, we identified schedule and technical performance goals and outcomes using the contracting office responses and contract documents. We obtained total action values from the conformed contracts. For contracts and orders containing award fees, we calculated the amount of award fees earned based on data provided by the contracting offices, including total action values and total potential award fee amounts, and corresponding contract documentation. We examined award fee plans to identify provisions related to rollover and criteria for award, and we examined award fee board memos and other documents to understand how award fee amounts were determined.

To supplement our understanding of how incentive provisions helped achieve acquisition outcomes, we interviewed two contractors about the effectiveness of incentive provisions in motivating performance toward desired outcomes. To identify contractors to interview, we focused on a sub-selection of contracts containing multiple incentives to obtain perspectives on how the incentives interacted with respect to the contractors’ performance. The information obtained from interviews with the two contractors cannot be generalized to all contractors; however, the interviews provide important insights on the experiences of contractors.

We also reviewed documents and interviewed DOD officials about relevant data systems and processes used to collect and analyze contract and program data, including the Contract Business Analysis Repository, the Cost Assessment Data Enterprise, the Contractor Performance Assessment Reporting System (CPARS), and peer reviews, to understand how these systems may provide some insight into DOD’s use of incentive contracts. Specifically, we reviewed 13 CPARS reports from contracts in our sample, selected for a distribution among the military departments and various contract types. We also reviewed a nongeneralizable selection of 30 peer review summaries from fiscal years 2014 and 2015 to develop an understanding of how peer reviews were used to provide guidance on key contracting decisions.

The performance audit upon which this report is based was conducted from January 2016 to May 2017 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. DOD deemed that certain information in the draft report related to contract costs and performance was sensitive and must be protected from public disclosure. We subsequently worked with DOD from May 2017 to July 2017 to revise our presentation of this information and prepare this report for public release.

Appendix II: Trends in Obligations for Incentive Contracts by DOD Component from Fiscal Years 2005 through 2015

We analyzed data from the Federal Procurement Data System-Next Generation on incentive contract obligations by Department of Defense (DOD) component: Air Force, Army, Navy, and all other DOD. We found that, similar to DOD, the three military departments increased their obligations for incentive fee contracts overall from fiscal years 2005 through 2015. The only notable anomaly we found was that other DOD components obligated fewer dollars for incentive fee contracts over the period we reviewed (see figure 4).

We also found that each of the military departments as well as other DOD components had decreased their obligations for award fee contracts from fiscal years 2005 through 2015 (see figure 5).

Appendix III: Comments from the Department of Defense

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact name above, the following staff members made key contributions to this report: Janet McKelvey (Assistant Director), Pete Anderson, MacKenzie Cooper, Brenna Derritt, Alexandra Dew Silva, Lorraine Ettaro, Kurt Gurka, Julia Kennon, Liam O’Laughlin, Carol Petersen, Raffaele (Ralph) Roffo, Ann Marie Udale, and Robin Wilson.