Satellite Acquisitions: Agencies May Recover a Limited Portion of Contract Value When Satellites Fail
Released: 2017-06-09T00:00:00

Letter

Background

In any acquisition, the contract type provides the foundation for incentivizing a contractor’s performance and is just one element of the contract, which may also include performance, cost, or delivery incentives, and other contract terms and conditions that incentivize performance. The type of contract used for any given acquisition inherently determines how risk will be distributed between the government and the contractor. Since the contract type and the contract price are interrelated, they must be considered together. The government’s objective is to negotiate a contract type and price (including cost and the contractor’s fee or profit) that will result in an acceptable level of risk to the contractor, while also providing the contractor with the greatest incentive for effective and efficient performance.

Incentive contracts, which include award fee contracts, are designed to attain specific acquisition objectives by including incentive arrangements that (1) motivate contractor efforts that might not otherwise be emphasized, and (2) discourage contractor inefficiency and waste. One of the main characteristics of award fees and other incentives is how they are administered in the contract. Award fees are generally subjectively determined and incentive fees are generally objectively determined. When incentive arrangements are done properly, the contractor has profit motive to keep costs low, deliver a product on time, and make decisions that help ensure the quality of the product. Our prior work has shown, however, that incentives are not always effective tools for achieving desired acquisition outcomes and that, in some cases, there are significant disconnects between program results and incentives paid. Additionally, we have repeatedly found that some agencies did not have methods to evaluate the effectiveness of award fees. More recently, in March 2017, we found that fixed-price incentive shipbuilding contracts did not always lead to desired outcomes, and that the Navy had not assessed whether adding incentives improved contractor performance. We made recommendations to the Navy, including that it conduct a portfolio-wide assessment of its use of additional incentives on fixed-price incentive contracts across its shipbuilding programs. DOD agreed with our recommendations.

Numerous contract types are available to the government to provide flexibility in acquiring the supplies and services agencies need, including satellite acquisitions. Table 1 provides an overview of typical contract types and how they may be used to acquire satellites.

When determining the contract and incentive structure for satellite acquisitions, the government may consider a number of factors:

Where the satellite is in the acquisition phase. Satellite contracts can include design, development, integration, and testing, and can cover more than 15 years. The government can tailor the contract type and fees to meet the specific circumstances of the acquisition, for example, the phase of the acquisition cycle. Figure 1 shows the notional acquisition phases for a satellite.

During the design and development phase, when technology risks are higher, the government typically uses cost-reimbursement contracts. When the government is acquiring a “production-model” satellite, or a copy of a satellite with a proven design and build, a fixed-price contract may be used. In addition, for the on-orbit phase, the government typically negotiates with the contractor regarding the amount of incentives related to successful on-orbit performance.

Which satellite component is being acquired. Satellites are generally comprised of the bus and the instruments or payloads. The bus is the body of the satellite. It carries the payload and is composed of a number of subsystems, like the power supply, antennas, telemetry and tracking, and mechanical and thermal control subsystems. The bus provides electrical power, stability, and propulsion for the entire satellite. The payload of a satellite, which is carried by the bus, refers to all the devices or instruments a satellite needs to perform its mission, and can differ for each type of satellite. Some examples include cameras to take pictures of cloud formations for a weather satellite, and transponders to relay television signals for a communications satellite. Generally, developing payloads is riskier than developing buses.

Number of contracts used to acquire satellite components. The government can choose to award one contract for the development and production of the satellite bus and all associated payloads, or it can award separate contracts for the development and production of the bus and each individual instrument. In the contracts we reviewed, DOD typically awarded a single contract to a prime contractor for the development of the bus and payloads, and according to officials, the prime contractor often awards contracts to subcontractors to provide the various instruments and parts. In contrast, for the NASA contracts and orders we reviewed, NASA typically awarded separate contracts and orders to multiple contractors to develop the bus and each instrument.

Number of space vehicles being acquired on a contract. The government can acquire a single satellite or multiple satellites under one contract, and one contract may also include components for multiple satellites. Further, depending on the mission, an agency may buy a single satellite (such as for scientific discovery) or blocks of satellites (such as for global communications).

On-Orbit Performance Incentives

The quality and performance of a satellite—including whether it meets on- orbit requirements—usually cannot be determined until after it launches and reaches its intended orbit. Satellite development and production contracts may include on-orbit performance incentives aimed at ensuring that the satellite will meet performance requirements. Because the incentive is not earned until a satellite successfully demonstrates its performance on-orbit, an on-orbit performance incentive can be thought of as the dollar amount at risk if the contractor fails to meet the performance requirements specified in the contract (see figure 2).

On-orbit incentives are typically documented in a satellite contract’s fee plans or in contract clauses. There are generally three mechanisms for on-orbit performance incentives: negative incentives, positive incentives, and withholding of milestone payments. Agencies can use a combination of these when designing on-orbit incentives, which are generally tied to objective performance criteria, such as successfully getting into the right orbit and achieving critical performance parameters once there.

When using negative incentives, the government generally pays the contractor incentives as the contractor completes work during the development or production phase. The contractor would have to pay back some or all of the incentives if the satellite fails to meet on-orbit performance parameters. With positive incentives, the government assesses satellite performance in any given period, such as 6 or 12 months, to determine how much the contractor earns for that period. The amount at risk for the contractor could be the same for a positive incentive or a negative incentive.

Firm-fixed-price contracts for satellite programs may have progress or performance-based payment plans that require fixed payments to be made upon successful completion of milestones, such as preliminary design review, final system test, and successful on-orbit check-out. If a satellite fails prior to check-out, the government may withhold the final milestone payment from the contractor.

Because satellite acquisitions are unlike other acquisitions, the Air Force’s Space and Missile Systems Center (SMC) and NASA have tailored guidance on space acquisitions, and each specifically addresses on-orbit performance incentives.

SMC’s March 2007 incentives guide states that on-orbit incentives should be clearly written to ensure enforceability, and require fully demonstrable performance in order to earn fee. The guidance also specifies that the contract should contain specific descriptions as to the rights of the parties in the event of a failure, caused either by the contractor or by the government. One senior SMC contracting official stated that their contracting directorate maintains a repository of on- orbit incentive plans and contract clauses that have been implemented. These incentives arrangements can be used as guides for SMC programs as they develop new contracts.

NASA’s FAR supplement generally requires certain contracts for hardware deliverables worth more than $25 million to include performance incentives. In the case of satellite acquisitions, the on- orbit incentives serve as performance incentives. In addition, NASA’s FAR supplement requires that for cost-plus-award-fee contracts for end items, where the true quality of contractor performance cannot be measured until the end of the contract, only the last evaluation is final. This allows NASA to evaluate the contractor’s total performance—after the end item is delivered—against the award fee plan to determine the total earned award fee from the contract award fee pool. In other words, under this “re-look provision,” the total award fee is not earned until the satellite has demonstrated its performance on-orbit.

In addition to on-orbit performance incentives, it is important to acknowledge the totality of the incentives and other terms in satellite contracts that are designed to motivate contractors to achieve the cost, schedule, and performance goals of the program. In many cases, multiple incentives are used to achieve such goals. The FAR provides that when multiple incentives are used, a balance must be achieved in which no incentive is either so insignificant that it offers little reward for the contractor, or so large that it overshadows other areas and neutralizes their motivational effect. In addition, it requires all multiple incentive contracts to include a cost incentive that precludes rewarding a contractor for superior technical performance when their cost outweighs their value. For satellite programs, achievement of full mission performance is the primary objective, but cost and schedule goals also play key roles in a program’s success. Our body of prior work has shown that most major government satellite programs experience significant cost growth and schedule delays due to unmatched resources and requirements, immature technologies at program start, and inconsistent application of knowledge-based practices throughout the life of a program. As a result, although some programs may have successful on-orbit performance, they may also have had major cost overruns and schedule delays along the way.

Satellite Failure Definitions and Causes

There are many nuances when it comes to satellite failures. These can be especially important when considering government opportunities for recourse should a satellite fail or underperform. The Aerospace Corporation (Aerospace), a federally funded research and development center (FFRDC) that provides engineering and technical support to national security space programs, categorizes satellite anomalies based on the criticality or severity of the anomaly and distinguishes them based on their impact on the mission. For the purposes of this report, a catastrophic failure results in the total loss of a satellite, or a satellite that will never meet any of its mission requirements; a partial failure results in a satellite that fails to meet some of its mission requirements or that loses a redundant system.

When satellites fail, they often do so in the first few months they are in orbit. Aerospace reported in 2012 that mission-impacting anomalies—or failures—that occur during the first 120 days of a satellite’s on-orbit life account for approximately 40 percent of all such failures that occur during the first 3 years of operation. Causes of failure during a satellite’s first 3 years of operation varied according to Aerospace, but the top three reasons identified were issues related to a satellite’s design, parts, and software.

Agencies Generally Use Cost- Reimbursement Contracts with Incentives for Satellite Acquisitions

Given high development risks and the likelihood of requirements changes in satellite programs, most government satellite acquisitions use cost- reimbursement contracts. New technologies and unstable requirements mean satellite program officials are unlikely to accurately predict development costs and schedules, making cost-reimbursement contracts a prudent choice. When lower-risk items are being acquired, such as standard spacecraft and communications satellites, agencies are more likely to use firm-fixed-price contracts. In both cases, agencies tend to use incentives in their contracts to help achieve cost, schedule and performance goals.

Agencies Primarily Used Cost-Reimbursement Contracts to Acquire Satellites Given Development Risk

Across the 19 programs we reviewed at DOD, NASA and NOAA, about $43.1 billion of $52.1 billion (83 percent) was obligated on cost- reimbursement contracts and orders, while the remaining $9 billion (17 percent) of obligations were on firm-fixed-price and fixed-price incentive contracts and orders. DOD satellite obligations comprised nearly 80 percent of the government satellite acquisitions in our review (see figure 3).

Government satellite programs are often designed to develop and incorporate innovative technologies unavailable in the commercial market. We reported in 2010 that DOD accepts greater technology and development risks with space acquisitions, and as such, costs associated with technology invention are difficult to estimate.

For our 12 in-depth case study programs, 15 of the 23 contracts and orders we reviewed were cost-reimbursement type contracts (see appendix III for more detailed information on the contract types for our case study programs). Several of the contract files for these programs cited technical complexity leading to uncertainties in cost performance or uncertain requirements as reasons their program used cost- reimbursement contracts. According to one program’s contract files, a firm-fixed-price contract would not be appropriate for the design and development of the satellite’s primary instrument because of the program’s aggressive performance goals and flexible launch dates. In 2010, we reported that firm-fixed-price contracting normally does not work for DOD space systems because programs tend to start with many unknowns about the technologies and costs needed to develop satellites. Since our 2010 report, however, DOD has begun to consider acquisition approaches that might be more productive using fixed-price contracts such as disaggregating large satellites and advancing technology incrementally. One senior SMC contracting official noted that in recent years, SMC has acquired a number of satellite programs on a fixed-price basis. In these instances, once the programs matured the technologies and reduced risks, the programs were able to shift to a production-type state and use fixed-price contracts.

Five of the 23 case study contracts and orders used firm-fixed-price type contracts, mostly to acquire lower-risk items. These included standard spacecraft buses (meaning, those with proven designs) and communications satellites, both of which have relatively lower technical risks. For example, some NASA and NOAA programs we reviewed used firm-fixed-price orders to acquire spacecraft buses from the NASA “catalog,” which contains proven designs from multiple contractors. In some cases, such as communication satellites, the commercial market produces satellites that government programs can use with only minor modifications. These satellites have typically been used in a commercial market—lowering technical risk—and may also have adequate pricing data available to accurately estimate costs, both of which lend themselves to firm-fixed-price contracting.

The remaining three of the 23 case study contracts and orders we reviewed used fixed-price-incentive contracts to acquire additional satellites in production. For instance, DOD’s SBIRS program bought its first four satellites under cost-reimbursement contracts, but bought the next two satellites with a fixed-price-incentive contract. According to the SBIRS contract file, program officials deemed the fixed-price incentive contract type the most appropriate because, given the maturity of the satellite vehicle design, they believed they could assess a fair and equitable ceiling price for the acquisition. Fixed-price incentive contracts can be complex, as noted in SMC’s guide to structuring incentives for fixed-price contracts, issued in November 2012. For example, how risk is apportioned on a fixed-price-incentive contract may resemble that of a cost-reimbursable contract or a firm-fixed-price contract, depending on the share ratio—a calculation which represents the allocation of cost risk between the government and the contractor—and the ceiling price. The guide also notes that fixed-price-incentive contracts can lead to unintended, negative outcomes if structured poorly or managed improperly.

Satellite Acquisition Programs Generally Used Incentives to Help Achieve Cost, Schedule, and Performance Goals

All of our 12 case study programs used incentives—including award fees, incentive fees, or some combination of the incentives—to motivate contractors to achieve cost, schedule, or performance goals. Program officials told us they used award fees during the development phase to incentivize the contractor to achieve specific, short-term goals. With each award fee determination, which, for example, may occur every 6 months, the government can focus the contractor on specific tasks or areas. Program officials also said that award fee determinations can be effective in changing contractor behavior. For example, one NASA program official stated that during a particular award fee period, they informed the contractor that it needed to address planning for a complicated spacecraft thermal vacuum test that was behind schedule. The official said that after program officials documented their concerns in award fee letters, contractor performance improved, resolving the issue. Table 2 provides examples of how the government tied incentives to cost, schedule, and performance objectives for selected case studies.

However, in addition to the 12 in-depth case study programs, several of the NASA programs in our review included contracts or orders with FFRDCs, academic institutions, or non-profit organizations that did not include incentives. In most of these cases, NASA awarded cost-plus- fixed-fee or cost with no fee contracts or orders. For example, several programs used cost-plus-fixed-fee orders issued from an indefinite delivery / indefinite quantity contract between NASA and the California Institute of Technology, a private nonprofit educational institution, which establishes the relationship for the operation of the Jet Propulsion Laboratory (JPL), an FFRDC. The contract includes both service and product deliverables and encompasses a large number of discrete programs and projects. According to NASA procurement officials, the desired program outcomes or objectives and performance requirements are defined in task orders issued under the contract. NASA officials stated that when there are no fees to pay or withhold from a contractor, NASA still has tools available to motivate contractor performance. In these instances, agencies rely on non-fee incentives, such as providing positive or negative evaluations of contractors in the Contractor Performance Assessment Reporting System (CPARS). The CPARS evaluations may affect the ability of FFRDCs and academic institutions to secure future contracts. Officials also stated the general reputation of contractors are important in winning future contracts. NASA officials also stated that with science missions, the academic institutions are self-motivated because they are interested in the data that satellites are collecting and within the science community, there is a sense of pride to be associated with a successful NASA mission.

On-Orbit Incentives Varied among Selected Programs

Eleven of the 12 selected in-depth case study programs we reviewed used contract incentives tied to on-orbit performance on at least one contract. We found, however, that the characteristics of the on-orbit incentives—such as the amounts at risk or the timing of the incentive payments—varied widely. In one instance, a satellite program held the contractor’s entire fee at risk pending demonstration of satellite performance, but in other cases, the at-risk amount was a portion of the total fee or profit on the contract. Timing of payments also differed on a program-by-program basis, and in some cases, satellites within the same program had distinctive incentives. When asked how they developed on- orbit incentives for their respective contracts, program officials cited considerations such as other incentives in the contract, agency acquisition preferences, and past history.

Selected Programs Had Diverse On-Orbit Incentive Amounts and Payment Timing

On-orbit incentives varied widely across our 12 case study programs and in some cases, incentives varied between satellites under the same contract. To compare on-orbit incentives across these programs, we identified the two key characteristics that define each set of incentives— specifically, the amount and timing of the incentives.

On-orbit incentive amount. Based on our analysis of 23 contracts and orders representing our 12 case study programs, we found that the amount of on-orbit incentives relative to the overall contract value varied widely (see figure 4).

On-orbit incentives included on the contracts and orders for our 12 case study programs ranged from no on-orbit incentive to approximately 10 percent of the contract value (see appendix III for more information on the on-orbit incentive amounts by contract). In some cases, on-orbit incentives were only a portion of the contractor’s expected fee or profit. For one contract we reviewed, the contractor’s full fee was contingent upon successfully demonstrating that the satellite met on-orbit performance requirements. Most of the contracts and orders we reviewed included multiple satellite vehicles, and the on-orbit incentive could vary by vehicle. For example, in one DOD contract we reviewed, the at-risk amount for the entire contract was 7 percent, but the at-risk amount for individual vehicles ranged from 5 percent to 13 percent.

The at-risk amount for individual satellites can also change depending on contractor performance during satellite development. Several contracts we reviewed define the potential on-orbit incentive amount as a percentage of the total award fee available, rather than a specific dollar amount. This means that poor contractor performance during the satellite development phase could reduce the amount of the on-orbit incentive. For example, if the total award fee for a contract is $100, and the on-orbit incentive is defined as 50 percent of the available award fee, the contractor could potentially earn $50 for on-orbit performance. If, however, the contractor loses $30 in potential award fee during the satellite development phase due to poor performance, the most the contractor could earn for on-orbit performance falls to $35 (50 percent of $70). Similarly, in one contract we reviewed, the at-risk amount on-orbit was capped as the lesser of 50 percent of the contractor’s realized profit or 50 percent of the target profit. In this case, if the contractor’s realized profit was zero dollars, there would be no payback to the government in the event of a failure.

Because on-orbit incentives are realized near the end of a contract performance period, they can grow in importance to the contractor, relative to other incentives. For example, a contracting officer for one of our case study programs said that his program’s contract placed an equal priority on both on-orbit performance and cost and schedule incentives, but the contractor lost most of the cost incentives due to cost overruns. The contracting officer said that the on-orbit incentive was the largest remaining incentive available to the contractor, so in this case, the contractor was more intent on earning the on-orbit incentive and less focused on controlling costs—a potentially bad situation for the program.

Incentive timing. The timing of an on-orbit incentive represents how the at-risk amount is spread out over a satellite’s mission life. For some of the contracts and orders we reviewed for selected case study programs, the on-orbit incentives covered a satellite’s entire mission life. There were some contracts and orders, however, for which the on-orbit incentives covered only a portion of the mission life. Given that when satellites fail, it is usually early in their mission life, three of the five firm-fixed-price satellite contracts or orders in our case study programs had milestone payment plans that ended once the on-orbit vehicle completed its check- out. Under this arrangement, depending on the terms of the contract, the government could potentially withhold the last milestone incentive payment if a catastrophic failure occurred prior to check-out, but would not have on-orbit incentives that lasted the remainder of the mission life. In these cases, the government assumes all of the risk once the vehicles complete check-out. Similarly, the on-orbit incentives for one of the SBIRS contracts covered the first 4 years of the satellite’s mission life.

The WGS Block II and Block II follow-on contracts have a unique on-orbit incentive structure that includes a 10-year negative incentive followed by a 4-year positive incentive for the satellite’s 14-year mission life. The negative incentive includes calculations to determine how much money the contractor has to pay the government if its satellite fails to meet performance requirements during the first 10 years. The positive incentive, starting at year 11 of the satellite’s mission life, allows the contractor to offset any negative incentives assessed during that satellite’s first 10 years. At the end of 14 years, the government adds up the positive and negative incentive amounts to determine what, if anything, the contractor has to pay back.

Satellite or component storage on the ground may also affect on-orbit incentives. For example, the GPS IIF contract reduces the on-orbit performance period by 25 percent if any of the first six vehicles is stored on the ground for 4 years; more if stored longer.

Contracting Officials across Selected Programs Cited Several Considerations in Developing On-Orbit Incentives

DOD, NASA, and NOAA officials cited several factors when developing on-orbit incentives for their respective programs, including the other incentives in the contract, agency acquisition preferences, and individual program history.

The overall contract. Program officials do not view on-orbit incentives in isolation, but rather as part of the larger negotiated agreement with the contractor. Officials told us that during contract negotiations, they may be willing to reduce the amount of fee on-orbit or alter the timing of on-orbit incentives, in return for contractor concessions in other areas. Programs negotiated overall incentives for a contract as well as how the incentives were spread out over the contract, including how much was tied to on-orbit performance. Officials said contractors generally prefer front-loaded incentives, whereas agencies may tend to prefer placing incentives at the end of the contract. The resulting contract and incentive structure depends on what the government and the contractor can agree to.

Acquisition philosophy. Program officials told us on-orbit incentives reflect the acquisition policies, leadership preferences, and prevailing agency practices at the time the contract is being drafted. For example, DOD promotes the use of incentive fees, where possible, over award fees, to encourage greater use of objective fee criteria. Program officials also said they typically look at other satellite program contracts in their respective agencies, explaining that incentives for new programs may be structured based on what other programs have agreed to.

Program history, staff, and contractor experience. Program history can affect incentive structures in different ways. In the case of WGS, the on-orbit incentives for the Block II follow-on contracts were modeled after the incentives in the Block II contract. Further, program officials said they also modified past incentives or established new incentives based on their own contracting experience. They told us that prior experience executing specific incentive structures lends itself to structuring incentives the same way again. Similarly, they said that contractors may seek to negotiate incentive provisions based on their company’s past experience.

Government’s Recourse Is Limited When a Satellite Fails

The government’s recourse in the event of a catastrophic satellite failure on-orbit is generally limited to recovery of a portion of the on-orbit incentive. As discussed above, the on-orbit incentive amount on any given satellite contract can vary widely but is uniformly less than 10 percent of the total satellite contract value. In all likelihood, the amount retained by or paid back to the government would be even less, as what the government may actually recoup depends on the circumstances of the failure. For example, if a contract includes no-fault provisions, when one contractor is at fault for the total loss of a satellite, the government may still be responsible for paying fees to the remaining contractors— whose products were not to blame for the failure. By design, on-orbit incentives are only a portion of the total contract value and therefore will not make the government whole in the event of total failure. Overemphasizing on-orbit incentives could result in the contractor losing sight of cost and schedule goals. Further, the government accepts more of the on-orbit risk than the contractor, in part because catastrophic failures are rare, according to satellite studies and industry experts we spoke to.

Modest Payback to Government When Satellites Fail

For our 12 case study programs, we found that the aggregate on-orbit incentive amount at risk is around 4 percent of the aggregate contract value. This means that in a worst-case scenario in which all of the satellites failed prior to checkout, the maximum amount the government could recoup or withhold from the contractors is 4 percent of the total contract value. What the government could actually recoup or withhold depends on the terms of the contract and the circumstances of the failure, such as the extent to which technical parameters are met, the timing of the failure, and whether the contractor is found to be at fault.

Two satellites among the programs we reviewed experienced catastrophic or partial failures—DMSP-19 and SMAP. Program officials said the DMSP-19 spacecraft contractor repaid $2.7 million plus interest to the government as a result of the failure, but there was no repayment or payments withheld in the case of SMAP.

DMSP-19 suffered a catastrophic failure in the second year of its 5- year mission life when the Air Force lost the ability to control the satellite. There were two prime contractors on the DMSP contracts—one for the spacecraft and one for the sensors. According to program officials, the spacecraft contractor was found to be at fault for the failure, and had to reimburse $2.7 million in fee plus interest. Program officials stated that the sensor contractor was not responsible for the failure and therefore did not have to repay any fee.

SMAP experienced a partial failure. One of SMAP’s two primary sensors—the radar—failed, while the other—the radiometer—is operating as intended. NASA was unable to identify the exact cause, but determined that the failure was related to the radar’s high- powered amplifier power supply. According to NASA’s SMAP mishap investigation report, without the radar, SMAP will not be able to meet mission requirements because the radiometer alone cannot meet resolution requirements. Because JPL built SMAP under a task order with no on-orbit incentives, the government had no monetary recourse when the radar sensor failed. The mishap investigation report estimated that the SMAP radar failure resulted in more than $550 million in losses, though the cost of the radar accounted for only 11 percent of that amount. Most of the estimated losses were related to investments in the science of the mission. However, NASA officials told us that the total science value of the shortfall in capability is highly uncertain. These officials noted that although not operating to its full resolution, SMAP provides higher resolution and more accurate soil moisture and sea surface salinity data than any prior NASA missions.

No-fault provisions can have implications for satellites with multiple prime contractors, as mentioned in the DMSP example. If one contractor is at fault for the total loss of a satellite, the government may still be responsible for paying fees to the remaining contractors—whose products were not to blame for the failure—even though the satellite on which their products were riding was a total loss.

Representatives from commercial companies we spoke with said they typically purchase insurance to mitigate their risk in the event of satellite losses. According to satellite insurance company representatives, insurance for any one satellite is generally spread across multiple insurance companies, each of which insures only a portion of the total value of the satellite. The cost of insuring the launch and full mission life of a commercial satellite could add 10 to 20 percent of its total contract value, according to one insurance broker we spoke with, even though relatively few satellites suffer significant failures. He noted that, at this time, the small market of satellite insurance providers would not have the capacity to insure many government satellites, given their high costs to build and launch.

Overemphasizing On-Orbit Incentives May Not Benefit Programs

While tying some of a satellite contract’s incentive to on-orbit performance can help focus a contractor on building a quality satellite, overemphasizing performance through on-orbit incentives can unintentionally cause the contractor to lose sight of cost or schedule. For example, JWST program officials stated that the $56 million on-orbit incentive fee in the initial JWST contract encouraged the contractor to exceed performance requirements at the expense of cost and schedule. In other words, the cost and schedule incentives were relatively less significant to the contractor than the on-orbit performance incentive. In December 2014, we found that when the contract was renegotiated in December 2013, the JWST program and the contractor agreed to replace the on-orbit incentive with award fees that could be used to incentivize cost and schedule goals during development. Since that time, JWST officials told us the renegotiated award fees had contributed to better cost and schedule outcomes. Further, in another program’s contract negotiation documents we reviewed, the government was willing to put less of the contractor’s fee at-risk on-orbit in exchange for lower overall fees, which would reduce the contract price. Conversely, negotiation documents reflected that the government considered accepting higher overall fees in exchange for putting more of the contractor’s fee at-risk.

Also, it is unclear whether increased on-orbit incentives would decrease the likelihood of on-orbit failures. Program officials expressed a wide range of views on the relative importance of on-orbit incentives in achieving successful outcomes. Program officials agreed that poorly designed incentives might lead to bad program outcomes, but there was no consensus on the effect of well-designed incentives. Officials we spoke with believed that on-orbit incentives were important, but the reasons cited were sometimes more related to how they can used in negotiations with the contractor—leading up to contract award and, in some cases, for contract modifications—than achieving successful on- orbit performance. Attributing positive on-orbit performance directly to on- orbit incentives is challenging, given the many factors that contribute to a satellite program’s success, including requirements and funding stability, technology maturity, and government and contractor experience. Further, on-orbit incentives are unlikely to flow down to the workers who actually build a satellite, or to sub-contractors who produce key parts and components. On-orbit incentives can span 10 years of performance, so the people who were directly involved in building a satellite may not even be with the company when the incentives are paid out.

Although contractors may be motivated to achieve on-orbit performance through on-orbit incentives, they are also motivated by other factors. For example, in 2005, we found that various considerations, such as securing future contracts with the government, can be stronger motivators than earning additional profit. Officials from agencies and commercial companies that we spoke to confirmed that this is still true today. Specifically, program officials we interviewed stated that contractors react strongly to negative CPARS evaluations, as this could affect their ability to win future contracts. Officials also noted that contractors take pride in their work and believe in the missions their satellites support. They believed that contractors would do their best to succeed even without on- orbit incentives. They said that universities and FFRDCs are also motivated to do well because they are personally invested in advancing their scientific pursuits. According to program officials and commercial company representatives that we spoke to, the commercial satellite market is small but competitive, and satellite failures generate bad publicity that could affect a company’s ability to win commercial contracts and future government business.

Finally, satellite acquisition programs tend to have far more cost and schedule challenges than performance issues. We have a large body of work identifying cost growth and schedule delays for space programs, including some in our case studies, and have made a number of recommendations to address the causes of these challenges. For example, we have previously recommended that agencies should improve their program cost estimates, separate the process of technology discovery from acquisition, and match resources and requirements at program start. The cost growth for some of our case study programs is much larger than their on-orbit incentives. For example, JWST has experienced more than $3.6 billion in cost growth. That exceeds the combined on-orbit incentives for all 56 of the satellite vehicles in our case study programs. Similarly, the first two GPS III satellites have experienced more than $600 million in cost growth. That is more than double the entire amount of on-orbit incentives for all 10 GPS III satellites currently under contract.

A number of government officials and commercial companies we spoke with did not express concerns about the extent to which satellites experience catastrophic failures. Studies of satellite reliability vary depending on the timing and scope of the analysis, but the analyses we reviewed indicated that between 2 and 4 percent of satellites, including both government and commercial programs, experience a catastrophic failure before the end of their mission lives. Aerospace officials we spoke with said the failure rate for government satellites is around 2 percent, and that this is somewhat remarkable given that government satellites are often highly complex. Further, many government satellites utilize new, unproven technologies which pose more risk than commercial satellites; commercial satellites tend to use proven designs and rely more on mature technologies. As a result, it appears the most cost effective way to limit the government’s loss in the event of a catastrophic failure may be to reduce cost growth and schedule delays by using best practices during satellite development, as we have previously recommended.

Agency Comments

We provided a draft of this report to DOD, NASA, and NOAA for their review and comment. DOD and NOAA provided technical comments, which we incorporated as appropriate. NASA had no technical or written comments.

We are sending copies of this report to the appropriate congressional committees, the Secretaries of Defense and Commerce, and the NASA Administrator. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have questions about this report, please contact me at (202) 512-4841, or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix IV.

Appendix I: Descriptions of Missions, Costs, and Quantities for Satellite Programs in GAO Review

The following tables present the missions, original total program costs and quantities, current total program costs and quantities, and the number of satellites in orbit as of March 2017 for all 19 of the programs included in our review. The total acquisition or project costs include development and production costs of the space vehicles and in some cases, costs associated with the ground systems, launch vehicles, and other costs outside of the satellite vehicles. As a result, the total costs for a given program will be larger than the contracts values discussed in this report.

Appendix II: Objectives, Scope, and Methodology

To assess the Department of Defense (DOD), National Aeronautics and Space Administration (NASA), and the Department of Commerce’s National Oceanic and Atmospheric Administration’s (NOAA) contract and incentive structures, under the authority of the Comptroller General to conduct evaluations on his own initiative, we examined (1) the types of contracts and incentive structures government satellite programs use to develop satellites and why, (2) how selected programs structure on-orbit incentives, and (3) government options for recourse, if any, when a satellite fails or underperforms.

To determine the types of contracts and incentive structures government satellite programs use to develop satellites and the reasons why, we analyzed contract obligations data from the Federal Procurement Data System-Next Generation (FPDS-NG) as of September 2016 for 19 programs, comprising all current major satellite programs across DOD, NASA, and NOAA. To assess the reliability of the FPDS-NG data, we reviewed relevant internal control documents and data quality summaries. We determined that the FPDS-NG data were sufficiently reliable for the purposes of this engagement. For DOD, we included major defense acquisition programs, and at NASA and NOAA, we included programs with a life-cycle cost greater than $250 million. For our analysis, we included each program’s development and production contracts and orders for the spacecraft and instruments and generally excluded contracts and orders related to launch, ground systems, maintenance, operations, and support services if they were separate contracts and orders. If these items or services were included within the development and production contract or order, we included them in our analysis. For each contract and order, we determined the predominant contract type for the design, development, and production of the satellites on the contract. We also reviewed DOD, NASA, and NOAA policies and guidance on contracting and incentives, and Federal Acquisition Regulation (FAR) and agencies’ FAR supplements for regulations and procedures on contracting.

To determine the range of on-orbit incentive structures and government options for recourse should a satellite fail or underperform, of the 19 major satellite programs in our review, we selected 12 programs for in- depth analysis as case studies based on program size, contract type, contractor, mission, or notable on-orbit performance. For each case study, we analyzed contract files and conducted interviews with program and contracting officials at DOD, NASA, and NOAA to discuss contract types and incentive structures, rationale for the use of specific contract types and incentive structures, on-orbit performance and programmatic outcomes. Across the 12 case study programs, there were 23 contracts and orders and 56 space vehicles (see table 6).

To determine the at-risk on-orbit incentive amount by contract or order, we reviewed contract clauses and fee plans for each of the 23 contracts and orders associated with the 12 case study programs to calculate the amount of money the contractor would not be paid or would need to pay back in the case of a catastrophic failure. We calculated the at-risk amount based on a hypothetical “worst case scenario” for each contract or order, in which the maximum payback or forgone payment would be assessed for a complete failure of the satellites. The at-risk amounts included milestone payments that would be withheld, award and incentives fees that would not be paid, and amounts that would need to be paid back by the contractor. These at-risk amounts were divided by the current value of the contract or order to calculate a percentage of contract or order value that would be at risk. The contract scope varied across our case study contracts. For example, some contracts included costs associated with developing ground systems or sustainment costs whereas others only included satellite development and production costs. However, the contract scope did not significantly alter the at-risk calculations. For all of the contracts and orders we reviewed, regardless of scope, the at-risk amounts were less than 10 percent of the contract value.

For the seven remaining satellite programs at DOD and NASA that were not included in our case studies, we developed agency-specific data collection instruments (DCI) that we pre-tested, administered, and from which we analyzed the information collected. We tailored the DCIs to each agency based on agency policies and guidelines for contracting to obtain program-specific contract details, values, obligations to-date, special clauses used, incentive structures, and performance periods. We determined that the data collected from the DCIs were sufficiently reliable for the purposes of this engagement. The seven programs for which we executed DCIs included:

Advanced Extremely High Frequency (AEHF);

Global Precipitation Measurement (GPM) Mission,

Gravity Recovery and Climate Experiment Follow-On (GRACE-FO),

Landsat Data Continuity Mission (LDCM),

Magnetospheric Multiscale (MMS),

Orbiting Carbon Observatory 2 (OCO-2), and

Surface Water and Ocean Topography (SWOT).

In addition to the case studies above, we reviewed the Defense Meteorological Satellite Program 19 (DMSP-19) because of its recent on- orbit failure. We reviewed DMSP-19’s follow-on storage, maintenance, and support contracts rather than the development and production contracts because at the time of the failure, work was being performed under the follow-on contract. These contracts contained information on the on-orbit incentives and relevant options for government recourse after the on-orbit failure occurred.

We also interviewed officials from a nongeneralizable sample of commercial companies: Ball Aerospace, DigitalGlobe, IntelSat, Lockheed Martin, and ViaSat to identify how selected commercial companies use on-orbit incentives and how commercial satellite acquisitions differ from government satellite acquisitions. We selected the companies based on the type of satellites they build or acquire. In addition to commercial satellite companies, we also consulted satellite insurance brokers from Marsh and one underwriter, XL Catlin, to obtain information on the likelihood of satellite failures in the commercial and government markets, the general capacity of the commercial satellite insurance market, and the dollar amounts associated with insuring commercial satellites. We reviewed Aerospace Corporation studies and briefings and interviewed Aerospace Corporation officials to identify the point during the life of a satellite in which most satellite failures occur, the frequency of government satellite failures, and to identify the various categories of satellite failures.

We conducted this performance audit from March 2016 to June 2017 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix III: Contract Type and On-Orbit Incentives At-Risk for GAO Satellite Program Case Studies

The 12 in-depth case study satellite programs we reviewed included 23 contracts and orders. For these contracts and orders, we determined the predominant contract type for the design, development, and production of the satellites; and percent of contract value at-risk for on-orbit performance; which are presented in the tables below. We determined the percent of contract value at-risk for on-orbit performance for each contract or order by dividing the maximum dollar amounts the contractor would not be paid or would have to pay back in the event of a catastrophic failure by the current contract value.

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, key contributors to this report were Rich Horiuchi, Assistant Director; Emily Bond; Brandon Booth; Claire Buck; Claire Li; Michael Shaughnessy; Roxanna Sun; Jay Tallon; and Alyssa Weir.