NASA Commercial Crew Program: Continued Delays Pose Risks for Uninterrupted Access to the International Space Station
Released: 2018-01-17T00:00:00

Letter

Background

NASA’s Commercial Crew Program is a multi-phased effort that began in 2010. Across the five phases, NASA has engaged several companies using both agreements and contract vehicles to develop and demonstrate crew transportation capabilities. As the program has passed through these phases, NASA has generally narrowed down the number of participants. The early phases of the program were under Space Act agreements, which is NASA’s other transaction authority. These types of agreements are generally not subject to the Federal Acquisition Regulation (FAR) and allow the government and its contractors greater flexibility in many areas. Under these Space Act agreements, NASA relied on the commercial companies to propose specifics related to their crew transportation systems, including their design, the capabilities they would provide, and the level of private investment. In these phases, NASA provided technical support and determined if the contractors met certain technical milestones. In most cases, NASA also provided funding.

For the final two phases of the program, NASA awarded FAR-based contracts. By using FAR-based contracts, NASA gained the ability to levy specific requirements on the contractors and procure missions to the ISS, while continuing to provide technical expertise and funding to the contractors. Under these contracts, NASA will also evaluate whether contractors have met its requirements and certify their final systems for use.

In September 2014, NASA awarded firm-fixed-price contracts to Boeing and SpaceX, valued at up to $4.2 billion and $2.6 billion, respectively, for the Commercial Crew Transportation Capability phase. Under a firm- fixed-price contract, the contractor must perform a specified amount of work for the price negotiated by the contractor and government. This is in contrast to a cost-reimbursement contract, in which the government agrees to pay the contractor’s reasonable costs regardless of whether work is completed. Thus, under the fixed-price contracts, the contractors must generally bear the risk of cost overruns or schedule delays.

During this phase, the contractors will complete development of crew transportation systems that meet NASA requirements, provide NASA with the evidence it needs to certify that those systems meet its requirements, and fly initial crewed missions to the ISS. Under the contracts, NASA and the companies originally planned to complete the certification review for each system by 2017. Figure 1 shows the spacecraft and launch vehicles for Boeing and SpaceX’s crew transportation systems.

The Commercial Crew Transportation Capability phase contracts include three types of services:

Contract Line Item 001 encompasses the firm-fixed-price design, development, test, and evaluation work needed to support NASA’s final certification of the contractor’s spacecraft, launch vehicle, and ground support systems.

Contract Line Item 002 covers any service missions that NASA orders to transport astronauts to and from the ISS. Under this indefinite-delivery, indefinite-quantity line item, NASA has ordered six missions from each contractor. Each service mission is its own firm- fixed-price task order. NASA must certify the contractors’ systems before they can fly these missions.

Contract Line Item 003 is an indefinite-delivery, indefinite-quantity line item for any special studies, tests, and analyses that NASA may request. These tasks do not include any work necessary to accomplish the requirements under contract line item 001 and 002. As of July 2017, NASA had issued four orders under this contract line item to Boeing, worth approximately $1.8 million, including an approximately $180,000 study of the spacecraft’s seat incline. NASA has issued one order under this contract line item to SpaceX, which did not affect the value of this line item. The maximum value of this contract line item is $150 million.

NASA divided the certification work under contract line item 001 into two acceptance events: the design certification review and the certification review. An acceptance event occurs when NASA approves a contractor’s designs and acknowledges that the contractor’s work is complete and meets the requirements of the contract. The design certification review verifies the contractor’s crew transportation system’s capability to safely approach, dock, mate, and depart from the ISS, among other requirements. After the contractor has successfully completed all of its flight tests, as well as various other activities, the certification review determines whether the crew transportation system meets the Commercial Crew Program’s requirements. The contractors must complete both acceptance events to receive NASA certification.

NASA and the contractors also identified discrete performance-based events, called interim milestones, which occur as the contractors progress toward the two acceptance events. Each interim milestone has pre- determined entrance and exit criteria that establish the work that must be completed in order for the contractor to receive payment. The interim milestones serve several functions, allowing the government to finance work from development to completion, review the contractors’ progress, and provide approval to proceed with key demonstrations and tests. The program also uses these milestones to inform its annual budget request. Since the contracts were awarded, the Commercial Crew Program and the contractors have agreed to split several of the interim milestones. The contractors have also added new milestones, in part to capture changes in their development plans.

NASA has also made changes to the contracts that have increased their value. While the contracts are fixed-price, their values can increase if NASA adds to the scope of the work or otherwise changes requirements. As of July 2017, NASA had increased the value of contract line item 001 for Boeing by approximately $48 million for hardware and software requirement changes, and contract line item 001 for SpaceX by approximately $91 million for a hardware requirement change and the addition of cargo during an ISS test flight.

In our February 2017 report, we found the following:

Both of the Commercial Crew Program’s contractors have made progress developing their crew transportation systems, but both also have aggressive development schedules that are increasingly under pressure. Both Boeing and SpaceX had determined that they would not be able to meet their original 2017 certification dates, and both expected certification to be delayed until 2018. We found that the schedule pressures were amplified by NASA’s need to provide a viable crew transportation option to the ISS before its current contract with Russia’s space agency runs out in 2019. If NASA needs to purchase additional seats from Russia, the contracting process typically takes 3 years. Without a viable contingency option for ensuring uninterrupted access to the ISS in the event of further Commercial Crew delays, we found that NASA was at risk of not being able to maximize the return on its multibillion dollar investment in the space station.

The Commercial Crew Program was using mechanisms laid out in its contracts to gain a high level of visibility into the contractors’ crew transportation systems, but maintaining the current level of visibility through certification could add schedule pressures. For example, due to NASA’s acquisition strategy for this program, its personnel are less involved in the testing, launching, and operation of the crew transportation system. And while the program has developed productive working relationships with both contractors, the level of visibility that the program had required thus far had also taken more time than the program or contractors anticipated. Ultimately, the program has the responsibility for ensuring the safety of U.S. astronauts, and its contracts give it deference to determine the level of visibility required to do so. Moving forward though, we found that the program office could face difficult choices about how to maintain the level of visibility it feels it needs without adding to the program’s schedule pressures.

In order to ensure that the United States had continued access to the ISS if the Commercial Crew Program’s contractors experienced additional schedule delays, we recommended that the NASA Administrator develop a contingency plan for maintaining a presence on the ISS beyond 2018, including options to purchase additional Russian Soyuz seats, and report to Congress on the results. NASA concurred with this recommendation, and in February 2017, NASA executed a contract modification to procure an option for three crewmember seats from Boeing on the Russian Soyuz vehicle. Our analysis found that these seats represented a contingency plan for U.S. access to the ISS through 2019. In April 2017, NASA informed the Congress of this action.

Both Contractors Have Made Progress but Continue to Experience Schedule Delays

Contractors Continue to Advance Development of Their Crew Transportation Systems

Both Boeing and SpaceX have continued to make progress finalizing their designs and building hardware as they work toward final certification of their crew transportation systems, since we last reported in February 2017. Each contractor’s system includes a spacecraft and a launch vehicle with supporting ground systems. The contractors are also manufacturing test articles and flight spacecraft to support the uncrewed and crewed flight tests. The contractors plan to use the test articles to demonstrate system performance and the flight spacecraft to demonstrate their ability to meet contract requirements.

As table 1 shows, these test articles and flight spacecraft are currently in varying stages of completion—some are completed and in testing while others are still early in the manufacturing phase. Should any issues arise during integration and test or the flight tests planned for 2018, the contractors may have to complete rework on the spacecraft already under construction.

Schedule Delays Continue, and Risks Remain to Final Certification Dates

The contractors have notified NASA that final certification dates have slipped to the first quarter of calendar year 2019 and, through our ongoing work, we have identified three key risk areas that could further delay certification of each contractor’s crew transportation system. These areas are (1) the contractors’ aggressive schedules, (2) programmatic and safety risks, and (3) Commercial Crew Program’s workload. These are consistent with the challenges we found facing the contractors and program in our February 2017 report.

Aggressive schedules. Since the award of the current Commercial Crew contracts in September 2014, the program, Boeing, and SpaceX have all identified the contractors’ delivery schedules as aggressive. Program officials told us that, from the outset, they knew delays were likely due to the developmental nature of the program. Multiple independent review bodies—including the program’s standing review board, the Aerospace Safety Advisory Panel, and the NASA Advisory Council-Human Exploration and Operations committee—also noted the aggressiveness of the contractors’ schedules as they move toward certification.

In February 2017, we found that both contractors had notified NASA that they would not be able to meet the 2017 final certification dates originally established in their contracts and expected final certification to be delayed until 2018. Based on our ongoing work, we found that the contractors have notified NASA that these dates have slipped further to the first quarter of calendar year 2019. Figure 2 shows the original Boeing and SpaceX contract schedule and the current proposed schedule for each contractor.

However, the extent to which these schedules represent an accurate estimate of each contractor’s final certification date is unclear for the following two reasons: 1. Each contractor provides schedule updates to the Commercial Crew Program at quarterly status reviews, and the dates frequently change. The program has held 12 quarterly reviews since each contract was awarded. Boeing has reported a delay six times and SpaceX has reported a delay nine times that included at least one key event identified in the timeline above at these quarterly reviews. 2. The Commercial Crew Program is tracking risks that both contractors could experience additional schedule delays and, based on our ongoing work, we found that the program’s own analysis indicates that certification is likely to slip into December 2019 for SpaceX and February 2020 for Boeing. Each month, the program updates its schedule risk analysis, based on the contractors’ internal schedules as well as the program’s perspectives and insight into specific technical risks. The Commercial Crew Program manager stated that differences between the contractors’ proposed schedules and the program’s schedule risk analysis include the following:

The contractors are aggressive and use their schedule dates to motivate their teams, while NASA adds additional schedule margin for testing.

Both contractors assume an efficiency factor in getting to the crewed flight test that NASA does not factor into its analysis.

The program manager explained further that the program meets with each contractor monthly to discuss schedules and everyone agrees to the relationships between events in the schedule even if they disagree on the length of time required to complete events. The program manager added, however, that she relies on her prior experience for a better sense of schedule timeframes as opposed to relying on the contractors’ schedules.

While NASA has a fixed-price contract with both SpaceX and Boeing, there are consequences to the delays to date and the lack of certainty surrounding the final certification date. The United States has spent tens of billions of dollars to develop, assemble, and operate the ISS over the past two decades, and NASA relies on uninterrupted crew access to help maintain and operate the station itself and conduct the research required to enable human exploration in deep space and eventually Mars, among other science and research goals. To ensure uninterrupted access to the ISS through 2019, which includes launch and return of the astronauts, NASA purchased five seats on the Soyuz spacecraft through Boeing for an undisclosed value. Boeing obtained these seats though a legal settlement with the Russian firm, RSC Energia, which manufactures the Soyuz. The NASA Office of Inspector General found in its annual report on NASA’s top management and performance challenges that if the Commercial Crew Program experiences additional delays, NASA may need to buy additional seats from Russia to ensure a continued U.S. presence on the ISS. Further, the ISS is planned to be operational through 2024. Unless there is a decision to extend the ISS’s operational life, additional delays by Boeing and SpaceX may lessen NASA’s return on investment with the contractors. We will continue to monitor this as part of our ongoing work.

Programmatic and safety risks. In addition to challenges facing Boeing and SpaceX’s aggressive schedules, both contractors face other risks that will need to be addressed to support their certification. This includes the contractors’ ability to meet the agency’s requirements related to the safety of their systems. These risks are not unusual; there are inherent technical, design, and integration risks in all NASA’s major acquisitions, as these projects are highly complex and specialized and often push the state of the art in space technology. The Commercial Crew Program monitors risks through two lenses—programmatic risks potentially affect the program’s cost and schedule or the performance of the crew transportation system, and safety risks could elevate the potential for the loss of crew.

SpaceX Risks

Similar to our findings in February 2017, our ongoing work indicates that the Commercial Crew Program’s top programmatic and safety risks for SpaceX, are in part, related to ongoing launch vehicle design and development efforts.

SpaceX must close several of the program’s top risks related to its upgraded launch vehicle design, the Falcon 9 Block 5, before it can be certified for human spaceflight. Included in this Block 5 design is SpaceX’s redesign of the composite overwrap pressure vessel. SpaceX officials stated the new design aims to eliminate risks identified in the older design, which was involved in an anomaly that caused a mishap in September 2016. Separately, SpaceX officials told us that the Block 5 design also includes design changes to address cracks in the turbine of its engine identified during development testing.

NASA program officials told us that they had informed SpaceX that the cracks were an unacceptable risk for human spaceflight. SpaceX officials told us that they have made design changes, captured in this Block 5 upgrade, that did not result in any cracking during initial life testing. However, this risk will not be closed until SpaceX successfully completes qualification testing in accordance with NASA’s standards without any cracks. SpaceX officials stated they expect this testing to be completed in first quarter calendar year 2018.

Finally, both the program and a NASA advisory group consider SpaceX’s plan to fuel the launch vehicle after the astronauts are on board the spacecraft to be a potential safety risk. SpaceX’s perspective is that this operation may be a lower risk to the crew. To better understand the propellant loading procedures, the program and SpaceX agreed to demonstrate the loading process five times from the launch site in the final crew configuration prior to the crewed flight test.

Boeing Risks

Our ongoing work indicates that Boeing is mitigating several risks in order to certify its crew transportation system, including challenges related to its abort system performance, parachutes, and its launch vehicle.

Boeing is addressing a risk that its abort system, which it needs for human spaceflight certification, may not meet the program’s requirement to have sufficient control of the vehicle through an abort. In some abort scenarios, Boeing has found that the spacecraft may tumble and that could pose a threat to the crew’s safety. To validate the effectiveness of its abort system, Boeing has conducted extensive wind tunnel testing and plans to complete a pad abort test in April 2018.

Boeing is also addressing a risk that during re-entry to the Earth’s atmosphere, a portion of the spacecraft’s forward heat shield may reconnect and damage the parachute system. NASA’s independent analysis indicates that this may occur if both parachutes that pull the forward heat shield away from the spacecraft deploy as expected. Boeing’s analysis indicates the risk exists only if one of two parachutes does not deploy as expected. If the program determines this risk is unacceptable, Boeing would need to redesign the parachute system, which the program estimates could result in at least a 6-month delay. Finally, one of the program’s top programmatic and safety concerns is that it may not have enough information from Boeing’s launch vehicle provider, United Launch Alliance, to assess if the launch vehicle prevents or controls cracking that could lead to catastrophic failures. The program and Boeing are in the process of negotiating next steps.

Program Safety Risk

The Commercial Crew Program has identified the ability of it and its contractors to meet a crew safety requirement as one of its top risks. NASA established the “loss of crew” metric as a way to measure the safety of a crew transportation system. The metric captures the probability of death or permanent disability to one or more crew members. Under each contract, the current loss of crew requirement is 1 in 270, meaning that the contractors’ systems must carry no more than a 1 in 270 probability of incurring loss of crew. Near the end of the Space Shuttle program, the probability of loss of crew was approximately 1 in 90. As part of our ongoing work, we continue to work with NASA to understand how the loss of crew requirement was established for the Commercial Crew Program.

Program officials told us that Commercial Crew is the first NASA program that the agency will evaluate against a probabilistic loss of crew requirement. They said that if the contractors cannot meet the loss of crew requirement at 1 in 270, NASA could still certify their systems by employing operational mitigations. They said this would entail a potentially increased level of risk or uncertainty related to the level of risk for the crew.

Program officials told us their main focus is to work with the contractors to ensure that the spacecraft designs are robust from a safety perspective. The loss of crew metric and the associated models used to measure it are tools that help achieve that goal. For example, Boeing told us that in early 2016, it needed to identify ways to reduce the mass of its spacecraft. As Boeing found opportunities to reduce the spacecraft mass, the program stated that it had to consider how implementing those design changes would affect its loss of crew analysis in addition to compliance with other performance and safety requirements. According to the program, it is working with both contractors to address the factors that drive loss of crew risk through design changes or additional testing to gain more information on the performance and reliability of systems. As part of our ongoing work, we will continue to assess the extent to which the contractors are meeting this requirement and what tools the program and NASA will use to determine if the contractors meet the requirement.

Program office workload. In February 2017, we found that the Commercial Crew Program was using contractually defined mechanisms to gain a high level of visibility into the contractors’ crew transportation systems, but also found that the Commercial Crew Program’s workload was an emerging schedule risk. At that time, program officials told us that one of their greatest upcoming challenges will be to keep pace with the contractors’ schedules so that the program does not delay certification. Specifically, they told us they are concerned about an upcoming “bow wave” of work because the program must complete two oversight activities—phased safety reviews and verification closure notices—concurrently in order to support the contractors’ design certification reviews, uncrewed and crewed flight test missions, and final certification.

The Commercial Crew Program is working to complete its three-phased safety review, which will ensure that the contractors have identified all safety-critical hazards and implemented associated controls, but it is behind schedule. Both the contractors and the program have contributed to these delays.

In phase one, Boeing and SpaceX identified risks in their designs and developed reports on potential hazards, the controls they put in place to mitigate them, and explanations for how the controls will mitigate the hazards.

In phase two, which is ongoing, the program reviews and approves the contractors’ hazard reports, and develops strategies to verify and validate that the controls are effective.

In phase three, the contractors plan to conduct the verification activities and incrementally close the reports.

The Commercial Crew Program’s review and approval of the contractors’ hazard reports have taken longer than planned. The program originally planned to complete phase two in early 2016, but through our ongoing work, we have found that as of October 2017, neither contractor had completed this phase. At that time, Boeing had completed 90 percent and SpaceX had completed 70 percent of the Phase 2 reports.

The Commercial Crew Program’s verification closure notice process, which is used to verify that the contractors have met all requirements, is one of the other key oversight activities and potential workload challenges for the program. The program is completing that process concurrently with the phased safety reviews. The verification closure process is initiated by the contractor when it provides the program with data and evidence to substantiate that it has met each requirement, and is completed when the program has reviewed and approved the contractor’s evidence to verify that each requirement has been met. The Commercial Crew Program must also approve a subset of verification closure notices before key tests or milestones can occur. For example, the ISS requirements and a portion of the Commercial Crew Program requirements must be met before Boeing and SpaceX’s uncrewed flights to the ISS, which are currently planned for the third quarter of 2018. The program’s ability to smooth its workload is limited because the contractors generally control their development schedules. In February 2017, we found, however, that proposed changes to the Boeing and SpaceX schedules could help alleviate some of the concurrency between the program’s phased safety reviews and verification closure process. We will continue to monitor the efforts as part of our ongoing work.

In conclusion, Boeing and SpaceX continue to make progress developing crew transportation systems to help the United States re-establish its domestic ability to provide crew access to the ISS. But, when the current phase of the Commercial Crew Program began, there was widespread acknowledgment that the contractors’ development and certification schedules were aggressive and the anticipated schedule risks have now materialized. Further, programmatic and safety risks remain with schedules that frequently change making a final certification date uncertain. Delays and uncertain final certification dates raise questions about whether the United States will have uninterrupted access to the International Space Station beyond 2019, and may lessen NASA’s return on investment with the contractors. We look forward to continuing to work with NASA and this subcommittee as we assess the contractors’ and program’s progress to final certification.

Chairman Babin, Ranking Member Bera, and Members of the Subcommittee, this completes my prepared statement. I would be pleased to respond to any questions that you may have at this time.

GAO Contact and Staff Acknowledgments

If you or your staff have any questions about this testimony, please contact Cristina T. Chaplain, Director, Acquisition and Sourcing Management at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. GAO staff who made key contributions to this statement include Molly Traci, Assistant Director; Susan Ditto; Lisa Fisher; Laura Greifner; Juli Steinhouse; Roxanna Sun; and Kristin Van Wychen.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.