NASA Commercial Crew Program: Schedule Pressure Increases as Contractors Delay Key Events
Published: 2017-02-16T00:00:00

Fast Facts:
NASA contracted with two companies, Boeing and SpaceX, to develop and produce vehicles to transport astronauts to the International Space Station. But first, NASA has to certify that the vehicles meet its requirements and are safe for human spaceflight.
Both companies have experienced delays and do not expect to meet their original 2017 certification dates. If these delays continue, certification could be pushed into 2019—which is when the U.S.'s contract with Russia for ISS transportation ends.
We recommended that NASA come up with a backup plan for ISS transportation in case certification is delayed, and report on that plan to Congress.

Highlights:
section_title: Why GAO Did This Study
paragraphs: Since the Space Shuttle was retired in 2011, the United States has been relying on Russia to transport astronauts to and from the ISS. The purpose of NASA's Commercial Crew Program is to facilitate the development of a domestic transport capability. In 2014, NASA awarded two firm-fixed-price contracts to Boeing and SpaceX with a combined total value up to $6.8 billion for the development of crew transportation systems that meet NASA requirements and initial missions to the ISS. The contractors were originally required to provide NASA all the evidence it needed to certify that their systems met its requirements by 2017.
A house report accompanying H.R. 2578 included a provision for GAO to review the progress of NASA's human exploration programs. This report examines the Commercial Crew Program including (1) the extent to which the contractors have made progress towards certification, (2) the risks facing the program, and (3) the extent to which the program has visibility into the contractors' efforts. To do this work, GAO analyzed contracts, schedules, and other documentation; and spoke with officials from NASA, the Commercial Crew Program, Boeing, SpaceX, and independent review bodies.
section_title: What GAO Found
paragraphs: Both of the Commercial Crew Program's contractors have made progress developing their crew transportation systems, but both also have aggressive development schedules that are increasingly under pressure. The two contractors—Boeing and Space Exploration Technologies, Corp. (SpaceX)—are developing transportation systems that must meet the National Aeronautics and Space Administration's (NASA) standards for human spaceflight—a process called certification. Both Boeing and SpaceX have determined that they will not be able to meet their original 2017 certification dates and both expect certification to be delayed until 2018, as shown in the figure below. The schedule pressures are amplified by NASA's need to provide a viable crew transportation option to the International Space Station (ISS) before its current contract with Russia's space agency runs out in 2019. If NASA needs to purchase additional seats from Russia, the contracting process typically takes 3 years. Without a viable contingency option for ensuring uninterrupted access to the ISS in the event of further Commercial Crew delays, NASA risks not being able to maximize the return on its multibillion dollar investment in the space station.
Both contractors are also dealing with a variety of risks that could further delay certification, including program concerns about the adequacy of information on certain key systems to support certification. Another top program risk is the ability of NASA and its contractors to meet crew safety requirements.
The Commercial Crew Program is using mechanisms laid out in its contracts to gain a high level of visibility into the contractors' crew transportation systems. The program is using a different model than every other spacecraft NASA has built for humans. For example, NASA personnel are less involved in the testing, launching, and operation of the crew transportation system. The program has developed productive working relationships with both contractors, but the level of visibility that the program has required thus far has also taken more time than the program or contractors anticipated. Ultimately, the program has the responsibility for ensuring the safety of U.S. astronauts and its contracts give it deference to determine the level of visibility required to do so. Moving forward though, the program office could face difficult choices about how to maintain the level of visibility it feels it needs without adding to the program's schedule pressures.
section_title: What GAO Recommends
paragraphs: Given the delays in the Commercial Crew Program, GAO recommends that NASA develop and report to Congress on its contingency plans for maintaining a U.S. presence on the ISS beyond 2018. NASA concurred with the recommendation and intends to develop a contingency plan.

section_title: Letter
paragraphs: Following the retirement of the Space Shuttle in 2011, the United States  was left with no domestic ability to provide crew access to the  International Space Station (ISS). Since then, the National Aeronautics  and Space Administration (NASA) has relied on purchasing seats from  Russia on its Soyuz spacecraft to maintain a U.S. presence on the  station. NASA’s Commercial Crew Program is intended to end this  dependency by facilitating the commercial development of a crew  transportation system that can provide safe, reliable, and cost-effective  transportation to and from the ISS no later than 2017. NASA’s goal is to  have one or more contractors that can provide crew transportation  services to the ISS, which is planned to be operational until at least 2024.
NASA’s acquisition strategy on the Commercial Crew Program is similar  to the one it used successfully on the Commercial Cargo program, but  different than every other spacecraft it has built for humans, from Mercury  to Gemini and Apollo to the Space Shuttle. Each contractor designs,  develops, builds, owns, and operates its spaceflight system and  infrastructure. The contractors have access to NASA’s expertise and  resources throughout the development process, but NASA engineers are  not making design decisions and NASA personnel will be less involved in  processing, testing, launching, and operating the crew transportation  system. In the end, NASA will buy a crew transportation service—a ride  for its astronauts to and from the ISS—much like it does for ISS cargo.
In the most recent phase of the Commercial Crew Program, NASA  awarded firm-fixed-price contracts in 2014 to Boeing and Space  Exploration Technologies Corporation (SpaceX), valued at up to $4.2  billion and $2.6 billion, respectively, for the development of crew  transportation systems that meet NASA requirements and initial missions  to the ISS. According to the contracts, the companies are supposed to  provide NASA all the evidence it needs to certify that their systems meet  its performance and safety requirements by 2017.
The House Committee on Appropriations report accompanying H.R. 2578  includes a provision for us to review the acquisition progress of NASA’s  human exploration programs, including the Commercial Crew Program.  For this review, we assessed (1) the extent to which the contractors have  made progress towards certification and the potential effects of any  certification delays on NASA’s access to the ISS; (2) the major  programmatic and safety risks facing the program and the contractors;  and (3) the extent to which the program has visibility into the contractors’  efforts. To assess the contractors’ progress toward certification, we  reviewed program and contract documents, including quarterly progress  updates and monthly schedule summaries, as well as interviewed  program and contractor officials. We also reviewed information from  NASA’s budget, its contract with the Russian Federal Space Agency, and  the ISS program to determine how NASA plans to mitigate the effects of  any certification delays on its access to the ISS. To assess major  programmatic and safety risks, we analyzed the program’s monthly risk  charts and contractor documentation. We also interviewed program and  contractor officials to identify steps being taken to mitigate risks. To  assess the program’s visibility into the contractors’ efforts, we reviewed  program and contract documents, interviewed officials to identify  mechanisms NASA uses to gain visibility into the contractors’ efforts, and  observed how these mechanisms functioned in practice for several key  contract events. For these events, we interviewed program and contractor  officials to learn how NASA uses various mechanisms to gain visibility,  analyzed supporting documents, and compared NASA’s visibility to what  is defined in the contract. For all three objectives, we met with  independent NASA expert groups to corroborate information we received  from the program and its contractors. Appendix I contains detailed  information about our scope and methodology.
We conducted this performance audit from March 2016 to February 2017  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: NASA’s Commercial Crew Program is a multi-phased effort that began in  2010. Across the five phases, NASA has engaged several companies  using both agreements and contract vehicles to develop and demonstrate  crew transportation capabilities. As the program has passed through  these phases, NASA has generally narrowed down the number of  participants. The early phases of the program were under Space Act  agreements, which is NASA’s other transaction authority. These types of  agreements are generally not subject to the Federal Acquisition  Regulation (FAR) and allow the government and its contractors greater  flexibility in many areas. Under these Space Act agreements, NASA  relied on the commercial companies to propose specifics related to their  crew transportation systems, including their design, the capabilities they  would provide, and the level of private investment. In these phases,  NASA provided technical support and determined if the contractors met  certain technical milestones. In most cases, NASA also provided funding.  For the final two phases of the program, NASA awarded FAR-based  contracts. By using FAR-based contracts, NASA gained the ability to levy  specific requirements on the contractors and procure missions to the ISS,  while continuing to provide technical expertise and funding to the  contractors. Under the contracts, NASA will also evaluate whether  contractors have met its requirements and certify their final systems for  use. Appendix II contains a description of each of the Commercial Crew  Program’s phases, a list of the participants, and the level of funding  provided to each participant by NASA.
subsections: section_title: Program Requirements
paragraphs: Before a company’s crew transportation system can be certified by  NASA, it must meet two sets of requirements. The ISS program levies a  set of 332 requirements that must be met by all visiting spacecraft,  whether they are carrying cargo or crew to the station. There are three  major areas outlined in the ISS requirements document: 1) interface  requirements for both the ISS and the spacecraft; 2) performance  requirements for ground systems supporting the spacecraft; and 3)  design requirements for spacecraft to ensure safe integration with the  ISS. The second set of requirements is levied by the Commercial Crew  Program. These include 280 high-level requirements related to the  design, manufacturing, testing, qualification, production, and operation of  crew transportation systems that deliver NASA astronauts to the ISS.  Contractors are responsible for developing the lower-level system  specifications and system and subsystem designs that implement those  requirements. For example, there is a Commercial Crew Program  requirement for the contractor’s crew transportation system to provide a  continuous autonomous launch abort capability, in which the spacecraft  self-initiates the abort procedure independent from the supporting ground  systems, from lift-off through orbital insertion with a 95 percent probability  of success. The requirement does not specify how elements of the crew  transportation system, such as its sensors, software, propulsion, or  capsule, should be designed to fulfill it.
section_title: Current Program Contracts
paragraphs: In September 2014, NASA awarded firm-fixed-price contracts to Boeing  and SpaceX, valued at up to $4.2 billion and $2.6 billion, respectively, for  the Commercial Crew Transportation Capability phase. Under a firm- fixed-price contract, the contractor must perform a specified amount of  work for the price negotiated by the contractor and government. This is in  contrast to a cost-reimbursement contract, in which the government  agrees to pay the contractor’s reasonable costs regardless of whether  work is completed. During this phase, the contractors will complete  development of crew transportation systems that meet NASA  requirements, provide NASA with the evidence it needs to certify that  those systems meet its requirements, and fly initial crewed missions to  the ISS. Under the contracts, NASA and the companies originally planned  to complete the certification review for each system by 2017. Figure 1  shows the spacecraft and launch vehicles for Boeing and SpaceX’s crew  transportation systems.
The Commercial Crew Transportation Capability phase contracts include  three types of services:
Contract Line Item 001 encompasses the firm-fixed-price design,  development, test, and evaluation work needed to support NASA’s  final certification of the contractor’s spacecraft, launch vehicle, and  ground support systems.
Contract Line Item 002 covers any service missions that NASA  orders to transport astronauts to and from the ISS. Under this  indefinite-delivery, indefinite-quantity line item, NASA must order at  least two missions from each contractor, which it has done, and can  order up to six missions. Each service mission is its own firm-fixed- price task order. NASA must certify the contractors’ systems before  they can fly these missions.
Contract Line Item 003 is an indefinite-delivery, indefinite-quantity  line item for any special studies, tests, and analyses that NASA may  request. These tasks do not include any work necessary to  accomplish the requirements under contract line item 001 and 002. As  of September 2016, NASA had issued one order under this contract  line item to Boeing—an approximately $180,000 study of the  spacecraft’s seat incline. The maximum value of this contract line item  is $150 million.
NASA divided the certification work under contract line item 001 into two  acceptance events: the ISS design certification review and the  certification review. An acceptance event occurs when NASA approves a  contractor’s designs and acknowledges that the contractor’s work is  complete and meets the requirements of the contract. The ISS design  certification review verifies the contractor’s crew transportation system’s  capability to safely approach, dock, mate, and depart from the ISS,  among other requirements. After the contractor has successfully  completed all its flight tests, as well as various other activities, the  certification review determines whether the crew transportation system  meets the Commercial Crew Program’s requirements. The contractors  must complete both acceptance events to receive NASA certification.
NASA and the contractors also identified discrete performance-based  events, called interim milestones, which occur as the contractors progress  towards the two acceptance events. Each interim milestone has pre- determined entrance and exit criteria that establish the work that must be  completed in order for the contractor to receive payment. The interim  milestones serve several functions, allowing the government to finance  work from development to completion, review the contractors’ progress,  and provide approval to proceed with key demonstrations and tests.  These milestones are also used by the program to inform its annual  budget request. Since the contracts were awarded, the Commercial Crew  Program and the contractors have agreed to split several of the interim  milestones. The contractors have also added new milestones, in part to  capture changes in their development plans.
NASA has also made changes to the contracts that have increased their  value. While the contracts are fixed-price, their values can increase if  NASA adds to the scope of the work or otherwise changes requirements.  As of October 2016, NASA had increased the value of contract line item  001 for Boeing by $47 million for hardware and software requirement  changes, and contract line item 001 for SpaceX by $91 million for a  hardware requirement change and the addition of cargo during an ISS  test flight.
section_title: Program Oversight
paragraphs: NASA has tailored its management approach for the Commercial Crew  Program because the contractors, not the government, are responsible  for the design, development, test, and evaluation of their crew  transportation systems and make their own decisions about when to  build, integrate, and test hardware. For example, NASA policy outlines  key decision point reviews, which allow NASA management to determine  if the program is ready to progress to the next phase. NASA  management relies on annual program reviews, instead of key decision  point reviews, to provide oversight of the Commercial Crew Program.  According to NASA management officials, they chose to do so because  the contractors’ development approaches and schedules did not align  with NASA’s typical key decision points. Other elements of the reviews  are similar though. For example, NASA’s first annual review of the  program included updates on overall program risk, an update on the  program’s cost and schedule risks, and perspectives from the  independent review board, much like a key decision point review. The  firm-fixed price nature of the current contracts also led NASA to alter its  typical management approach. For example, the NASA Associate  Administrator told us that he decided not to require an agency baseline  commitment—cost or schedule baseline—for the Commercial Crew  Program, in part, because the firm-fixed-price contracts with each  contractor essentially serve as the cost baselines for the program.
NASA also provides regular updates to Congress on the Commercial  Crew Program through a variety of reporting mechanisms. NASA includes  information on all of its major projects, including Commercial Crew, in its  annual budget submission, although the type and specificity of the  information can differ. For example, table 1 shows the key differences in  the types of information NASA provided on two human space flight  projects—the Commercial Crew Program and the Orion Crew Vehicle  development in its fiscal year 2017 budget request. In addition to its  budget, NASA submits quarterly reports to the appropriations committees  on the Commercial Crew Program’s progress that include cost and  schedule updates from each contractor and information on contractor  specific-risks, including procurement-sensitive material that is not  publically releasable.
section_title: Neither Contractor Expects to Achieve Certification before 2018 and NASA Has Not Yet Determined How It Will Ensure ISS Access in Case of Further Delays
paragraphs: Since September 2014, both Boeing and SpaceX have made progress  developing their crew transportation systems, but neither contractor will  be able to meet their original 2017 certification dates and both expect  certification to be delayed until 2018. As the contractors have fallen  behind, the time between key test events has decreased, which reduces  the time the contractors have to learn and implement changes, increasing  the likelihood of additional delays. The schedule pressures are amplified  by NASA’s need to provide a viable crew transportation option to the ISS  before its current contract with Russia’s space agency runs out in 2019.  NASA has not yet developed a contingency plan to ensure an  uninterrupted presence on the ISS should the Commercial Crew Program  experience further delays, although the ISS program has begun to  discuss potential options.
subsections: section_title: Both Contractors Have Begun Manufacturing Spacecraft and Modifying Launch Facilities
paragraphs: Both Boeing and SpaceX have made progress finalizing their designs and  building hardware as they work towards final certification of their crew  transportation systems. Each contractor’s system includes a spacecraft  and a launch vehicle with supporting ground systems. Examples of the  contractors’ development progress since September 2014 include the  following:
Boeing: In 2015, Boeing completed its critical design review, which  determines whether a system’s design is stable enough to proceed  with final design and fabrication, and began fabricating the first of its  three planned spacecraft. This spacecraft will be tested to validate the  design, including in a planned January 2018 pad abort test where the  ability of the capsule’s abort system is tested from a static position on  the launch pad. Further, in May 2016, Boeing shipped the structural  test article for its service module to its testing facility and shipped the  crew module in January 2017. These modules will be used to conduct  tests that simulate different operating environments, such as the  vibrations during a launch ascent, on key spacecraft components.  Boeing and its launch vehicle provider, United Launch Alliance (ULA),  have also made significant modifications to its launch facilities at the  Cape Canaveral Air Force Station, including installing a crew access  tower and crew access arm, which the crew will use to board the crew  capsule prior to launch.
SpaceX: SpaceX completed its critical design review in 2016 and has  begun assembling and integrating components of the two spacecraft it  will use for its uncrewed and crewed flight tests, which are scheduled  for November 2017 and May 2018, respectively. SpaceX officials told  us they completed integrated testing and qualification of several major  spacecraft components, including its environmental control and life  support system—which controls air quality and temperature, among  other functions, to ensure crew survivability—and its spacesuit in  November 2016. SpaceX also made significant modifications to its  launch facilities at the Kennedy Space Center, including constructing  a new hangar and an upgraded flame trench, which is critical to safely  contain the plume exhaust from rocket launches, and substantially  upgrading the existing crew access tower, which was last used during  the Space Shuttle era.
section_title: Both Contractors Have Delayed Certification and Reduced Time between Key Test Events
paragraphs: Neither Boeing nor SpaceX will be able to meet their original 2017  certification dates and both now expect certification to be delayed until  2018. Since the award of the current Commercial Crew contracts, the  program, Boeing, and SpaceX have all identified the contractors’ delivery  schedules as aggressive. Program officials told us that, from the outset,  they knew delays were likely due to the development nature of the  program. Multiple independent review bodies—including the program’s  standing review board, the Aerospace Safety Advisory Panel, and the  NASA Advisory Council-Human Exploration and Operations committee— also noted the aggressiveness of the contractors’ schedules as they  move toward certification.
Both contractors have notified NASA that they would not be able to meet  the 2017 certification dates originally established in their contracts. These  notices required the parties to renegotiate their contracts to reflect the  contractors’ delays. As of October 2016, both contractors have submitted,  and NASA has agreed to, updated schedules that reflect delays to their  final certification review dates—Boeing by 5 months, from August 2017  until January 2018, and SpaceX by 6 months, from April 2017 until  October 2017. Boeing and SpaceX’s internal schedules both show  additional certification delays that, as of November 2016, were not yet  reflected in their contracts. As the contractors have made changes to their  development schedules, they have also reduced the number of months  between critical test events leading up to final certification, which reduces  the time the contractors have to learn, make any needed design changes,  and implement those changes, increasing the likelihood of additional  delays. Figures 2 and 3 below show the total proposed certification delay  and current proposed schedule for each contractor.
Since Boeing and the Commercial Crew Program agreed to move  Boeing’s certification review from August 2017 to January 2018, Boeing  has proposed moving the review out by at least 9 additional months to the  fourth quarter of 2018. The current proposed schedule is the fourth  significant schedule change by Boeing since the contract was awarded.  The most recent schedule update includes delays related to a  manufacturing error. In September 2016, a subcontractor damaged a  major component of Boeing’s second spacecraft during machining to trim  mass from the spacecraft. This is expected to delay delivery of the  component by 2-3 months, which is reflected in the current proposed  schedule. The second spacecraft is important because, according to  Boeing’s program schedule, it will be used to support a significant portion  of Boeing’s planned testing as well as its crewed flight test.
Boeing also reduced the time between some key testing events that will  be used to provide data necessary for Boeing to demonstrate its ability to  meet NASA’s requirements. In the original schedule, Boeing allocated 3  months between its uncrewed flight test and its crewed flight test, but  under its current proposed schedule, there are only 2 months between  these critical test events. Although these events are not formal milestones  in Boeing’s contract with NASA, they are critical learning events, and we  have previously found that reducing the time between key test events  limits time for the contractor to learn and adapt to any changes that may  be required. Boeing is also tracking a risk that the schedule could be  further delayed because there is little time between test events.
Since SpaceX and the Commercial Crew Program agreed to move its  certification review from April 2017 to October 2017, SpaceX has  proposed moving the review out by at least 9 additional months to the  third quarter of 2018. SpaceX officials stated that these delays are largely  driven by development challenges, changes in NASA requirements, and  implementation of corrective actions stemming from a launch vehicle  mishap in 2016. On September 1, 2016, SpaceX experienced an anomaly  during a standard, pre-launch static fire test of its Falcon 9 launch vehicle  that resulted in an explosion and the loss of the vehicle. The mishap  investigation is complete and SpaceX returned to flight in January 2017  with a commercial launch. However, additional schedule changes are  possible. SpaceX officials told us that they continue to assess the effect  of the mishap on their Commercial Crew schedule. Further, as SpaceX  has made schedule changes, it has also reduced the time between its  uncrewed flight test and crewed flight test from 7 months to 6 months.
The Commercial Crew Program is tracking risks that both contractors  could experience additional schedule delays and its own analysis  indicates that certification is likely to slip into 2019. One of the program’s  top six programmatic risks for each contractor is the likelihood of  additional schedule delays. Each month, the program updates its  schedule risk analysis, based on the contractors’ internal schedules as  well as the program’s perspectives and insight into specific technical  risks. As of October 2016, the program’s schedule risk analysis indicated  that both contractors’ certification dates would likely slip into early 2019.
The mounting schedule pressure on the Commercial Crew Program from  the contractors’ delays is amplified by NASA’s need to provide a viable  crew transportation option to the ISS before its current contract with  Roscosmos, the Russian Federal Space Agency, runs out. The United  States has spent tens of billions of dollars to develop, assemble, and  operate the ISS over the past two decades, and NASA relies on  uninterrupted crew access to help maintain and operate the station itself  and conduct the research required to enable human exploration in deep  space and eventually Mars, among other science and research goals. In  2015, the United States modified its contract with Roscosmos to provide  crew transportation to the ISS for six astronauts through 2018 with rescue  and return through late spring 2019. The contract extension was valued at  $491 million or approximately $82 million per seat. NASA’s contract with  Roscosmos permits it to delay the use of the final seat by up to 6 months  to late spring 2019, with a return flight approximately 6 months later.
NASA has not yet developed a contingency plan to ensure an  uninterrupted presence on the ISS should the Commercial Crew Program  experience further delays. The ISS program office stated there are other  options it could consider to ensure uninterrupted access to the space  station if neither contractor can provide crew transportation capabilities by  the end of 2018; however, it may already be too late to pursue one of  these options. If NASA determined it needed to purchase additional  Soyuz seats, the process for contracting for those seats typically takes 3  years. In order to avoid a potential crew transportation gap in 2019, the  contracting process would have needed to start in early 2016. In  September 2016, senior NASA officials told us that they do not currently  plan to purchase additional seats from Russia. The ISS program office  stated that NASA could construct another “year in space” mission, similar  to the mission undertaken by NASA astronaut Scott Kelly. According to  the ISS program office, NASA and Roscosmos are discussing this option;  however, there is no agreement in place. Under this scenario, an  astronaut could begin a mission near the end of calendar year 2018 using  one of the final Soyuz seats and would return to Earth via one of the  Commercial Crew Program contractors’ crew transportation systems  approximately one year later in 2019. Finally, the NASA officials reported  that NASA could consider negotiating the acquisition of “stand by” Soyuz  seats, but the availability of those seats is dependent on Russia’s plans  for staffing the station. If NASA does not develop a viable contingency  plan for ensuring access to the ISS in the event of further Commercial  Crew delays, it risks not being able to maximize the return on its  multibillion dollar investment in the space station.
section_title: Programmatic and Safety Risks Also Pose Challenges
paragraphs: In addition to Boeing and SpaceX’s schedule challenges, both contractors  face other risks that will need to be addressed to support their  certification. This includes the contractors’ ability to meet the agency’s  requirements related to the safety of their systems. These risks are not  unusual; there are inherent technical, design, and integration risks in all  NASA’s major acquisitions, as these projects are highly complex and  specialized and often push the state of the art in space technology. The  Commercial Crew Program monitors risks through two lenses:  programmatic risks potentially affect the program’s cost and schedule or  the performance of the crew transportation system, and safety risks could  elevate the potential for the loss of crew. The contractors maintain their  own risk management systems and do not always view their risks in the  same way as the program.
subsections: section_title: Program’s Top Risks for Boeing
paragraphs: The Commercial Crew Program’s top programmatic and safety risks for  Boeing are, in part, related to having adequate information on certain  systems to support certification. For example, the Commercial Crew  Program is tracking a risk about having the data it needs to certify  Boeing’s launch vehicle, ULA’s Atlas V, for manned spaceflight. The Atlas  V’s first stage is powered by the Russian-built ULA-procured RD-180  engine, which has previously been certified to launch national security  and science spacecraft but not humans. ULA and Commercial Crew  Program officials have been working to get access to data about the  engine design, so that they can verify and validate that it meets the  program’s human certification requirements. The program and Boeing  report that access to the data is highly restricted by agreements between  the U.S. and Russian governments. As an alternative, the program has  stated that it is considering whether to certify the engine based on  available data, but program officials believe doing so would be a high risk  for the program. Boeing officials told us that they do not view this as a  safety risk because NASA will not certify the engines without reviewing  the data it needs.
The program is also tracking a risk about having adequate information on  the parachute system. In March 2016, Boeing modified its previously  approved parachute test plan by replacing six drop tests, which simulate  select forces—for example, mass—on the parachute system for one full- scale test event, which simulates all aspects of a parachute system.  Through discussions with the program, Boeing has increased the number  of full-scale test events to five, with an option for two additional tests if  deemed necessary. The program is in the process of reviewing the new  test plan to determine if it will generate enough data for the program to  evaluate the system. Regardless of whether the program approves  Boeing’s new parachute test plan, program officials told us that they plan  to gather additional data on the performance and reliability of both  contractors’ parachute systems. NASA has several contractual options  available to mitigate this risk, if needed. For example, NASA could  choose to add additional analyses or parachute tests to the contract.
section_title: Program’s Top Risks for SpaceX
paragraphs: The Commercial Crew Program’s top programmatic and safety risks for  SpaceX are, in part, related to ongoing launch vehicle design and  development efforts. Prior to SpaceX’s September 2016 loss of a Falcon  9 during pre-launch operations, the program was tracking several risks  related to SpaceX’s launch vehicle. SpaceX has identified five major  block upgrades to its Falcon 9 launch vehicle. SpaceX officials told us  that they have flown the first three block upgrades and are on track to  implement the fourth and fifth block upgrades in 2017. Among other  things, the updated design includes upgrades to the engines and  avionics. The program is tracking a risk that there may not be enough  time for SpaceX to implement these changes and get them approved  prior to the first uncrewed flight test in November 2017. This test flight is a  key activity to demonstrate how SpaceX’s system meets the program’s  requirements. SpaceX needs to have a stable design to support  certification. In addition to planned design changes, there could be  unplanned design changes for the Falcon 9. During qualification testing in  2015, SpaceX identified cracks in the turbines of its engine. Additional  cracks were later identified. Program officials told us that they have  informed SpaceX that the cracks are an unacceptable risk for human  spaceflight. SpaceX officials told us that they are working closely with  NASA to eliminate these cracks in order to meet NASA’s stringent targets  for human rating. Specifically, SpaceX has made design changes that,  according to its officials, did not result in any cracking during initial life  testing. Finally, both the program and a NASA advisory group consider  SpaceX’s plan to fuel the launch vehicle after the astronauts are on board  the spacecraft to be a potential safety risk. SpaceX’s perspective is that  this operation may be a lower risk to the crew; NASA and SpaceX’s risk  evaluation is ongoing. NASA and SpaceX may also need to re-examine  SpaceX’s safety controls related to the fueling process if the investigation  of the September 2016 Falcon 9 mishap identifies issues with the fueling  of the vehicle.
At the time of our review, SpaceX also had other elements in its design  that had not yet been completed and reviewed. SpaceX requested, and  the program approved, proposals to split its critical design review into  three reviews because portions of its design had not been ready at  previous reviews. The critical design review is the time in a project’s life  cycle when the integrity of the product’s design and its ability to meet  mission requirements are assessed, and it is important that a project’s  design is stable enough to warrant continuation with design and  fabrication. A stable design can minimize changes prior to fabrication,  which can help avoid costly re-engineering and rework effort due to  design changes. SpaceX’s final planned design review was held in  August 2016; however, the program reported that a number of  outstanding areas, primarily related to ground systems, still needed to be  reviewed. SpaceX officials told us these areas were reviewed in  November 2016. Further, according to SpaceX, these separate reviews  were in order to perform review of designs that were completed earlier  than anticipated, to allow SpaceX and NASA teams to focus in greater  detail on certain systems, and to accommodate design updates driven in  part by changes to NASA requirements.
section_title: Program and Its Contractors Could Have Difficulty Meeting Safety Requirements
paragraphs: The Commercial Crew Program has identified the ability of it and its  contractors to meet crew safety requirements as one of its top risks.  NASA established the “loss of crew” metric as a way to measure the  safety of a crew transportation system. The metric captures the  probability of death or permanent disability to one or more crew members.  Under each contract, the current loss of crew requirement is 1 in 270,  meaning that the contractors’ systems must carry no more than a 1 in 270  probability of incurring loss of crew. Near the end of the Space Shuttle  program, the probability of loss of crew was approximately 1 in 90.  Program officials told us that Commercial Crew is the first NASA program  that the agency will evaluate against a probabilistic loss of crew  requirement. They said that if the contractors cannot meet the agency  loss of crew requirement at 1 in 270, NASA could still certify their systems  by employing operational alternatives. This would entail a potentially  increased level of risk or uncertainty related to the level of risk for the  crew.
Program officials told us their main focus is to work with the contractors to  ensure that the spacecraft designs are robust from a safety perspective.  The loss of crew metric and the associated models used to measure it are  tools that help achieve that goal. For example, Boeing told us that in early  2016, it needed to identify ways to reduce the mass of its spacecraft. As  Boeing found opportunities to reduce the spacecraft mass, the program  stated that it also had to consider how implementing those design  changes would affect its loss of crew analysis. According to the program,  it is working with both contractors to address the factors that drive loss of  crew risk through design changes or additional testing to gain more  information on the performance and reliability of systems.
The Commercial Crew Program is tracking three main crew safety risks.  First, the contractors’ computer models may not accurately predict the  loss of crew. These models are a weighted treatment of scenarios,  likelihoods, and consequences throughout the flight, and are continually  being updated by the contractors. According to program officials, they  have been working closely with the contractors to improve their loss of  crew models. For example, the program identified risk factors, such as  bird strikes, that were not included in the contractors’ models and worked  to update them. Both contractors told us they have confidence in their  models to accurately predict the loss of crew risk associated with their  spacecraft. Second, the contractors’ spacecraft may not be able to  tolerate the micrometeoroid and orbital debris environment, which is the  most significant driver of the loss of crew metric, according to the  program’s analysis. Both contractors have lowered this risk through  testing, which provides insight into how well their systems perform in  these environments, and by making design changes. If the contractors  have to make future design changes to improve their spacecraft’s  performance in the debris environment, certification could be delayed  significantly. Finally, if the contractors cannot meet the loss of crew  requirement, there are several actions the Commercial Crew Program  could take to help meet it; but, according to the program, these actions  may not be enough to completely close the gap. The program has  reported it is exploring options, such as the use of ISS cameras to  conduct on-orbit inspections of the spacecraft.
section_title: Crew Program Has Visibility into Contractor Progress, but Maintaining Its Current Level of Visibility through Certification Could Add to Schedule Pressures
paragraphs: The Commercial Crew Program is using contractually-defined  mechanisms to gain a high level of visibility into the contractors’ crew  transportation systems to achieve its goal of obtaining safe, reliable, and  cost-effective access to and from the ISS. The program has developed  productive working relationships with both contractors, but the level of  visibility that the program has required thus far to assess the contractors’  systems and ensure their safety has taken more time than the program or  contractors anticipated. The early upfront investment in time may  ultimately make the certification process go smoother, but the program  office could face difficult choices as the program progresses about how to  maintain the level of visibility into contractor efforts it feels it needs without  adding to the program’s schedule pressures. Further, the program faces  potential workload challenges as it works to complete upcoming oversight  activities, while completing others that were already behind schedule.
subsections: section_title: Crew Program Uses Contractually-Defined Mechanisms to Gain Visibility into Contractor Progress
paragraphs: The Commercial Crew Program included mechanisms in its firm-fixed- price contracts that it believed would enable it to gain the visibility into the  contractors’ technical efforts it needs to support a final certification  decision. Contracting officials told us that the program included these  mechanisms because it was concerned that the typical fixed-price  inspection clauses may not give the program the level of visibility it felt it  would need in order to certify the contractors’ systems for human  spaceflight. Using a firm-fixed-price contract for a human spaceflight  development program represents a different way of doing business for  NASA, which has typically used cost-reimbursement contracts for these  efforts due to their complexity and the risks involved. Cost-reimbursement  contracts require the government to maintain extensive visibility into a  contractor’s technical progress and financial performance. The visibility  that NASA receives under its firm-fixed-price contracts with Boeing and  SpaceX is similar to that of cost-reimbursement type contracts, with the  exception that NASA does not receive any of the contractors’ business- related cost and performance data such as earned value management.
In order to gain the needed visibility into the contractors’ efforts, the  Commercial Crew Program used a mix of standard and tailored contract  clauses. The program began by leveraging NASA’s Federal Acquisition  Regulation Supplement, which contemplates NASA conducting  inspections and other quality assurance requirements through “insight”  and “oversight.” In the supplement,  insight is defined as the monitoring of contractor quality data and  government-identified metrics and contract milestones, and any  review of contractor work procedures and records; and oversight is defined as the government’s right to concur or non-concur  with the contractor’s decisions affecting product conformity, and non- concurrence must be resolved before the contractor can proceed.
The Commercial Crew contracting officer explained that the program built  upon what is in the NASA Federal Acquisition Regulation Supplement by  including an additional “insight clause” in the contracts to set clear  expectations that the program intended to obtain extensive information  about how the contractors were meeting contract requirements. For  example, the Commercial Crew contracting officer told us that one such  expectation is that the contractors will provide the program access to  virtually all data produced under or relevant to the contract, including  subcontractor data.
The insight clause includes three mechanisms for the program to gain  visibility into what progress the contractors are making and how the  contractors are performing work. The clause also requires each  contractor to develop an insight implementation plan for how it will provide  access and data to NASA and facilitate the program’s visibility into its  crew transportation systems. Table 2 describes the three mechanisms  and provides examples of how NASA has used them to gain visibility into  the contractors’ progress.
The visibility that the Commercial Crew Program gains through the use of  these contract mechanisms is designed to assist in its oversight and final  certification of the contractors’ crew transportation systems, as shown in  figure 4.
The Commercial Crew Program and its contractors have made progress  working together to ensure the program obtains the level of visibility that it  feels it needs to achieve certification. All three organizations said that  their relationships and communication have improved over the course of  the contract, even as they have addressed difficult issues. For example,  program officials said that Boeing is doing a much better job of  communicating NASA’s safety requirements to ULA than it did during  previous contract phases, and that ULA has made great strides  embracing a crew safety culture as demonstrated through its efforts to  understand the program’s crewed flight requirements. We also heard a  strong consensus across several independent bodies and the contractors  that the Commercial Crew program manager is providing critical  leadership to the program to help the NASA workforce operate effectively  in the firm-fixed-price contract environment. For example, Aerospace  Safety Advisory Panel officials told us that, while they originally had  concerns about how the firm-fixed-price environment would work for a  human spaceflight development, they are confident in the progress of the  program and its future success because of the program manager’s  leadership and transparency.
section_title: Sustaining Program’s Level of Visibility Might Be Difficult as Schedule Pressure Builds
paragraphs: The Commercial Crew Program has developed productive working  relationships with both contractors, but the level of visibility that the  program has required thus far has also taken more time than the program  or contractors anticipated. The program’s standing review board has  stated that the contract is structured to allow NASA unprecedented levels  of visibility, but that it was intended to be used primarily for high-risk  areas. However, the standing review board found, and both contractors  told us, that the program has requested high levels of visibility on most  items and there are signs that the contractors’ patience is waning. Both  contractors expressed concerns that the program requests more  interaction and data than they originally anticipated at the time of the  contract award. For example, Boeing and SpaceX officials told us that the  program often requests additional in-person engagement with their  engineers, such as repeat presentations to multiple boards on the same  technical issue, and has also asked for the same data in multiple formats  or from multiple stakeholders. Program officials told us that they are  constantly working to find a balance between obtaining the visibility they  need to be able to eventually certify the crew transportation systems for  human spaceflight while giving the contractors room to independently  work through issues for their systems.
As the Commercial Crew Program progresses, the program office could  also face difficult choices about how to maintain the level of visibility into  contractor efforts it feels it needs without adding to the program’s  schedule pressures. Independent review bodies, including the standing  review board, the Aerospace Safety Advisory Panel, and the NASA  Advisory Council-Human Exploration and Operations committee,  expressed concern that the program may not have the capacity to sustain  the level of visibility it has had to date and still meet the current  certification schedule. The early upfront investment in time may ultimately  make the certification process go smoother, but finding the right balance  of visibility, and recognizing that a high level of visibility takes time and  may impact the schedule, will be especially important as the contractors  approach final certification when the government will need to determine if  the systems are safe enough for human spaceflight.
section_title: Program Office Workload Is an Emerging Schedule Risk
paragraphs: Program officials told us that one of their greatest upcoming challenges  will be to keep pace with the contractors’ schedules so that the program  does not delay certification. Specifically, they told us they are concerned  about an upcoming “bow wave” of work because the program must  complete two oversight activities—phased safety reviews and verification  closure notices—concurrently in order to support the contractors’ ISS  design certification reviews, uncrewed and crewed flight test missions,  and final certification.
The Commercial Crew Program is working to complete its three-phased  safety review, which will ensure that the contractors have identified all  safety-critical hazards and implemented associated controls, but it is  behind schedule. Both the contractors and the program have contributed  to these delays. In phase one, Boeing and SpaceX identified risks in their  designs and developed reports on potential hazards, the controls they put  in place to mitigate them, and explanations for how the controls will  mitigate the hazards. In phase two, which is ongoing, the program  reviews and approves the contractors’ hazard reports, and develops  strategies to verify and validate that the controls are effective. In phase  three, the contractors will conduct the verification activities and  incrementally close the reports.
The Commercial Crew Program’s review and approval of the contractors’  hazard reports have taken longer than planned. The program originally  planned to complete phase two in early 2016 but currently does not  expect to complete this phase until June 2017. The Commercial Crew  Program has a goal of reviewing hazard reports within 8 weeks of  receiving them, but a recent report by the NASA Office of Inspector  General found that the reviews are taking longer than anticipated and a  backlog has developed. In response to the Inspector General’s report,  NASA officials noted that, while the timeliness of the hazard review  process is important, what is more important is having thorough, detailed  hazard reports in order to understand safety risks and ensuring the safety  of each system. As of October 2016, the Commercial Crew Program had  approved 117 of the anticipated 195 hazard reports and planned to  approve approximately half of the remaining reports for both contractors  by the end of 2016. Program officials told us that the hazard reports that  are still open are related to items that they would not expect to be closed  because they involve some of the more complicated design work that the  contractors have not yet finalized. Program officials also pointed out other  ways that the contractors have contributed to phase two delays, including  receiving incomplete hazard reports that required several iterations before  the program could begin its formal review.
The Commercial Crew Program’s verification closure notice process,  which is used to verify that the contractors have met all requirements, is  one of the other key oversight activities and potential workload challenges  for the program. The program is completing that process concurrently  with the phased safety reviews. The verification closure process is  initiated by the contractor when it provides the program with data and  evidence to substantiate that it has met each requirement, and is  completed when the program has reviewed and approved the contractor’s  evidence to verify that each requirement has been met. A verification  closure notice is required for each of the 332 ISS requirements, which are  applicable to anyone who docks with the station, and the 280 Commercial  Crew Program requirements. The Commercial Crew Program must also  approve a subset of verification closure notices before key tests or  milestones can occur. For example, the ISS requirements must be met  before SpaceX and Boeing’s uncrewed flights to the ISS, which are  currently planned for November 2017 and June 2018, respectively. The  program’s ability to smooth its workload is limited because the contractors  generally control their development schedules. Proposed changes to the  Boeing and SpaceX schedules could help alleviate some of the  concurrency between the program’s phased safety reviews and  verification closure process.
section_title: Conclusions
paragraphs: NASA’s Commercial Crew Program is a multibillion dollar effort intended  to end the United States’ reliance on Russia to maintain an uninterrupted  presence on the ISS. To do so, NASA embarked on a different way of  doing business. It awarded firm-fixed-price contracts to Boeing and  SpaceX that transferred the financial risks to the contractors while giving  them freedom to develop unique transportation systems that meet  NASA’s standards. An independent review group initially raised concerns  about how well this model would work for a human spaceflight program.  Parts of the new business model have worked relatively well and NASA  has had the visibility it needs into the technical details and risks of each  system. Gaining this level of visibility has taken more time than the  program or contractors anticipated, but the early upfront investment in  time should ultimately make the certification process go smoother. In  addition, while the Commercial Crew Program should be mindful about  placing undue burdens on its contractors, it ultimately has the  responsibility for ensuring the safety of U.S. astronauts, and its contracts  with Boeing and SpaceX give it deference to determine the level of insight  required to do so.
When the current phase of the Commercial Crew Program began, there  was widespread acknowledgment that the contractors’ development and  certification schedules were aggressive, and the anticipated schedule  risks have now materialized. Both contractors’ certification dates have  been delayed into 2018, and the program’s analysis indicates that neither  contractor is likely to be ready until 2019. NASA has purchased seats for  U.S. astronauts on the Russian Soyuz vehicle, so that it can maintain an  uninterrupted presence on the ISS. But those seats will run out in spring  2019, at the latest, and it generally takes 3 years for NASA and Russia to  negotiate the purchase of additional seats. If delays on the Commercial  Crew Program persist, having contingency plans, including options to  expedite the purchase of additional Soyuz seats, will become increasingly  important to ensure an uninterrupted U.S. presence on the ISS. Without a  viable contingency plan that has been communicated to Congress, NASA  puts at risk its ability to maximize the utility of the ISS and the return on  the multibillion dollar investment it has made in the space station.
section_title: Recommendation for Executive Action
paragraphs: In order to ensure that the United States has continued access to the ISS  if the Commercial Crew Program’s contractors experience additional  schedule delays, we recommend the NASA Administrator develop a  contingency plan for maintaining a presence on the ISS beyond 2018,  including options to purchase additional Russian Soyuz seats, and report  to Congress on the results.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to NASA for comment. In written  comments, NASA agreed with our recommendation and intends to  develop a contingency plan by mid-March 2017. These comments are  reproduced in Appendix III. NASA also provided technical comments,  which have been addressed in the report, as appropriate.
We are sending copies of the report to the NASA Administrator and  interested congressional committees. In addition, the report will be  available at no charge on GAO’s website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202-512-4841 or chaplainc@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made major contributions to  this report are listed in Appendix IV.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: The objectives of our review were to assess (1) the extent to which the  contractors have made progress towards certification and the potential  effects of any certification delays on the National Aeronautics and Space  Administration’s (NASA) access to the International Space Station (ISS);  (2) the major programmatic and safety risks facing the program and the  contractors; and (3) the extent to which the program has visibility into the  contractors’ efforts.
To assess the contractors’ progress towards certification, we obtained  and reviewed program and contractor documents, including monthly and  quarterly updates as well as monthly schedule summaries, from March  2016 through December 2016. We interviewed program and contractor  officials to discuss the contractors’ recent progress as well as their  upcoming events and any expected delays. To identify total delays to  date, we compared original contract schedules to Boeing’s October 2016  working schedule and SpaceX’s November 2016 working schedule, which  identify their most recent proposed delays to some milestones. To identify  whether the contractors have introduced compression into their  schedules, we judgmentally selected four key events and analyzed the  movement of these key events relative to each other. Based on our  review of program and contractor documents, we selected the contract’s  acceptance events—the ISS design certification review and final  certification review—as well as the uncrewed and crewed flight tests to  conduct this analysis. We selected the acceptance events, as these occur  when NASA approves a contractor’s designs and acknowledges that the  contractor’s work is complete and meets the requirements of the contract.  We selected the two flight tests for each contractor, as they are intended  to test key system capabilities including the ability to launch, dock with the  ISS, and return safely to Earth. Finally, to assess the potential effects of  any certification delays on NASA’s access to ISS, we reviewed  information from NASA’s budget, which includes the planned ISS launch  manifest, and its contract with the Russian Federal Space Agency for  transportation on the Soyuz vehicle. We also obtained information from  the ISS program and the NASA Associate Administrator to determine if  the agency had developed contingency plans to mitigate the effects of  any certification delays on its access to the ISS.
To assess the programmatic and safety risks for the Commercial Crew  Program and its contractors, we obtained and reviewed monthly and  quarterly reports, as well as the risks tracked in both the program’s and  the contractors’ risk management systems, from March 2016 through  December 2016. We interviewed program and contractor officials with  knowledge of the technical risks to understand the risks and potential  impacts and how they are planning to mitigate those risks.
To assess how the program obtains visibility into its contractors’ systems  and efforts, we reviewed the program’s contracts with its two contractors.  In particular, we reviewed the contract “insight” clause, which outlines  three mechanisms available to the program for obtaining visibility. We  interviewed program officials in order to understand the three  mechanisms and how they are used to gain visibility. We also interviewed  officials from both contractors to gain their perspectives on the insight  mechanisms. We also analyzed how NASA obtained visibility into  contractor efforts at key events. We defined key events as those that are  necessary for final certification. We selected three key events to examine  for each contractor. We chose one historical event, so we could assess  how the program had gained visibility over an entire event, and one  upcoming event in both 2016 and 2017, in order to understand how the  program obtains visibility in advance of an event. For each event, we  obtained detailed information on how NASA obtained visibility and  analyzed supporting documentation for these events to corroborate the  information we obtained from NASA officials. In this report, we used these  events as examples when describing how the program uses its three  insight mechanisms and when describing the program’s relationships with  its contractors. We also interviewed the program and contracting officials  to obtain their perspectives on the level of visibility that the program has  received into the contractors’ systems thus far. Finally, we analyzed  program documentation and interviewed program officials to understand  the level of work that NASA needs to complete leading up to certification.  We assessed the timing of NASA’s reviews and spoke with program  officials to determine if the program’s workload could be affected by the  contractors’ schedules.
For all three objectives, we met with officials from all three organizations  that provide NASA with independent assessments of the program: the  program’s standing review board, the NASA Advisory Council-Human  Exploration and Operations subcommittee, and the Aerospace Safety  Advisory Panel. We spoke with these three groups to gain their  perspectives on the program’s oversight of each contractor’s technical  risks and schedules as well as the program’s level of visibility into the  contractors’ systems.
We conducted this performance audit from March 2016 to February 2017  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Commercial Crew Program Phases and Participants
paragraphs: The Commercial Crew Program is a multi-phased effort that began in  2010. Table 3 describes each phase, its purpose, the participants, and  the level of funding provided by the National Aeronautics and Space  Administration.
section_title: Appendix III: Comments from the National Aeronautics and Space Administration
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the contact named above, Molly Traci, Assistant Director;  Ron Schwenn, Assistant Director; Carly Gerbig; Laura Greifner; Kurt  Gurka; Michael Kaeser; Katherine Lenane; Roxanna T. Sun; and Kristin  Van Wychen made significant contributions to this report.