NASA Commercial Crew Program: Schedule Pressure Increases as Contractors Delay Key Events
Released: 2017-02-16T00:00:00

Letter

Background

NASA’s Commercial Crew Program is a multi-phased effort that began in 2010. Across the five phases, NASA has engaged several companies using both agreements and contract vehicles to develop and demonstrate crew transportation capabilities. As the program has passed through these phases, NASA has generally narrowed down the number of participants. The early phases of the program were under Space Act agreements, which is NASA’s other transaction authority. These types of agreements are generally not subject to the Federal Acquisition Regulation (FAR) and allow the government and its contractors greater flexibility in many areas. Under these Space Act agreements, NASA relied on the commercial companies to propose specifics related to their crew transportation systems, including their design, the capabilities they would provide, and the level of private investment. In these phases, NASA provided technical support and determined if the contractors met certain technical milestones. In most cases, NASA also provided funding. For the final two phases of the program, NASA awarded FAR-based contracts. By using FAR-based contracts, NASA gained the ability to levy specific requirements on the contractors and procure missions to the ISS, while continuing to provide technical expertise and funding to the contractors. Under the contracts, NASA will also evaluate whether contractors have met its requirements and certify their final systems for use. Appendix II contains a description of each of the Commercial Crew Program’s phases, a list of the participants, and the level of funding provided to each participant by NASA.

Program Requirements

Before a company’s crew transportation system can be certified by NASA, it must meet two sets of requirements. The ISS program levies a set of 332 requirements that must be met by all visiting spacecraft, whether they are carrying cargo or crew to the station. There are three major areas outlined in the ISS requirements document: 1) interface requirements for both the ISS and the spacecraft; 2) performance requirements for ground systems supporting the spacecraft; and 3) design requirements for spacecraft to ensure safe integration with the ISS. The second set of requirements is levied by the Commercial Crew Program. These include 280 high-level requirements related to the design, manufacturing, testing, qualification, production, and operation of crew transportation systems that deliver NASA astronauts to the ISS. Contractors are responsible for developing the lower-level system specifications and system and subsystem designs that implement those requirements. For example, there is a Commercial Crew Program requirement for the contractor’s crew transportation system to provide a continuous autonomous launch abort capability, in which the spacecraft self-initiates the abort procedure independent from the supporting ground systems, from lift-off through orbital insertion with a 95 percent probability of success. The requirement does not specify how elements of the crew transportation system, such as its sensors, software, propulsion, or capsule, should be designed to fulfill it.

Current Program Contracts

In September 2014, NASA awarded firm-fixed-price contracts to Boeing and SpaceX, valued at up to $4.2 billion and $2.6 billion, respectively, for the Commercial Crew Transportation Capability phase. Under a firm- fixed-price contract, the contractor must perform a specified amount of work for the price negotiated by the contractor and government. This is in contrast to a cost-reimbursement contract, in which the government agrees to pay the contractor’s reasonable costs regardless of whether work is completed. During this phase, the contractors will complete development of crew transportation systems that meet NASA requirements, provide NASA with the evidence it needs to certify that those systems meet its requirements, and fly initial crewed missions to the ISS. Under the contracts, NASA and the companies originally planned to complete the certification review for each system by 2017. Figure 1 shows the spacecraft and launch vehicles for Boeing and SpaceX’s crew transportation systems.

The Commercial Crew Transportation Capability phase contracts include three types of services:

Contract Line Item 001 encompasses the firm-fixed-price design, development, test, and evaluation work needed to support NASA’s final certification of the contractor’s spacecraft, launch vehicle, and ground support systems.

Contract Line Item 002 covers any service missions that NASA orders to transport astronauts to and from the ISS. Under this indefinite-delivery, indefinite-quantity line item, NASA must order at least two missions from each contractor, which it has done, and can order up to six missions. Each service mission is its own firm-fixed- price task order. NASA must certify the contractors’ systems before they can fly these missions.

Contract Line Item 003 is an indefinite-delivery, indefinite-quantity line item for any special studies, tests, and analyses that NASA may request. These tasks do not include any work necessary to accomplish the requirements under contract line item 001 and 002. As of September 2016, NASA had issued one order under this contract line item to Boeing—an approximately $180,000 study of the spacecraft’s seat incline. The maximum value of this contract line item is $150 million.

NASA divided the certification work under contract line item 001 into two acceptance events: the ISS design certification review and the certification review. An acceptance event occurs when NASA approves a contractor’s designs and acknowledges that the contractor’s work is complete and meets the requirements of the contract. The ISS design certification review verifies the contractor’s crew transportation system’s capability to safely approach, dock, mate, and depart from the ISS, among other requirements. After the contractor has successfully completed all its flight tests, as well as various other activities, the certification review determines whether the crew transportation system meets the Commercial Crew Program’s requirements. The contractors must complete both acceptance events to receive NASA certification.

NASA and the contractors also identified discrete performance-based events, called interim milestones, which occur as the contractors progress towards the two acceptance events. Each interim milestone has pre- determined entrance and exit criteria that establish the work that must be completed in order for the contractor to receive payment. The interim milestones serve several functions, allowing the government to finance work from development to completion, review the contractors’ progress, and provide approval to proceed with key demonstrations and tests. These milestones are also used by the program to inform its annual budget request. Since the contracts were awarded, the Commercial Crew Program and the contractors have agreed to split several of the interim milestones. The contractors have also added new milestones, in part to capture changes in their development plans.

NASA has also made changes to the contracts that have increased their value. While the contracts are fixed-price, their values can increase if NASA adds to the scope of the work or otherwise changes requirements. As of October 2016, NASA had increased the value of contract line item 001 for Boeing by $47 million for hardware and software requirement changes, and contract line item 001 for SpaceX by $91 million for a hardware requirement change and the addition of cargo during an ISS test flight.

Program Oversight

NASA has tailored its management approach for the Commercial Crew Program because the contractors, not the government, are responsible for the design, development, test, and evaluation of their crew transportation systems and make their own decisions about when to build, integrate, and test hardware. For example, NASA policy outlines key decision point reviews, which allow NASA management to determine if the program is ready to progress to the next phase. NASA management relies on annual program reviews, instead of key decision point reviews, to provide oversight of the Commercial Crew Program. According to NASA management officials, they chose to do so because the contractors’ development approaches and schedules did not align with NASA’s typical key decision points. Other elements of the reviews are similar though. For example, NASA’s first annual review of the program included updates on overall program risk, an update on the program’s cost and schedule risks, and perspectives from the independent review board, much like a key decision point review. The firm-fixed price nature of the current contracts also led NASA to alter its typical management approach. For example, the NASA Associate Administrator told us that he decided not to require an agency baseline commitment—cost or schedule baseline—for the Commercial Crew Program, in part, because the firm-fixed-price contracts with each contractor essentially serve as the cost baselines for the program.

NASA also provides regular updates to Congress on the Commercial Crew Program through a variety of reporting mechanisms. NASA includes information on all of its major projects, including Commercial Crew, in its annual budget submission, although the type and specificity of the information can differ. For example, table 1 shows the key differences in the types of information NASA provided on two human space flight projects—the Commercial Crew Program and the Orion Crew Vehicle development in its fiscal year 2017 budget request. In addition to its budget, NASA submits quarterly reports to the appropriations committees on the Commercial Crew Program’s progress that include cost and schedule updates from each contractor and information on contractor specific-risks, including procurement-sensitive material that is not publically releasable.

Neither Contractor Expects to Achieve Certification before 2018 and NASA Has Not Yet Determined How It Will Ensure ISS Access in Case of Further Delays

Since September 2014, both Boeing and SpaceX have made progress developing their crew transportation systems, but neither contractor will be able to meet their original 2017 certification dates and both expect certification to be delayed until 2018. As the contractors have fallen behind, the time between key test events has decreased, which reduces the time the contractors have to learn and implement changes, increasing the likelihood of additional delays. The schedule pressures are amplified by NASA’s need to provide a viable crew transportation option to the ISS before its current contract with Russia’s space agency runs out in 2019. NASA has not yet developed a contingency plan to ensure an uninterrupted presence on the ISS should the Commercial Crew Program experience further delays, although the ISS program has begun to discuss potential options.

Both Contractors Have Begun Manufacturing Spacecraft and Modifying Launch Facilities

Both Boeing and SpaceX have made progress finalizing their designs and building hardware as they work towards final certification of their crew transportation systems. Each contractor’s system includes a spacecraft and a launch vehicle with supporting ground systems. Examples of the contractors’ development progress since September 2014 include the following:

Boeing: In 2015, Boeing completed its critical design review, which determines whether a system’s design is stable enough to proceed with final design and fabrication, and began fabricating the first of its three planned spacecraft. This spacecraft will be tested to validate the design, including in a planned January 2018 pad abort test where the ability of the capsule’s abort system is tested from a static position on the launch pad. Further, in May 2016, Boeing shipped the structural test article for its service module to its testing facility and shipped the crew module in January 2017. These modules will be used to conduct tests that simulate different operating environments, such as the vibrations during a launch ascent, on key spacecraft components. Boeing and its launch vehicle provider, United Launch Alliance (ULA), have also made significant modifications to its launch facilities at the Cape Canaveral Air Force Station, including installing a crew access tower and crew access arm, which the crew will use to board the crew capsule prior to launch.

SpaceX: SpaceX completed its critical design review in 2016 and has begun assembling and integrating components of the two spacecraft it will use for its uncrewed and crewed flight tests, which are scheduled for November 2017 and May 2018, respectively. SpaceX officials told us they completed integrated testing and qualification of several major spacecraft components, including its environmental control and life support system—which controls air quality and temperature, among other functions, to ensure crew survivability—and its spacesuit in November 2016. SpaceX also made significant modifications to its launch facilities at the Kennedy Space Center, including constructing a new hangar and an upgraded flame trench, which is critical to safely contain the plume exhaust from rocket launches, and substantially upgrading the existing crew access tower, which was last used during the Space Shuttle era.

Both Contractors Have Delayed Certification and Reduced Time between Key Test Events

Neither Boeing nor SpaceX will be able to meet their original 2017 certification dates and both now expect certification to be delayed until 2018. Since the award of the current Commercial Crew contracts, the program, Boeing, and SpaceX have all identified the contractors’ delivery schedules as aggressive. Program officials told us that, from the outset, they knew delays were likely due to the development nature of the program. Multiple independent review bodies—including the program’s standing review board, the Aerospace Safety Advisory Panel, and the NASA Advisory Council-Human Exploration and Operations committee— also noted the aggressiveness of the contractors’ schedules as they move toward certification.

Both contractors have notified NASA that they would not be able to meet the 2017 certification dates originally established in their contracts. These notices required the parties to renegotiate their contracts to reflect the contractors’ delays. As of October 2016, both contractors have submitted, and NASA has agreed to, updated schedules that reflect delays to their final certification review dates—Boeing by 5 months, from August 2017 until January 2018, and SpaceX by 6 months, from April 2017 until October 2017. Boeing and SpaceX’s internal schedules both show additional certification delays that, as of November 2016, were not yet reflected in their contracts. As the contractors have made changes to their development schedules, they have also reduced the number of months between critical test events leading up to final certification, which reduces the time the contractors have to learn, make any needed design changes, and implement those changes, increasing the likelihood of additional delays. Figures 2 and 3 below show the total proposed certification delay and current proposed schedule for each contractor.

Since Boeing and the Commercial Crew Program agreed to move Boeing’s certification review from August 2017 to January 2018, Boeing has proposed moving the review out by at least 9 additional months to the fourth quarter of 2018. The current proposed schedule is the fourth significant schedule change by Boeing since the contract was awarded. The most recent schedule update includes delays related to a manufacturing error. In September 2016, a subcontractor damaged a major component of Boeing’s second spacecraft during machining to trim mass from the spacecraft. This is expected to delay delivery of the component by 2-3 months, which is reflected in the current proposed schedule. The second spacecraft is important because, according to Boeing’s program schedule, it will be used to support a significant portion of Boeing’s planned testing as well as its crewed flight test.

Boeing also reduced the time between some key testing events that will be used to provide data necessary for Boeing to demonstrate its ability to meet NASA’s requirements. In the original schedule, Boeing allocated 3 months between its uncrewed flight test and its crewed flight test, but under its current proposed schedule, there are only 2 months between these critical test events. Although these events are not formal milestones in Boeing’s contract with NASA, they are critical learning events, and we have previously found that reducing the time between key test events limits time for the contractor to learn and adapt to any changes that may be required. Boeing is also tracking a risk that the schedule could be further delayed because there is little time between test events.

Since SpaceX and the Commercial Crew Program agreed to move its certification review from April 2017 to October 2017, SpaceX has proposed moving the review out by at least 9 additional months to the third quarter of 2018. SpaceX officials stated that these delays are largely driven by development challenges, changes in NASA requirements, and implementation of corrective actions stemming from a launch vehicle mishap in 2016. On September 1, 2016, SpaceX experienced an anomaly during a standard, pre-launch static fire test of its Falcon 9 launch vehicle that resulted in an explosion and the loss of the vehicle. The mishap investigation is complete and SpaceX returned to flight in January 2017 with a commercial launch. However, additional schedule changes are possible. SpaceX officials told us that they continue to assess the effect of the mishap on their Commercial Crew schedule. Further, as SpaceX has made schedule changes, it has also reduced the time between its uncrewed flight test and crewed flight test from 7 months to 6 months.

The Commercial Crew Program is tracking risks that both contractors could experience additional schedule delays and its own analysis indicates that certification is likely to slip into 2019. One of the program’s top six programmatic risks for each contractor is the likelihood of additional schedule delays. Each month, the program updates its schedule risk analysis, based on the contractors’ internal schedules as well as the program’s perspectives and insight into specific technical risks. As of October 2016, the program’s schedule risk analysis indicated that both contractors’ certification dates would likely slip into early 2019.

The mounting schedule pressure on the Commercial Crew Program from the contractors’ delays is amplified by NASA’s need to provide a viable crew transportation option to the ISS before its current contract with Roscosmos, the Russian Federal Space Agency, runs out. The United States has spent tens of billions of dollars to develop, assemble, and operate the ISS over the past two decades, and NASA relies on uninterrupted crew access to help maintain and operate the station itself and conduct the research required to enable human exploration in deep space and eventually Mars, among other science and research goals. In 2015, the United States modified its contract with Roscosmos to provide crew transportation to the ISS for six astronauts through 2018 with rescue and return through late spring 2019. The contract extension was valued at $491 million or approximately $82 million per seat. NASA’s contract with Roscosmos permits it to delay the use of the final seat by up to 6 months to late spring 2019, with a return flight approximately 6 months later.

NASA has not yet developed a contingency plan to ensure an uninterrupted presence on the ISS should the Commercial Crew Program experience further delays. The ISS program office stated there are other options it could consider to ensure uninterrupted access to the space station if neither contractor can provide crew transportation capabilities by the end of 2018; however, it may already be too late to pursue one of these options. If NASA determined it needed to purchase additional Soyuz seats, the process for contracting for those seats typically takes 3 years. In order to avoid a potential crew transportation gap in 2019, the contracting process would have needed to start in early 2016. In September 2016, senior NASA officials told us that they do not currently plan to purchase additional seats from Russia. The ISS program office stated that NASA could construct another “year in space” mission, similar to the mission undertaken by NASA astronaut Scott Kelly. According to the ISS program office, NASA and Roscosmos are discussing this option; however, there is no agreement in place. Under this scenario, an astronaut could begin a mission near the end of calendar year 2018 using one of the final Soyuz seats and would return to Earth via one of the Commercial Crew Program contractors’ crew transportation systems approximately one year later in 2019. Finally, the NASA officials reported that NASA could consider negotiating the acquisition of “stand by” Soyuz seats, but the availability of those seats is dependent on Russia’s plans for staffing the station. If NASA does not develop a viable contingency plan for ensuring access to the ISS in the event of further Commercial Crew delays, it risks not being able to maximize the return on its multibillion dollar investment in the space station.

Programmatic and Safety Risks Also Pose Challenges

In addition to Boeing and SpaceX’s schedule challenges, both contractors face other risks that will need to be addressed to support their certification. This includes the contractors’ ability to meet the agency’s requirements related to the safety of their systems. These risks are not unusual; there are inherent technical, design, and integration risks in all NASA’s major acquisitions, as these projects are highly complex and specialized and often push the state of the art in space technology. The Commercial Crew Program monitors risks through two lenses: programmatic risks potentially affect the program’s cost and schedule or the performance of the crew transportation system, and safety risks could elevate the potential for the loss of crew. The contractors maintain their own risk management systems and do not always view their risks in the same way as the program.

Program’s Top Risks for Boeing

The Commercial Crew Program’s top programmatic and safety risks for Boeing are, in part, related to having adequate information on certain systems to support certification. For example, the Commercial Crew Program is tracking a risk about having the data it needs to certify Boeing’s launch vehicle, ULA’s Atlas V, for manned spaceflight. The Atlas V’s first stage is powered by the Russian-built ULA-procured RD-180 engine, which has previously been certified to launch national security and science spacecraft but not humans. ULA and Commercial Crew Program officials have been working to get access to data about the engine design, so that they can verify and validate that it meets the program’s human certification requirements. The program and Boeing report that access to the data is highly restricted by agreements between the U.S. and Russian governments. As an alternative, the program has stated that it is considering whether to certify the engine based on available data, but program officials believe doing so would be a high risk for the program. Boeing officials told us that they do not view this as a safety risk because NASA will not certify the engines without reviewing the data it needs.

The program is also tracking a risk about having adequate information on the parachute system. In March 2016, Boeing modified its previously approved parachute test plan by replacing six drop tests, which simulate select forces—for example, mass—on the parachute system for one full- scale test event, which simulates all aspects of a parachute system. Through discussions with the program, Boeing has increased the number of full-scale test events to five, with an option for two additional tests if deemed necessary. The program is in the process of reviewing the new test plan to determine if it will generate enough data for the program to evaluate the system. Regardless of whether the program approves Boeing’s new parachute test plan, program officials told us that they plan to gather additional data on the performance and reliability of both contractors’ parachute systems. NASA has several contractual options available to mitigate this risk, if needed. For example, NASA could choose to add additional analyses or parachute tests to the contract.

Program’s Top Risks for SpaceX

The Commercial Crew Program’s top programmatic and safety risks for SpaceX are, in part, related to ongoing launch vehicle design and development efforts. Prior to SpaceX’s September 2016 loss of a Falcon 9 during pre-launch operations, the program was tracking several risks related to SpaceX’s launch vehicle. SpaceX has identified five major block upgrades to its Falcon 9 launch vehicle. SpaceX officials told us that they have flown the first three block upgrades and are on track to implement the fourth and fifth block upgrades in 2017. Among other things, the updated design includes upgrades to the engines and avionics. The program is tracking a risk that there may not be enough time for SpaceX to implement these changes and get them approved prior to the first uncrewed flight test in November 2017. This test flight is a key activity to demonstrate how SpaceX’s system meets the program’s requirements. SpaceX needs to have a stable design to support certification. In addition to planned design changes, there could be unplanned design changes for the Falcon 9. During qualification testing in 2015, SpaceX identified cracks in the turbines of its engine. Additional cracks were later identified. Program officials told us that they have informed SpaceX that the cracks are an unacceptable risk for human spaceflight. SpaceX officials told us that they are working closely with NASA to eliminate these cracks in order to meet NASA’s stringent targets for human rating. Specifically, SpaceX has made design changes that, according to its officials, did not result in any cracking during initial life testing. Finally, both the program and a NASA advisory group consider SpaceX’s plan to fuel the launch vehicle after the astronauts are on board the spacecraft to be a potential safety risk. SpaceX’s perspective is that this operation may be a lower risk to the crew; NASA and SpaceX’s risk evaluation is ongoing. NASA and SpaceX may also need to re-examine SpaceX’s safety controls related to the fueling process if the investigation of the September 2016 Falcon 9 mishap identifies issues with the fueling of the vehicle.

At the time of our review, SpaceX also had other elements in its design that had not yet been completed and reviewed. SpaceX requested, and the program approved, proposals to split its critical design review into three reviews because portions of its design had not been ready at previous reviews. The critical design review is the time in a project’s life cycle when the integrity of the product’s design and its ability to meet mission requirements are assessed, and it is important that a project’s design is stable enough to warrant continuation with design and fabrication. A stable design can minimize changes prior to fabrication, which can help avoid costly re-engineering and rework effort due to design changes. SpaceX’s final planned design review was held in August 2016; however, the program reported that a number of outstanding areas, primarily related to ground systems, still needed to be reviewed. SpaceX officials told us these areas were reviewed in November 2016. Further, according to SpaceX, these separate reviews were in order to perform review of designs that were completed earlier than anticipated, to allow SpaceX and NASA teams to focus in greater detail on certain systems, and to accommodate design updates driven in part by changes to NASA requirements.

Program and Its Contractors Could Have Difficulty Meeting Safety Requirements

The Commercial Crew Program has identified the ability of it and its contractors to meet crew safety requirements as one of its top risks. NASA established the “loss of crew” metric as a way to measure the safety of a crew transportation system. The metric captures the probability of death or permanent disability to one or more crew members. Under each contract, the current loss of crew requirement is 1 in 270, meaning that the contractors’ systems must carry no more than a 1 in 270 probability of incurring loss of crew. Near the end of the Space Shuttle program, the probability of loss of crew was approximately 1 in 90. Program officials told us that Commercial Crew is the first NASA program that the agency will evaluate against a probabilistic loss of crew requirement. They said that if the contractors cannot meet the agency loss of crew requirement at 1 in 270, NASA could still certify their systems by employing operational alternatives. This would entail a potentially increased level of risk or uncertainty related to the level of risk for the crew.

Program officials told us their main focus is to work with the contractors to ensure that the spacecraft designs are robust from a safety perspective. The loss of crew metric and the associated models used to measure it are tools that help achieve that goal. For example, Boeing told us that in early 2016, it needed to identify ways to reduce the mass of its spacecraft. As Boeing found opportunities to reduce the spacecraft mass, the program stated that it also had to consider how implementing those design changes would affect its loss of crew analysis. According to the program, it is working with both contractors to address the factors that drive loss of crew risk through design changes or additional testing to gain more information on the performance and reliability of systems.

The Commercial Crew Program is tracking three main crew safety risks. First, the contractors’ computer models may not accurately predict the loss of crew. These models are a weighted treatment of scenarios, likelihoods, and consequences throughout the flight, and are continually being updated by the contractors. According to program officials, they have been working closely with the contractors to improve their loss of crew models. For example, the program identified risk factors, such as bird strikes, that were not included in the contractors’ models and worked to update them. Both contractors told us they have confidence in their models to accurately predict the loss of crew risk associated with their spacecraft. Second, the contractors’ spacecraft may not be able to tolerate the micrometeoroid and orbital debris environment, which is the most significant driver of the loss of crew metric, according to the program’s analysis. Both contractors have lowered this risk through testing, which provides insight into how well their systems perform in these environments, and by making design changes. If the contractors have to make future design changes to improve their spacecraft’s performance in the debris environment, certification could be delayed significantly. Finally, if the contractors cannot meet the loss of crew requirement, there are several actions the Commercial Crew Program could take to help meet it; but, according to the program, these actions may not be enough to completely close the gap. The program has reported it is exploring options, such as the use of ISS cameras to conduct on-orbit inspections of the spacecraft.

Crew Program Has Visibility into Contractor Progress, but Maintaining Its Current Level of Visibility through Certification Could Add to Schedule Pressures

The Commercial Crew Program is using contractually-defined mechanisms to gain a high level of visibility into the contractors’ crew transportation systems to achieve its goal of obtaining safe, reliable, and cost-effective access to and from the ISS. The program has developed productive working relationships with both contractors, but the level of visibility that the program has required thus far to assess the contractors’ systems and ensure their safety has taken more time than the program or contractors anticipated. The early upfront investment in time may ultimately make the certification process go smoother, but the program office could face difficult choices as the program progresses about how to maintain the level of visibility into contractor efforts it feels it needs without adding to the program’s schedule pressures. Further, the program faces potential workload challenges as it works to complete upcoming oversight activities, while completing others that were already behind schedule.

Crew Program Uses Contractually-Defined Mechanisms to Gain Visibility into Contractor Progress

The Commercial Crew Program included mechanisms in its firm-fixed- price contracts that it believed would enable it to gain the visibility into the contractors’ technical efforts it needs to support a final certification decision. Contracting officials told us that the program included these mechanisms because it was concerned that the typical fixed-price inspection clauses may not give the program the level of visibility it felt it would need in order to certify the contractors’ systems for human spaceflight. Using a firm-fixed-price contract for a human spaceflight development program represents a different way of doing business for NASA, which has typically used cost-reimbursement contracts for these efforts due to their complexity and the risks involved. Cost-reimbursement contracts require the government to maintain extensive visibility into a contractor’s technical progress and financial performance. The visibility that NASA receives under its firm-fixed-price contracts with Boeing and SpaceX is similar to that of cost-reimbursement type contracts, with the exception that NASA does not receive any of the contractors’ business- related cost and performance data such as earned value management.

In order to gain the needed visibility into the contractors’ efforts, the Commercial Crew Program used a mix of standard and tailored contract clauses. The program began by leveraging NASA’s Federal Acquisition Regulation Supplement, which contemplates NASA conducting inspections and other quality assurance requirements through “insight” and “oversight.” In the supplement, insight is defined as the monitoring of contractor quality data and government-identified metrics and contract milestones, and any review of contractor work procedures and records; and oversight is defined as the government’s right to concur or non-concur with the contractor’s decisions affecting product conformity, and non- concurrence must be resolved before the contractor can proceed.

The Commercial Crew contracting officer explained that the program built upon what is in the NASA Federal Acquisition Regulation Supplement by including an additional “insight clause” in the contracts to set clear expectations that the program intended to obtain extensive information about how the contractors were meeting contract requirements. For example, the Commercial Crew contracting officer told us that one such expectation is that the contractors will provide the program access to virtually all data produced under or relevant to the contract, including subcontractor data.

The insight clause includes three mechanisms for the program to gain visibility into what progress the contractors are making and how the contractors are performing work. The clause also requires each contractor to develop an insight implementation plan for how it will provide access and data to NASA and facilitate the program’s visibility into its crew transportation systems. Table 2 describes the three mechanisms and provides examples of how NASA has used them to gain visibility into the contractors’ progress.

The visibility that the Commercial Crew Program gains through the use of these contract mechanisms is designed to assist in its oversight and final certification of the contractors’ crew transportation systems, as shown in figure 4.

The Commercial Crew Program and its contractors have made progress working together to ensure the program obtains the level of visibility that it feels it needs to achieve certification. All three organizations said that their relationships and communication have improved over the course of the contract, even as they have addressed difficult issues. For example, program officials said that Boeing is doing a much better job of communicating NASA’s safety requirements to ULA than it did during previous contract phases, and that ULA has made great strides embracing a crew safety culture as demonstrated through its efforts to understand the program’s crewed flight requirements. We also heard a strong consensus across several independent bodies and the contractors that the Commercial Crew program manager is providing critical leadership to the program to help the NASA workforce operate effectively in the firm-fixed-price contract environment. For example, Aerospace Safety Advisory Panel officials told us that, while they originally had concerns about how the firm-fixed-price environment would work for a human spaceflight development, they are confident in the progress of the program and its future success because of the program manager’s leadership and transparency.

Sustaining Program’s Level of Visibility Might Be Difficult as Schedule Pressure Builds

The Commercial Crew Program has developed productive working relationships with both contractors, but the level of visibility that the program has required thus far has also taken more time than the program or contractors anticipated. The program’s standing review board has stated that the contract is structured to allow NASA unprecedented levels of visibility, but that it was intended to be used primarily for high-risk areas. However, the standing review board found, and both contractors told us, that the program has requested high levels of visibility on most items and there are signs that the contractors’ patience is waning. Both contractors expressed concerns that the program requests more interaction and data than they originally anticipated at the time of the contract award. For example, Boeing and SpaceX officials told us that the program often requests additional in-person engagement with their engineers, such as repeat presentations to multiple boards on the same technical issue, and has also asked for the same data in multiple formats or from multiple stakeholders. Program officials told us that they are constantly working to find a balance between obtaining the visibility they need to be able to eventually certify the crew transportation systems for human spaceflight while giving the contractors room to independently work through issues for their systems.

As the Commercial Crew Program progresses, the program office could also face difficult choices about how to maintain the level of visibility into contractor efforts it feels it needs without adding to the program’s schedule pressures. Independent review bodies, including the standing review board, the Aerospace Safety Advisory Panel, and the NASA Advisory Council-Human Exploration and Operations committee, expressed concern that the program may not have the capacity to sustain the level of visibility it has had to date and still meet the current certification schedule. The early upfront investment in time may ultimately make the certification process go smoother, but finding the right balance of visibility, and recognizing that a high level of visibility takes time and may impact the schedule, will be especially important as the contractors approach final certification when the government will need to determine if the systems are safe enough for human spaceflight.

Program Office Workload Is an Emerging Schedule Risk

Program officials told us that one of their greatest upcoming challenges will be to keep pace with the contractors’ schedules so that the program does not delay certification. Specifically, they told us they are concerned about an upcoming “bow wave” of work because the program must complete two oversight activities—phased safety reviews and verification closure notices—concurrently in order to support the contractors’ ISS design certification reviews, uncrewed and crewed flight test missions, and final certification.

The Commercial Crew Program is working to complete its three-phased safety review, which will ensure that the contractors have identified all safety-critical hazards and implemented associated controls, but it is behind schedule. Both the contractors and the program have contributed to these delays. In phase one, Boeing and SpaceX identified risks in their designs and developed reports on potential hazards, the controls they put in place to mitigate them, and explanations for how the controls will mitigate the hazards. In phase two, which is ongoing, the program reviews and approves the contractors’ hazard reports, and develops strategies to verify and validate that the controls are effective. In phase three, the contractors will conduct the verification activities and incrementally close the reports.

The Commercial Crew Program’s review and approval of the contractors’ hazard reports have taken longer than planned. The program originally planned to complete phase two in early 2016 but currently does not expect to complete this phase until June 2017. The Commercial Crew Program has a goal of reviewing hazard reports within 8 weeks of receiving them, but a recent report by the NASA Office of Inspector General found that the reviews are taking longer than anticipated and a backlog has developed. In response to the Inspector General’s report, NASA officials noted that, while the timeliness of the hazard review process is important, what is more important is having thorough, detailed hazard reports in order to understand safety risks and ensuring the safety of each system. As of October 2016, the Commercial Crew Program had approved 117 of the anticipated 195 hazard reports and planned to approve approximately half of the remaining reports for both contractors by the end of 2016. Program officials told us that the hazard reports that are still open are related to items that they would not expect to be closed because they involve some of the more complicated design work that the contractors have not yet finalized. Program officials also pointed out other ways that the contractors have contributed to phase two delays, including receiving incomplete hazard reports that required several iterations before the program could begin its formal review.

The Commercial Crew Program’s verification closure notice process, which is used to verify that the contractors have met all requirements, is one of the other key oversight activities and potential workload challenges for the program. The program is completing that process concurrently with the phased safety reviews. The verification closure process is initiated by the contractor when it provides the program with data and evidence to substantiate that it has met each requirement, and is completed when the program has reviewed and approved the contractor’s evidence to verify that each requirement has been met. A verification closure notice is required for each of the 332 ISS requirements, which are applicable to anyone who docks with the station, and the 280 Commercial Crew Program requirements. The Commercial Crew Program must also approve a subset of verification closure notices before key tests or milestones can occur. For example, the ISS requirements must be met before SpaceX and Boeing’s uncrewed flights to the ISS, which are currently planned for November 2017 and June 2018, respectively. The program’s ability to smooth its workload is limited because the contractors generally control their development schedules. Proposed changes to the Boeing and SpaceX schedules could help alleviate some of the concurrency between the program’s phased safety reviews and verification closure process.

Conclusions

NASA’s Commercial Crew Program is a multibillion dollar effort intended to end the United States’ reliance on Russia to maintain an uninterrupted presence on the ISS. To do so, NASA embarked on a different way of doing business. It awarded firm-fixed-price contracts to Boeing and SpaceX that transferred the financial risks to the contractors while giving them freedom to develop unique transportation systems that meet NASA’s standards. An independent review group initially raised concerns about how well this model would work for a human spaceflight program. Parts of the new business model have worked relatively well and NASA has had the visibility it needs into the technical details and risks of each system. Gaining this level of visibility has taken more time than the program or contractors anticipated, but the early upfront investment in time should ultimately make the certification process go smoother. In addition, while the Commercial Crew Program should be mindful about placing undue burdens on its contractors, it ultimately has the responsibility for ensuring the safety of U.S. astronauts, and its contracts with Boeing and SpaceX give it deference to determine the level of insight required to do so.

When the current phase of the Commercial Crew Program began, there was widespread acknowledgment that the contractors’ development and certification schedules were aggressive, and the anticipated schedule risks have now materialized. Both contractors’ certification dates have been delayed into 2018, and the program’s analysis indicates that neither contractor is likely to be ready until 2019. NASA has purchased seats for U.S. astronauts on the Russian Soyuz vehicle, so that it can maintain an uninterrupted presence on the ISS. But those seats will run out in spring 2019, at the latest, and it generally takes 3 years for NASA and Russia to negotiate the purchase of additional seats. If delays on the Commercial Crew Program persist, having contingency plans, including options to expedite the purchase of additional Soyuz seats, will become increasingly important to ensure an uninterrupted U.S. presence on the ISS. Without a viable contingency plan that has been communicated to Congress, NASA puts at risk its ability to maximize the utility of the ISS and the return on the multibillion dollar investment it has made in the space station.

Recommendation for Executive Action

In order to ensure that the United States has continued access to the ISS if the Commercial Crew Program’s contractors experience additional schedule delays, we recommend the NASA Administrator develop a contingency plan for maintaining a presence on the ISS beyond 2018, including options to purchase additional Russian Soyuz seats, and report to Congress on the results.

Agency Comments and Our Evaluation

We provided a draft of this report to NASA for comment. In written comments, NASA agreed with our recommendation and intends to develop a contingency plan by mid-March 2017. These comments are reproduced in Appendix III. NASA also provided technical comments, which have been addressed in the report, as appropriate.

We are sending copies of the report to the NASA Administrator and interested congressional committees. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202-512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in Appendix IV.

Appendix I: Objectives, Scope, and Methodology

The objectives of our review were to assess (1) the extent to which the contractors have made progress towards certification and the potential effects of any certification delays on the National Aeronautics and Space Administration’s (NASA) access to the International Space Station (ISS); (2) the major programmatic and safety risks facing the program and the contractors; and (3) the extent to which the program has visibility into the contractors’ efforts.

To assess the contractors’ progress towards certification, we obtained and reviewed program and contractor documents, including monthly and quarterly updates as well as monthly schedule summaries, from March 2016 through December 2016. We interviewed program and contractor officials to discuss the contractors’ recent progress as well as their upcoming events and any expected delays. To identify total delays to date, we compared original contract schedules to Boeing’s October 2016 working schedule and SpaceX’s November 2016 working schedule, which identify their most recent proposed delays to some milestones. To identify whether the contractors have introduced compression into their schedules, we judgmentally selected four key events and analyzed the movement of these key events relative to each other. Based on our review of program and contractor documents, we selected the contract’s acceptance events—the ISS design certification review and final certification review—as well as the uncrewed and crewed flight tests to conduct this analysis. We selected the acceptance events, as these occur when NASA approves a contractor’s designs and acknowledges that the contractor’s work is complete and meets the requirements of the contract. We selected the two flight tests for each contractor, as they are intended to test key system capabilities including the ability to launch, dock with the ISS, and return safely to Earth. Finally, to assess the potential effects of any certification delays on NASA’s access to ISS, we reviewed information from NASA’s budget, which includes the planned ISS launch manifest, and its contract with the Russian Federal Space Agency for transportation on the Soyuz vehicle. We also obtained information from the ISS program and the NASA Associate Administrator to determine if the agency had developed contingency plans to mitigate the effects of any certification delays on its access to the ISS.

To assess the programmatic and safety risks for the Commercial Crew Program and its contractors, we obtained and reviewed monthly and quarterly reports, as well as the risks tracked in both the program’s and the contractors’ risk management systems, from March 2016 through December 2016. We interviewed program and contractor officials with knowledge of the technical risks to understand the risks and potential impacts and how they are planning to mitigate those risks.

To assess how the program obtains visibility into its contractors’ systems and efforts, we reviewed the program’s contracts with its two contractors. In particular, we reviewed the contract “insight” clause, which outlines three mechanisms available to the program for obtaining visibility. We interviewed program officials in order to understand the three mechanisms and how they are used to gain visibility. We also interviewed officials from both contractors to gain their perspectives on the insight mechanisms. We also analyzed how NASA obtained visibility into contractor efforts at key events. We defined key events as those that are necessary for final certification. We selected three key events to examine for each contractor. We chose one historical event, so we could assess how the program had gained visibility over an entire event, and one upcoming event in both 2016 and 2017, in order to understand how the program obtains visibility in advance of an event. For each event, we obtained detailed information on how NASA obtained visibility and analyzed supporting documentation for these events to corroborate the information we obtained from NASA officials. In this report, we used these events as examples when describing how the program uses its three insight mechanisms and when describing the program’s relationships with its contractors. We also interviewed the program and contracting officials to obtain their perspectives on the level of visibility that the program has received into the contractors’ systems thus far. Finally, we analyzed program documentation and interviewed program officials to understand the level of work that NASA needs to complete leading up to certification. We assessed the timing of NASA’s reviews and spoke with program officials to determine if the program’s workload could be affected by the contractors’ schedules.

For all three objectives, we met with officials from all three organizations that provide NASA with independent assessments of the program: the program’s standing review board, the NASA Advisory Council-Human Exploration and Operations subcommittee, and the Aerospace Safety Advisory Panel. We spoke with these three groups to gain their perspectives on the program’s oversight of each contractor’s technical risks and schedules as well as the program’s level of visibility into the contractors’ systems.

We conducted this performance audit from March 2016 to February 2017 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Commercial Crew Program Phases and Participants

The Commercial Crew Program is a multi-phased effort that began in 2010. Table 3 describes each phase, its purpose, the participants, and the level of funding provided by the National Aeronautics and Space Administration.

Appendix III: Comments from the National Aeronautics and Space Administration

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Molly Traci, Assistant Director; Ron Schwenn, Assistant Director; Carly Gerbig; Laura Greifner; Kurt Gurka; Michael Kaeser; Katherine Lenane; Roxanna T. Sun; and Kristin Van Wychen made significant contributions to this report.